www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Memory safety depends entirely on GC ?

reply "FrankLike" <1150015857 qq.com> writes:
Now,some people think D is a 'Memory safety depends entirely on 
GC' system Language,what do you think?
Feb 21 2015
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Sat, 21 Feb 2015 10:00:06 +0000, FrankLike wrote:

 Now,some people think D is a 'Memory safety depends entirely on GC'
 system Language,what do you think?
this is wrong.=
Feb 21 2015
prev sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 21 February 2015 at 10:00:07 UTC, FrankLike wrote:
 Now,some people think D is a 'Memory safety depends entirely on 
 GC' system Language,what do you think?
It's kind of right, at the moment, since nogc D is still quite difficult to use, mostly due to exceptions and library artifacts. Both of these can be solved though.
Feb 21 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 21 February 2015 at 18:06:57 UTC, Peter Alexander 
wrote:
 On Saturday, 21 February 2015 at 10:00:07 UTC, FrankLike wrote:
 Now,some people think D is a 'Memory safety depends entirely 
 on GC' system Language,what do you think?
It's kind of right, at the moment, since nogc D is still quite difficult to use, mostly due to exceptions and library artifacts. Both of these can be solved though.
That wouldn't make nogc safe in any way.
Feb 21 2015
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 21 February 2015 at 18:42:54 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 18:06:57 UTC, Peter Alexander 
 wrote:
 On Saturday, 21 February 2015 at 10:00:07 UTC, FrankLike wrote:
 Now,some people think D is a 'Memory safety depends entirely 
 on GC' system Language,what do you think?
It's kind of right, at the moment, since nogc D is still quite difficult to use, mostly due to exceptions and library artifacts. Both of these can be solved though.
That wouldn't make nogc safe in any way.
safe nogc :-) (I rewrote the post a few times. Originally I just wrote "mark main safe nogc and you're fine", but I think it's a bit misleading since nogc is still difficult to use, so I wrote about that instead and forgot to mention safe at all. Thanks for pointing out.)
Feb 21 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 21 February 2015 at 19:38:02 UTC, Peter Alexander 
wrote:
  safe  nogc

 :-)

 (I rewrote the post a few times. Originally I just wrote "mark 
 main  safe  nogc and you're fine", but I think it's a bit 
 misleading since  nogc is still difficult to use, so I wrote 
 about that instead and forgot to mention  safe at all. Thanks 
 for pointing out.)
free is an unsafe operation. Unless you don't allocate at all or choose to leak everything, you won't be able to be safe and nogc. The only way out that I know of is an ownership system.
Feb 21 2015
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 21 February 2015 at 20:13:26 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 19:38:02 UTC, Peter Alexander 
 wrote:
  safe  nogc

 :-)

 (I rewrote the post a few times. Originally I just wrote "mark 
 main  safe  nogc and you're fine", but I think it's a bit 
 misleading since  nogc is still difficult to use, so I wrote 
 about that instead and forgot to mention  safe at all. Thanks 
 for pointing out.)
free is an unsafe operation. Unless you don't allocate at all or choose to leak everything, you won't be able to be safe and nogc. The only way out that I know of is an ownership system.
malloc+free can be trusted if wrapped in something like a ref counted pointer, no?
Feb 21 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/21/15 2:13 PM, Peter Alexander wrote:
 On Saturday, 21 February 2015 at 20:13:26 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 19:38:02 UTC, Peter Alexander wrote:
  safe  nogc

 :-)

 (I rewrote the post a few times. Originally I just wrote "mark main
  safe  nogc and you're fine", but I think it's a bit misleading since
  nogc is still difficult to use, so I wrote about that instead and
 forgot to mention  safe at all. Thanks for pointing out.)
free is an unsafe operation. Unless you don't allocate at all or choose to leak everything, you won't be able to be safe and nogc. The only way out that I know of is an ownership system.
malloc+free can be trusted if wrapped in something like a ref counted pointer, no?
There's more to it, e.g. access to the underlying raw pointer must be carefully restricted. -- Andrei
Feb 21 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 21 February 2015 at 22:13:09 UTC, Peter Alexander 
wrote:
 malloc+free can be trusted if wrapped in something like a ref 
 counted pointer, no?
Foo bazoom; class Foo { void bar() { bazoom = this; } } void foo() { RefCounted!Foo f = ... f.bar(); // bazoom is now a dandling pointer. }
Feb 21 2015
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter Alexander 
 wrote:
 malloc+free can be trusted if wrapped in something like a ref 
 counted pointer, no?
Foo bazoom; class Foo { void bar() { bazoom = this; } } void foo() { RefCounted!Foo f = ... f.bar(); // bazoom is now a dandling pointer. }
I see, thanks. Is assigning 'this' from a member function the only problem case?
Feb 22 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 22 February 2015 at 14:41:43 UTC, Peter Alexander 
wrote:
 On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter Alexander 
 wrote:
 malloc+free can be trusted if wrapped in something like a ref 
 counted pointer, no?
Foo bazoom; class Foo { void bar() { bazoom = this; } } void foo() { RefCounted!Foo f = ... f.bar(); // bazoom is now a dandling pointer. }
I see, thanks. Is assigning 'this' from a member function the only problem case?
No. There's also returning the reference from a member function, storing it in a passed-in reference (pointer, ref, out or slice), and passing it to other functions that in turn leak the reference, as well as throwing it. And leaking closures containing the reference. That's all that I can think of now...
Feb 22 2015
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 22 February 2015 at 14:49:37 UTC, Marc Schütz wrote:
 On Sunday, 22 February 2015 at 14:41:43 UTC, Peter Alexander 
 wrote:
 On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter 
 Alexander wrote:
 malloc+free can be trusted if wrapped in something like a 
 ref counted pointer, no?
Foo bazoom; class Foo { void bar() { bazoom = this; } } void foo() { RefCounted!Foo f = ... f.bar(); // bazoom is now a dandling pointer. }
I see, thanks. Is assigning 'this' from a member function the only problem case?
No. There's also returning the reference from a member function, storing it in a passed-in reference (pointer, ref, out or slice), and passing it to other functions that in turn leak the reference, as well as throwing it. And leaking closures containing the reference. That's all that I can think of now...
Sorry, I meant things other than moving this from a member function to somewhere else, i.e. the RefCounted shouldn't leak the pointer itself in any other way. Just wondering if there are ways to avoid the problem by making 'this' unescapable in certain situations.
Feb 22 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 22 February 2015 at 14:54:26 UTC, Peter Alexander 
wrote:
 On Sunday, 22 February 2015 at 14:49:37 UTC, Marc Schütz wrote:
 On Sunday, 22 February 2015 at 14:41:43 UTC, Peter Alexander 
 wrote:
 On Sunday, 22 February 2015 at 04:19:32 UTC, deadalnix wrote:
 On Saturday, 21 February 2015 at 22:13:09 UTC, Peter 
 Alexander wrote:
 malloc+free can be trusted if wrapped in something like a 
 ref counted pointer, no?
Foo bazoom; class Foo { void bar() { bazoom = this; } } void foo() { RefCounted!Foo f = ... f.bar(); // bazoom is now a dandling pointer. }
I see, thanks. Is assigning 'this' from a member function the only problem case?
No. There's also returning the reference from a member function, storing it in a passed-in reference (pointer, ref, out or slice), and passing it to other functions that in turn leak the reference, as well as throwing it. And leaking closures containing the reference. That's all that I can think of now...
Sorry, I meant things other than moving this from a member function to somewhere else, i.e. the RefCounted shouldn't leak the pointer itself in any other way. Just wondering if there are ways to avoid the problem by making 'this' unescapable in certain situations.
Ah, ok. But escaping the reference is unavoidable if you don't want to pass the RC object around (and you don't, because it's expensive). And the only safe way to escape it is an ownership system.
Feb 22 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 No. There's also returning the reference from a member function, storing
 it in a passed-in reference (pointer, ref, out or slice), and passing it
 to other functions that in turn leak the reference, as well as throwing
 it. And leaking closures containing the reference.

 That's all that I can think of now...
Consider class C { ... client code ... } alias T = RefCounted!C; ... more client code ... For reference counting to work transparently, access to the symbol "C" must be restricted. RefCounted obviously needs access to it. Client code should never have access to it, even in the definition of C. That means: 1. client code must not be able to declare variables of type C or issue calls like "new C" etc. 2. The type of "this" in methods of C must be RefCounted!C, not C. 3. Conversions of C to bases of C and interfaces must be forbidden; only their RefCounted!Base versions must be allowed. 4. Returning references to direct members of C must be restricted the same way they are for structs (see http://wiki.dlang.org/DIP25). A GC class object does not have that restriction. I think reference counting is an important component of a complete solution to resource management. D should implement world-class reference counting for safe code. For 1-4 above, although I am a staunch supporter of library-exclusive abstractions, I have reached the conclusion there is no way to implement RC in safe code for D classes without changes to the language. The more we realize that as a community the quicker we can move to effect it. Andrei
Feb 22 2015
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu 
wrote:
 Consider

 class C { ... client code ... }
 alias T = RefCounted!C;
 ... more client code ...

 For reference counting to work transparently, access to the 
 symbol "C" must be restricted. RefCounted obviously needs 
 access to it. Client code should never have access to it, even 
 in the definition of C.
What ??? That mean writing all library code twice, for client that want GC and for these who don't. That is a looser strategy.
 That means:

 1. client code must not be able to declare variables of type C 
 or issue calls like "new C" etc.

 2. The type of "this" in methods of C must be RefCounted!C, not 
 C.

 3. Conversions of C to bases of C and interfaces must be 
 forbidden; only their RefCounted!Base versions must be allowed.

 4. Returning references to direct members of C must be 
 restricted the same way they are for structs (see 
 http://wiki.dlang.org/DIP25). A GC class object does not have 
 that restriction.

 I think reference counting is an important component of a 
 complete solution to resource management. D should implement 
 world-class reference counting for safe code.
Sounds like a world class RC would not force all the code to be written twice.
 For 1-4 above, although I am a staunch supporter of 
 library-exclusive abstractions, I have reached the conclusion 
 there is no way to implement RC in safe code for D classes 
 without changes to the language. The more we realize that as a 
 community the quicker we can move to effect it.
I don't think we want to implement RC in the language, but implement what would allow to have safe RC as library.
Feb 22 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/15 10:26 AM, deadalnix wrote:
 On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu wrote:
 Consider

 class C { ... client code ... }
 alias T = RefCounted!C;
 ... more client code ...

 For reference counting to work transparently, access to the symbol "C"
 must be restricted. RefCounted obviously needs access to it. Client
 code should never have access to it, even in the definition of C.
What ??? That mean writing all library code twice, for client that want GC and for these who don't.
I'm not 100% convinced but it seems to me RC vs. GC is a class design time decision.
 That is a looser strategy.
I'm sure there are tighter ones :o). Andrei
Feb 22 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 22 February 2015 at 20:48:58 UTC, Andrei Alexandrescu 
wrote:
 What ??? That mean writing all library code twice, for client 
 that want
 GC and for these who don't.
I'm not 100% convinced but it seems to me RC vs. GC is a class design time decision.
The right strategy for memory management depend on the usage you'll do of an object, not of the object itself. Or, in other terms, the client code know more about the adapted memory management. Complex library solution probably need to adopt some strategy, but for most library (especially for something like phobos), it make sense to offload that choice on the user.
 That is a looser strategy.
I'm sure there are tighter ones :o).
Padam tshhhhhh !
Feb 22 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/22/15 8:50 PM, deadalnix wrote:
 On Sunday, 22 February 2015 at 20:48:58 UTC, Andrei Alexandrescu wrote:
 What ??? That mean writing all library code twice, for client that want
 GC and for these who don't.
I'm not 100% convinced but it seems to me RC vs. GC is a class design time decision.
The right strategy for memory management depend on the usage you'll do of an object, not of the object itself.
I used to think the same. But then I considered typechecking things like: class Widget { private char name[1024]; char[] getName() { return name[]; } ... } Such code is safe if Widget is a GC class but not if it uses reference counting. Andrei
Feb 23 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 23 February 2015 at 17:48:27 UTC, Andrei Alexandrescu 
wrote:
 I used to think the same. But then I considered typechecking 
 things like:

 class Widget
 {
     private char name[1024];
     char[] getName() { return name[]; }
     ...
 }

 Such code is safe if Widget is a GC class but not if it uses 
 reference counting.
This is not a valid argument against my point, simply expressing difficulties of implementation and lack of imagination of what this implementation can be. I'm not sure how to convey the idea anymore. That is an ownership issue and solutions exist for it. Several of them have been proposed already. It feels like we are gonna add a bazillion of stupid hacks to work around not introducing ownership into D. The return ref thing is one of them. What you are suggesting for refcounting is another.
Feb 23 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu 
wrote:
 On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 No. There's also returning the reference from a member 
 function, storing
 it in a passed-in reference (pointer, ref, out or slice), and 
 passing it
 to other functions that in turn leak the reference, as well as 
 throwing
 it. And leaking closures containing the reference.

 That's all that I can think of now...
Consider class C { ... client code ... } alias T = RefCounted!C; ... more client code ... For reference counting to work transparently, access to the symbol "C" must be restricted. RefCounted obviously needs access to it. Client code should never have access to it, even in the definition of C. That means: 1. client code must not be able to declare variables of type C or issue calls like "new C" etc.
No, this would require the class to be specialized for refcounting. But the memory management method needs to be the client code's decision; it can decide to manage some instance by refcounting, some by Unique!C, and leave others to the GC. The class implementer shouldn't need to care about all that.
 2. The type of "this" in methods of C must be RefCounted!C, not 
 C.

 3. Conversions of C to bases of C and interfaces must be 
 forbidden; only their RefCounted!Base versions must be allowed.
These two points have undesirable consequences: All consumers such objects need to be aware of the exact type, which includes the management strategy (RC, Unique, GC). But this is a violation of the principle of separation of concerns: a consumer shouldn't need to have information about the management strategy, it should work equally with `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it doesn't take ownership of the resource.
 4. Returning references to direct members of C must be 
 restricted the same way they are for structs (see 
 http://wiki.dlang.org/DIP25). A GC class object does not have 
 that restriction.
This is only a partial solution that doesn't work efficiently with anything other then value members. Slices, pointers and classes require introducing an additional, useless indirection, and that's not the only problem with it.
 I think reference counting is an important component of a 
 complete solution to resource management. D should implement 
 world-class reference counting for safe code.

 For 1-4 above, although I am a staunch supporter of 
 library-exclusive abstractions, I have reached the conclusion 
 there is no way to implement RC in safe code for D classes 
 without changes to the language. The more we realize that as a 
 community the quicker we can move to effect it.
I'm not sure this is what you're implying, but do you want to restrict it to classes only? Why not structs and slices, too? Of course I agree that a language change is necessary, but I'm convinced what you suggest above is not the right direction at all. (And I see that deadalnix has already replied and basically said the same thing.) In general, this and related proposals tend to limit themselves on memory management (as witnessed by the importance that `ref` and ` safe` play in them). This is too narrow IMO. A well thought-out solution can be equally applicable to the broader field of resource management.
Feb 23 2015
next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 23 February 2015 at 14:56:11 UTC, Marc Schütz wrote:
 violation of the principle of separation of concerns: a 
 consumer shouldn't need to have information about the 
 management strategy, it should work equally with 
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take ownership of the resource.
Just be aware that by not making the refcount part of C you either: 1. Need one more indirection. 2. Need a more complicated allocator to avoid alignment padding when C has 32byte/64 byte alignment requirements. 3. Risk having the refcount landing on a different cacheline, causing more cache misses. If you do excessive refcounting (ARC) and care about performance you actually need to let the implementor of C decide where the RC_counter is embedded...
Feb 23 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 23 February 2015 at 15:35:52 UTC, Ola Fosheim Grøstad 
wrote:
 On Monday, 23 February 2015 at 14:56:11 UTC, Marc Schütz wrote:
 violation of the principle of separation of concerns: a 
 consumer shouldn't need to have information about the 
 management strategy, it should work equally with 
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take ownership of the resource.
Just be aware that by not making the refcount part of C you either: 1. Need one more indirection. 2. Need a more complicated allocator to avoid alignment padding when C has 32byte/64 byte alignment requirements. 3. Risk having the refcount landing on a different cacheline, causing more cache misses.
The refcount would be embedded right next to the object by the constructor of `RefCounted!C`, which is responsible for allocating memory for it, using an appropriate allocator.
 If you do excessive refcounting (ARC) and care about 
 performance you actually need to let the implementor of C 
 decide where the RC_counter is embedded...
Of course there can be an additional (template) parameter to `RefCounted`, or it could be a different, user-implemented type altogether. But my point was that all of `Unique!C`, `RefCounted!C` and `C` should decay to `scope C`. This way, most consumers don't need to be aware of the actual wrapper type (if any) at all.
Feb 23 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 23 February 2015 at 15:35:52 UTC, Ola Fosheim Grøstad 
wrote:
 If you do excessive refcounting (ARC) and care about 
 performance you actually need to let the implementor of C 
 decide where the RC_counter is embedded...
C providing facility to be refcountable do not equate with C's user want refcounting. It means that if C's user want it to be refcounted, it is gonna be faster.
Feb 23 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 On Sunday, 22 February 2015 at 17:01:45 UTC, Andrei Alexandrescu wrote:
 On 2/22/15 6:49 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>"
 wrote:
 No. There's also returning the reference from a member function, storing
 it in a passed-in reference (pointer, ref, out or slice), and passing it
 to other functions that in turn leak the reference, as well as throwing
 it. And leaking closures containing the reference.

 That's all that I can think of now...
Consider class C { ... client code ... } alias T = RefCounted!C; ... more client code ... For reference counting to work transparently, access to the symbol "C" must be restricted. RefCounted obviously needs access to it. Client code should never have access to it, even in the definition of C. That means: 1. client code must not be able to declare variables of type C or issue calls like "new C" etc.
No, this would require the class to be specialized for refcounting. But the memory management method needs to be the client code's decision; it can decide to manage some instance by refcounting, some by Unique!C, and leave others to the GC. The class implementer shouldn't need to care about all that.
That's not feasible. Code that assumes the class object will live forever can simply do things that are not allowed to code that must assume the object will go away at some determined point. Consider this which I just posted: class Widget { private char name[1024]; char[] getName() { return name[]; } ... } The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know whether getName() is okay or not.
 2. The type of "this" in methods of C must be RefCounted!C, not C.

 3. Conversions of C to bases of C and interfaces must be forbidden;
 only their RefCounted!Base versions must be allowed.
These two points have undesirable consequences: All consumers such objects need to be aware of the exact type, which includes the management strategy (RC, Unique, GC). But this is a violation of the principle of separation of concerns: a consumer shouldn't need to have information about the management strategy, it should work equally with `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it doesn't take ownership of the resource.
Well I don't know of another way.
 4. Returning references to direct members of C must be restricted the
 same way they are for structs (see http://wiki.dlang.org/DIP25). A GC
 class object does not have that restriction.
This is only a partial solution that doesn't work efficiently with anything other then value members. Slices, pointers and classes require introducing an additional, useless indirection, and that's not the only problem with it.
So what do you propose instead? I am well aware of the disadvantages of a solution that _works_. Do you have something better?
 I think reference counting is an important component of a complete
 solution to resource management. D should implement world-class
 reference counting for safe code.

 For 1-4 above, although I am a staunch supporter of library-exclusive
 abstractions, I have reached the conclusion there is no way to
 implement RC in safe code for D classes without changes to the
 language. The more we realize that as a community the quicker we can
 move to effect it.
I'm not sure this is what you're implying, but do you want to restrict it to classes only? Why not structs and slices, too?
Structs and slices can be implemented safe-ly after the advent of DIP25. It's classes that are the troublemaker.
 Of course I agree that a language change is necessary, but I'm convinced
 what you suggest above is not the right direction at all. (And I see
 that deadalnix has already replied and basically said the same thing.)
That does awfully little to help. I'm telling things as they are. It's difficult to disagree with e.g. "a refcounted class cannot be implicitly convertible to Object and stay refcounted".
 In general, this and related proposals tend to limit themselves on
 memory management (as witnessed by the importance that `ref` and ` safe`
 play in them). This is too narrow IMO. A well thought-out solution can
 be equally applicable to the broader field of resource management.
Looking forward to your insights. Andrei
Feb 23 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu 
wrote:
 The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT 
 know whether getName() is okay or not.
Aye, I haven't been following this thread closely, but I thought of this case a while ago myself and it actually led me to the belief that the this pointer needs to be scope unless the class itself is designed solely for GC use; escaping anything through it must not be allowed for guaranteed memory safety if it is manually freed in any form, whether refcounting, RAII, or free. If this is scope, then the usage site has freedom of deallocation method. If not, it must be known at design time of the class itself.
Feb 23 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 10:27 AM, Adam D. Ruppe wrote:
 On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu wrote:
 The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT know
 whether getName() is okay or not.
Aye, I haven't been following this thread closely, but I thought of this case a while ago myself and it actually led me to the belief that the this pointer needs to be scope unless the class itself is designed solely for GC use; escaping anything through it must not be allowed for guaranteed memory safety if it is manually freed in any form, whether refcounting, RAII, or free. If this is scope, then the usage site has freedom of deallocation method. If not, it must be known at design time of the class itself.
That's exactly right. There should be a DConf talk about that, apparently it's a widely misunderstood topic. -- Andrei
Feb 23 2015
prev sibling next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
Just brainstorming:

1. Move RC!T into the runtime because it can get special compiler 
attention.
2. Duplicate the const/mutable/immutable machinery for 
Scope/RC/GC classes ( / types).
3. RC!T is not struct RC { size_t cntr; T payload } but a T*. The 
allocator makes sure that the reference count is at T*-8 (or at 
T*+RCOffset!T or whatever).
4. RC!T and T (the GC managed type) implicitly convert to 
Scope!T. Scope!T makes sure that no reference escapes in an 
unsafe way (using the foundations in dip25).

Because of 3. we can have polymorphic functions that either 
return a GC allocated or an RC allocated class. It's type depends 
of the type of this, which leads to the equivalent of inout, 
let's call it  rcgc.
Feb 23 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu 
wrote:
 That's not feasible. Code that assumes the class object will 
 live forever can simply do things that are not allowed to code 
 that must assume the object will go away at some determined 
 point. Consider this which I just posted:

 class Widget
 {
     private char name[1024];
     char[] getName() { return name[]; }
     ...
 }

 The typechecker must WHILE COMPILING WIDGET, NOT ITS CLIENT 
 know whether getName() is okay or not.
Let's use that exemple.
 Well I don't know of another way.
I'm not sure if this is dishonest, but proposal has been made in this newgroup, by various person,s including myself. Let's get around the major concept. name is owned by the widget instance (let's call it w). That means its lifetime is the same as w's lifetime. w's owner will provide its lifetime. If w's owner is the GC, then its lifetime is infinite. if w's owner is some refounting system, its lifetime is defined by when the refcounting system will destroy it. getName here assumes the lifetime of the object is infinite. That's ok. It means you won't be able to call it when it is owned by a RC system. If you want to be able to do so, you'll ned have a way to specify the lifetime of the returned slice. For instance : class Widget { private char name[1024]; scope(char[]) getName() scope { return name[]; } ... } In this case, you can call the method when w is owned by a RC system, as it is now explicit to the caller that the lifetime of what is returned and the compiler can ensure the slice do not exceed its lifetime (ie w's lifetime). This require to be able to express lifetime and preferably ownership as well. So far, what I feel from you and Walter is the will to not go into that direction and instead created a myriad of hacks for various special cases. I do think this is wrong headed and generally not the way forward. Languages should provide the most generic and powerful tool so that interesting schemes can be implemented on top of it.
Feb 23 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 5:23 PM, deadalnix wrote:
 So far, what I feel from you and Walter is the will to not go into that
 direction and instead created a myriad of hacks for various special
 cases. I do think this is wrong headed and generally not the way
 forward. Languages should provide the most generic and powerful tool so
 that interesting schemes can be implemented on top of it.
This is a free world. Walter and I are working on a DIP. Please work on yours. I can't promise we'll choose to your liking, but this is the one way you can make your point heard and understood. What doesn't work is trash talk. I guarantee I recognize brilliance when I see it. So if you have a brilliant idea, it won't be missed. Have at it. One thing I cannot do is choose a solution that you prefer over one I prefer - this does remain a subjective topic. I can't help it. But please don't consider me an idiot because I don't like what you propose. Thanks, Andrei
Feb 23 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 5:43 PM, Andrei Alexandrescu wrote:
 But please don't consider me an idiot because I don't like what you
 propose.
Hmmm... actually I take that back :o). You are totally free to consider me such (or at least of intellect inferior to yours etc). What I mean is please don't consider me somehow socially or morally obligated to choose your solution just because you like it so much more than mine. Andrei
Feb 23 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 01:43:11 UTC, Andrei Alexandrescu 
wrote:
 This is a free world. Walter and I are working on a DIP. Please 
 work on yours. I can't promise we'll choose to your liking, but 
 this is the one way you can make your point heard and 
 understood. What doesn't work is trash talk.

 I guarantee I recognize brilliance when I see it. So if you 
 have a brilliant idea, it won't be missed. Have at it. One 
 thing I cannot do is choose a solution that you prefer over one 
 I prefer - this does remain a subjective topic. I can't help 
 it. But please don't consider me an idiot because I don't like 
 what you propose.
I don't think you are being fair here. Even if not formally expressed as a DIP, at least Mark and myself have come up with fairly detailed explanations, in topic you participated in, so we can't really do as if it didn't existed. Also, I do not think this is a subjective matter. Yes there is a part of it that is matter of taste and is subjective, but overall there is a big chunk of objectively discussable things is there proposal, like language complexity and expressiveness added to the language. But here is mostly what I think is going on. We are discussion various issues, including make the GC faster, enable safe RC, make nogc more usable (for instance exception usability), safe ref, enforcing type qualifier contraints, and so on... For each of these issues, solution are proposed. What I (and I think Mark would agree) propose would solve them all. Yes this is more complex than any of the solution proposed for each of these. But this is way simpler, and enable way more than having a unique, simpler solution for each of these problems.
Feb 23 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 24 February 2015 at 02:35:41 UTC, deadalnix wrote:
 For each of these issues, solution are proposed. What I (and I 
 think Mark would agree) propose would solve them all. Yes this 
 is more complex than any of the solution proposed for each of 
 these. But this is way simpler, and enable way more than having 
 a unique, simpler solution for each of these problems.
Indeed I agree. But judging by Andrei's response above, I'm longer so sure he is actually aware of what we proposed.
Feb 24 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/23/15 6:35 PM, deadalnix wrote:
 On Tuesday, 24 February 2015 at 01:43:11 UTC, Andrei Alexandrescu wrote:
 This is a free world. Walter and I are working on a DIP. Please work
 on yours. I can't promise we'll choose to your liking, but this is the
 one way you can make your point heard and understood. What doesn't
 work is trash talk.

 I guarantee I recognize brilliance when I see it. So if you have a
 brilliant idea, it won't be missed. Have at it. One thing I cannot do
 is choose a solution that you prefer over one I prefer - this does
 remain a subjective topic. I can't help it. But please don't consider
 me an idiot because I don't like what you propose.
I don't think you are being fair here. Even if not formally expressed as a DIP, at least Mark and myself have come up with fairly detailed explanations, in topic you participated in, so we can't really do as if it didn't existed. Also, I do not think this is a subjective matter. Yes there is a part of it that is matter of taste and is subjective, but overall there is a big chunk of objectively discussable things is there proposal, like language complexity and expressiveness added to the language. But here is mostly what I think is going on. We are discussion various issues, including make the GC faster, enable safe RC, make nogc more usable (for instance exception usability), safe ref, enforcing type qualifier contraints, and so on... For each of these issues, solution are proposed. What I (and I think Mark would agree) propose would solve them all. Yes this is more complex than any of the solution proposed for each of these. But this is way simpler, and enable way more than having a unique, simpler solution for each of these problems.
Just replied to Marc about this. I should have phrased my response as "I don't know of a _reasonable_ solution". BTW please don't call me dishonest anymore, it's doubly inappropriate seeing as we also work together. You wouldn't think there's an actual possibility I go around lying to people about such stuff. Thanks. Andrei
Feb 24 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 16:22:47 UTC, Andrei Alexandrescu 
wrote:
 Just replied to Marc about this. I should have phrased my 
 response as "I don't know of a _reasonable_ solution". BTW 
 please don't call me dishonest anymore, it's doubly 
 inappropriate seeing as we also work together. You wouldn't 
 think there's an actual possibility I go around lying to people 
 about such stuff. Thanks.

 Andrei
You have my apologies. Still, qualifying something of _reasonable_ or not do not constitute a solid argument IMO. That is where my frustration come from here. You are arguing as if nothing was proposed when various persons made proposals. That do not foster progress. If actual criticism of the proposals were made, then we could argue about these criticisms, or come up with improvement of the proposals, but if you argue as if these proposal do not exists, we are condemned to talk past each other and no progress can be made.
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 9:47 AM, deadalnix wrote:
 On Tuesday, 24 February 2015 at 16:22:47 UTC, Andrei Alexandrescu wrote:
 Just replied to Marc about this. I should have phrased my response as
 "I don't know of a _reasonable_ solution". BTW please don't call me
 dishonest anymore, it's doubly inappropriate seeing as we also work
 together. You wouldn't think there's an actual possibility I go around
 lying to people about such stuff. Thanks.

 Andrei
You have my apologies. Still, qualifying something of _reasonable_ or not do not constitute a solid argument IMO. That is where my frustration come from here. You are arguing as if nothing was proposed when various persons made proposals. That do not foster progress. If actual criticism of the proposals were made, then we could argue about these criticisms, or come up with improvement of the proposals, but if you argue as if these proposal do not exists, we are condemned to talk past each other and no progress can be made.
I also owe you apologies for not acknowledging that work. I find the proposal too complicated for what it provides and that's the short and long of it. It's easy to make a large and complex language addition to support any sensible abstraction. That doesn't make it automatically good. Andrei
Feb 24 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2015 10:00 AM, Andrei Alexandrescu wrote:
 I also owe you apologies for not acknowledging that work. I find the proposal
 too complicated for what it provides and that's the short and long of it. It's
 easy to make a large and complex language addition to support any sensible
 abstraction. That doesn't make it automatically good.
My criticisms of it centered around: 1. confusion about whether it was a storage class or a type qualifier. 2. I agree with Andrei that any annotation system can be made to work - but this one (as are most annotation systems) also struck me as wordy, tedious, and aesthetically unappealing. I just can't see myself throwing it up on a slide and trying to sell it to the audience as cool. 3. In line with (2), I want a system that relies much more on inference. We've made good progress with the existing annotations being inferred. 4. I didn't see how one could, for example, have an array of pointers: int*[] pointers; and then fill that array with pointers of varying ownership annotations. 5. The (4) homogeneity requirement would mean that templated types would get new instantiations every time they are used with a different ownership. This could lead to massive code bloat. 6. The 'return ref' scheme, which you have expressed distaste for, was one that required the fewest instances of the user having to add an annotation. It turned out that upgrading Phobos to this required only a handful of annotations. 7. 'return ref' makes memory safe ref counted types possible, finally, in D, without needing to upend the language or legacy code. And as the example I posted showed, they are straightforward to write. Only time and experience will tell if this will be successful, but it looks promising and I hope you'll be willing to give it a chance.
Feb 24 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 24 February 2015 at 20:53:24 UTC, Walter Bright wrote:
 On 2/24/2015 10:00 AM, Andrei Alexandrescu wrote:
 I also owe you apologies for not acknowledging that work. I 
 find the proposal
 too complicated for what it provides and that's the short and 
 long of it. It's
 easy to make a large and complex language addition to support 
 any sensible
 abstraction. That doesn't make it automatically good.
My criticisms of it centered around: 1. confusion about whether it was a storage class or a type qualifier. 2. I agree with Andrei that any annotation system can be made to work - but this one (as are most annotation systems) also struck me as wordy, tedious, and aesthetically unappealing. I just can't see myself throwing it up on a slide and trying to sell it to the audience as cool. 3. In line with (2), I want a system that relies much more on inference. We've made good progress with the existing annotations being inferred. 4. I didn't see how one could, for example, have an array of pointers: int*[] pointers; and then fill that array with pointers of varying ownership annotations. 5. The (4) homogeneity requirement would mean that templated types would get new instantiations every time they are used with a different ownership. This could lead to massive code bloat. 6. The 'return ref' scheme, which you have expressed distaste for, was one that required the fewest instances of the user having to add an annotation. It turned out that upgrading Phobos to this required only a handful of annotations. 7. 'return ref' makes memory safe ref counted types possible, finally, in D, without needing to upend the language or legacy code. And as the example I posted showed, they are straightforward to write. Only time and experience will tell if this will be successful, but it looks promising and I hope you'll be willing to give it a chance.
Thanks for summarizing your reasons. I knew that you were unhappy with 1 - 3); I wasn't aware of 4) and 5). I can't get into detail now, as it's already late at night here, but I'll try thinking about it tomorrow. FWIW, Zach just wrote in another thread that he will have his own proposal ready soon, based on DIP25, with `scope` being a storage class. Let's see how this goes.
Feb 24 2015
prev sibling parent "Zach the Mystic" <reachzach gggmail.com> writes:
Here's my best so far:

http://forum.dlang.org/post/offurllmuxjewizxedab forum.dlang.org

On Tuesday, 24 February 2015 at 20:53:24 UTC, Walter Bright wrote:
 My criticisms of it centered around:

 1. confusion about whether it was a storage class or a type 
 qualifier.
My system has neither. Instead, it just bans unsafe reference copying in safe code.
 2. I agree with Andrei that any annotation system can be made 
 to work - but this one (as are most annotation systems) also 
 struck me as wordy, tedious, and aesthetically unappealing. I 
 just can't see myself throwing it up on a slide and trying to 
 sell it to the audience as cool.

 3. In line with (2), I want a system that relies much more on 
 inference. We've made good progress with the existing 
 annotations being inferred.
Well you know I'm on board with this. The one penalty my system requires is two more parameter attributes, which I'm hoping can be alleviated by inference as much as possible.
 4. I didn't see how one could, for example, have an array of 
 pointers:

     int*[] pointers;

 and then fill that array with pointers of varying ownership 
 annotations.

 5. The (4) homogeneity requirement would mean that templated 
 types would get new instantiations every time they are used 
 with a different ownership. This could lead to massive code 
 bloat.
I deliberately designed my system to avoid all associations with type. No code bloat.
 6. The 'return ref' scheme, which you have expressed distaste 
 for, was one that required the fewest instances of the user 
 having to add an annotation. It turned out that upgrading 
 Phobos to this required only a handful of annotations.

 7. 'return ref' makes memory safe ref counted types possible, 
 finally, in D, without needing to upend the language or legacy 
 code. And as the example I posted showed, they are 
 straightforward to write. Only time and experience will tell if 
 this will be successful, but it looks promising and I hope 
 you'll be willing to give it a chance.
I do give it a chance! See my proposal!
Feb 26 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 23 February 2015 at 18:16:38 UTC, Andrei Alexandrescu 
wrote:
 On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 These two points have undesirable consequences: All consumers 
 such
 objects need to be aware of the exact type, which includes the
 management strategy (RC, Unique, GC). But this is a violation 
 of the
 principle of separation of concerns: a consumer shouldn't need 
 to have
 information about the management strategy, it should work 
 equally with
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take
 ownership of the resource.
Well I don't know of another way.
Ok, I wrote my reply assuming that you are aware of the various proposals deadalnix, myself and several other people have made in the past, some of them quite specific. But now that I think of it, I don't remember that you were ever directly referring to it in any of your posts. Maybe you just missed it? As one example, here is what I originally suggested: http://wiki.dlang.org/User:Schuetzm/scope It's not completely up to date, during discussions I gained many useful new insights to simplify it and make things more consistent. It's also part of a bigger picture (deadalnix's ideas about ownership play an important role, too), which unfortunately isn't easy to recognize, because this page has become quite large und unwieldy. I should make a post explaining this.
 In general, this and related proposals tend to limit 
 themselves on
 memory management (as witnessed by the importance that `ref` 
 and ` safe`
 play in them). This is too narrow IMO. A well thought-out 
 solution can
 be equally applicable to the broader field of resource 
 management.
Looking forward to your insights.
See above. I had the impression that you were aware of the proposals, but for some reason were opposed to them. Maybe that's not the case?
Feb 24 2015
next sibling parent "Zach the Mystic" <reachzach gggmail.com> writes:
On Tuesday, 24 February 2015 at 12:44:54 UTC, Marc Schütz wrote:
 On Monday, 23 February 2015 at 18:16:38 UTC, Andrei 
 Alexandrescu wrote:
 On 2/23/15 6:56 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 These two points have undesirable consequences: All consumers 
 such
 objects need to be aware of the exact type, which includes the
 management strategy (RC, Unique, GC). But this is a violation 
 of the
 principle of separation of concerns: a consumer shouldn't 
 need to have
 information about the management strategy, it should work 
 equally with
 `RefCounted!C`, `Unique!C` and bare (GC) `C`, as long as it 
 doesn't take
 ownership of the resource.
Well I don't know of another way.
Ok, I wrote my reply assuming that you are aware of the various proposals deadalnix, myself and several other people have made in the past, some of them quite specific. But now that I think of it, I don't remember that you were ever directly referring to it in any of your posts. Maybe you just missed it? As one example, here is what I originally suggested: http://wiki.dlang.org/User:Schuetzm/scope It's not completely up to date, during discussions I gained many useful new insights to simplify it and make things more consistent. It's also part of a bigger picture (deadalnix's ideas about ownership play an important role, too), which unfortunately isn't easy to recognize, because this page has become quite large und unwieldy. I should make a post explaining this.
I'm working on my own idea now. I make scope transitive, because it's both memory safe and simple to implement, but doing so may cause some things which are actually safe to be considered unsafe (but then you could just use system blocks or trusted lambdas to correct this). Also, I don't think `scope` needs to be part of the type. I'm about 90 percent sure, 10 percent unsure that my system will work. I'll have it soon enough. It needs DIP25 to be expanded to all reference types (not just `ref`), requires my own DIP71, http://wiki.dlang.org/DIP71 for total safety, and possibly one or two more additions for a reliable ownership. The only real cost is added complexity to function signatures (a la DIP25), which can and should be inferred in most cases, assuming we aren't crippled by an ancient and subpar linking mechanism which requires all this manual marking of signatures all the time. Stay tuned, sir!
Feb 24 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope
Walter and I discussed this proposal at length several times. I think it can be made to work. I also think it's overly complicated for what it does and that it's not brilliant. These are the most dangerous designs: complicated, not brilliant, but can be made to work - because they end up getting implemented. We want to get away without implementing it. Currently we want to explore DIP25, which we believe has a very good price/performance profile (whilst not being brilliant itself), and its impact on designing safe struct types. It is our suspicion that DIP25 is all we need for making involved struct types safe. If that's the case, that paves the road toward safe class types using reference counting with very small language changes. There will be stragglers, such as taking slices into stack-allocated statically-sized arrays. At that point the question will be whether we can implement that as a safe struct, we need a language change, or we can just live with it (after all, using stack-allocated arrays in safe code is a rather niche use). So the short answer is: yes, I've studied your proposal and I don't think it's good enough. Furthermore I don't think there are small changes to it that will make it good enough. This is not personal and please don't take offense over that. In contrast, I think DIP25 is good (partly because it's deceptively simple) and I want to pursue it and its consequences. Andrei
Feb 24 2015
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 using stack-allocated arrays in safe code is a rather niche 
 use).
I use stack-allocated arrays in supposedly safe code. And having a future D compiler that makes that kind of code actually safe is a good thing. Bye, bearophile
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 8:34 AM, bearophile wrote:
 Andrei Alexandrescu:

 using stack-allocated arrays in safe code is a rather niche use).
I use stack-allocated arrays in supposedly safe code. And having a future D compiler that makes that kind of code actually safe is a good thing.
It definitely is a good thing. The question is balancing its goodness against the costs of allowing it. It's not impossible you may get asked to change your code to use a struct instead of a fixed-sized array. Andrei
Feb 24 2015
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 It definitely is a good thing. The question is balancing its 
 goodness against the costs of allowing it. It's not impossible 
 you may get asked to change your code to use a struct instead 
 of a fixed-sized array.
I don't like the look of the annotations of DIP25. I'd like DIP25 removed from D language and replaced by a more principled solution. Sometimes a heavier solution can be simpler to understand (and it can be actually safe). Bye, bearophile
Feb 24 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 11:12 AM, bearophile wrote:
 Andrei Alexandrescu:

 It definitely is a good thing. The question is balancing its goodness
 against the costs of allowing it. It's not impossible you may get
 asked to change your code to use a struct instead of a fixed-sized array.
I don't like the look of the annotations of DIP25.
The way things are going you'll likely have to live with that. -- Andrei
Feb 24 2015
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/24/2015 11:12 AM, bearophile wrote:
 I don't like the look of the annotations of DIP25. I'd like DIP25 removed from
D
 language and replaced by a more principled solution. Sometimes a heavier
 solution can be simpler to understand (and it can be actually safe).
So far, DIP25 has been called by various posters unprincipled, hackish, and stupid, without supporting rationale. Note the thread "A Refcounted Array Type" thread, which uses DIP25 to implement a memory safe ref counted container. It is simple and requires exactly one annotation. It seems pretty straightforward to me, and so far nobody has shown any holes in it, or has even commented on the principle of it. (complaints about bounds checking, delete, etc., are off-topic)
Feb 24 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 21:03:38 UTC, Walter Bright wrote:
 On 2/24/2015 11:12 AM, bearophile wrote:
 I don't like the look of the annotations of DIP25. I'd like 
 DIP25 removed from D
 language and replaced by a more principled solution. Sometimes 
 a heavier
 solution can be simpler to understand (and it can be actually 
 safe).
So far, DIP25 has been called by various posters unprincipled, hackish, and stupid, without supporting rationale.
I do think this comes from looking at the larger picture. In isolation, DIP25 is not that bad. It solve a class of problems properly, which is already something. Where I think most people find it hacky, unprincipled, or whatever is when looking at the larger picture. It seems obvious at this point that DIP25 is not the alpha and omega of the problem (this very thread is an example of this). For instance, DIP25 do not do anything for classes to be safely reference counted. People are - legitimately - afraid that a set of DIP25 like solution will be adopted for each of these problems. When looking at the total cost of the set of solution, compared to a more complex, more principled solution, it seems that this is not the way forward. In that sense, DIP25 seems like a hack to make reference counting work rather than the addition of a basic language feature that add a new dimension of expressiveness to the language. A bit like Rvalue references in C++ have been added to support std::move and std::forward instead of being a generally useful basic block to build upon.
 Note the thread "A Refcounted Array Type" thread, which uses 
 DIP25 to implement a memory safe ref counted container. It is 
 simple and requires exactly one annotation. It seems pretty 
 straightforward to me, and so far nobody has shown any holes in 
 it, or has even commented on the principle of it. (complaints 
 about bounds checking, delete, etc., are off-topic)
I still have to review it in details, but I'm sure this is good or at least fixable in a way that make it good. Yet, as mentioned, the problem do not come with DIP25 in isolation. DIP25 does its job well. It just does a too limited job, and it seems that DIP25 and DIP25 like solution for other jobs, will ultimately lead to a more complex situation than where the 'too complex' solutions would lead us.
Feb 24 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 1:44 PM, deadalnix wrote:
 For instance, DIP25 do not do anything for classes to be safely
 reference counted.
It does. One small language change is necessary. DIP will be forthcoming. -- Andrei
Feb 24 2015
prev sibling next sibling parent reply "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= <kuettler gmail.com> writes:
On Tuesday, 24 February 2015 at 16:18:45 UTC, Andrei Alexandrescu 
wrote:
 On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope
Walter and I discussed this proposal at length several times.
Is there a place where the outcome of such discussions is documented? This feels inappropriate like asking for additional paperwork. Still, there are so many heated discussions going on here. It might be helpful to know what was considered, what was decided and for what reason. Then again, maybe I just did not get the memo.
Feb 24 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 9:35 AM, "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= 
<kuettler gmail.com>" wrote:
 On Tuesday, 24 February 2015 at 16:18:45 UTC, Andrei Alexandrescu wrote:
 On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>"
 wrote:
 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope
Walter and I discussed this proposal at length several times.
Is there a place where the outcome of such discussions is documented? This feels inappropriate like asking for additional paperwork. Still, there are so many heated discussions going on here. It might be helpful to know what was considered, what was decided and for what reason. Then again, maybe I just did not get the memo.
Closest thing to documentation are http://wiki.dlang.org/DIP69 and http://wiki.dlang.org/DIP25, both of which came at the same time inspired by that work, and in reaction to its complexity. We are looking at what parts of DIP69 are simplified/obviated by DIP25. If we can get away with no DIP69 at all, so much the better. -- Andrei
Feb 24 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 24 February 2015 at 16:18:45 UTC, Andrei Alexandrescu 
wrote:
 On 2/24/15 4:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 As one example, here is what I originally suggested:
 http://wiki.dlang.org/User:Schuetzm/scope
Walter and I discussed this proposal at length several times. I think it can be made to work. I also think it's overly complicated for what it does and that it's not brilliant. These are the most dangerous designs: complicated, not brilliant, but can be made to work - because they end up getting implemented. We want to get away without implementing it. Currently we want to explore DIP25, which we believe has a very good price/performance profile (whilst not being brilliant itself), and its impact on designing safe struct types. It is our suspicion that DIP25 is all we need for making involved struct types safe. If that's the case, that paves the road toward safe class types using reference counting with very small language changes. There will be stragglers, such as taking slices into stack-allocated statically-sized arrays. At that point the question will be whether we can implement that as a safe struct, we need a language change, or we can just live with it (after all, using stack-allocated arrays in safe code is a rather niche use). So the short answer is: yes, I've studied your proposal and I don't think it's good enough. Furthermore I don't think there are small changes to it that will make it good enough. This is not personal and please don't take offense over that. In contrast, I think DIP25 is good (partly because it's deceptively simple) and I want to pursue it and its consequences.
I'm certainly not offended. I don't think the proposal is "brilliant" either, it was meant as a starting point for discussions, not as a completely finished proposal aka DIP. Out of interest, because that will be crucial for the further development: are you completely opposed to any kind of ownership system, or is it something specific about this particular proposal that you don't like?
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 9:47 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:
 Out of interest, because that will be crucial for the further
 development: are you completely opposed to any kind of ownership system,
 or is it something specific about this particular proposal that you
 don't like?
There's no opposition on matters of principle/dogma. It's just too complicated for what it provides. Andrei
Feb 24 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 18:01:35 UTC, Andrei Alexandrescu 
wrote:
 On 2/24/15 9:47 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>" wrote:
 Out of interest, because that will be crucial for the further
 development: are you completely opposed to any kind of 
 ownership system,
 or is it something specific about this particular proposal 
 that you
 don't like?
There's no opposition on matters of principle/dogma. It's just too complicated for what it provides. Andrei
Compared to what ?
Feb 24 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/15 11:14 AM, deadalnix wrote:
 On Tuesday, 24 February 2015 at 18:01:35 UTC, Andrei Alexandrescu wrote:
 On 2/24/15 9:47 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>"
 wrote:
 Out of interest, because that will be crucial for the further
 development: are you completely opposed to any kind of ownership system,
 or is it something specific about this particular proposal that you
 don't like?
There's no opposition on matters of principle/dogma. It's just too complicated for what it provides. Andrei
Compared to what ?
There is no comparison involved. The proposal is too complicated for what it does. -- Andrei
Feb 24 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 24 February 2015 at 19:20:37 UTC, Andrei Alexandrescu 
wrote:
 Compared to what ?
There is no comparison involved. The proposal is too complicated for what it does. -- Andrei
There is always a comparison involved. Comparison options includes: - Not solving some of the issues we have and call it a day. - Add simpler solution dedicated to each of these problems. The problems we are talking about are: - safe RC - fast GC by using type qualifier guarantee. - less reliance on GC by producing less garbage. - enforcing type qualifier constraint. - enabling more code to be nogc (especially exception) - being able to delegate the memory management policy to the user. - immutable and shared building. - manipulating object of limited lifetime (DIP25 solve this partially). - `unique` message passing . - safe parallelism on graph of object instead of only value types. - lock on graph of objects (shared containers). Now it is clear that we can come up with simpler solution for some of these problem than whatever all encompassing solution would be. It is also clear that is is not possible to get a set of simpler solution that is, as a whole, simpler than an all encompassing solution. That means that some of the above mentioned point must be acknowledged as not going to be fixed. It is also to be noted that these above mentioned problems are not orthogonal. For instance, getting various GC optimizations in that take advantage of type qualifiers guarantee require that these guarantee are enforced. To get these constraints enforced, you need to be able to build immutable and/or shared in a way that don't break these constraints, and so on.
Feb 24 2015