www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Why are scope variables being deprecated?

reply Chad J <chadjoan __spam.is.bad__gmail.com> writes:
I keep hearing that scope variables are going away.  I missed the 
discussion on it.  Why is this happening?

When I read about this, I have these in mind:

void someFunc()
{
	// foo is very likely to get stack allocated
	scope foo = new SomeClass();
	foo.use();
	// ~foo is called.
}
Jul 26 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, July 26, 2012 21:09:09 Chad J wrote:
 I keep hearing that scope variables are going away.  I missed the
 discussion on it.  Why is this happening?
 
 When I read about this, I have these in mind:
 
 void someFunc()
 {
 	// foo is very likely to get stack allocated
 	scope foo = new SomeClass();
 	foo.use();
 	// ~foo is called.
 }

It's inherently unsafe. What happens if you returned a reference to foo from someFunc? Or if you assigned a reference to foo to anything and then tried to use it after someFunc has returned? You get undefined behavior, because foo doesn't exist anymore. If you really need foo to be on the stack, then maybe you should make it a struct. However, if you really do need scope for some reason, then you can use std.typecons.scoped, and it'll do the same thing. scope on local variables is going away for pretty much the same reason that delete is. They're unsafe, and the fact that they're in the core language encourages their use. So, they're being removed and put into the standard library instead. - Jonathan M Davis
Jul 26 2012
next sibling parent Chad J <chadjoan __spam.is.bad__gmail.com> writes:
On 07/26/2012 09:19 PM, Jonathan M Davis wrote:
 On Thursday, July 26, 2012 21:09:09 Chad J wrote:
 I keep hearing that scope variables are going away.  I missed the
 discussion on it.  Why is this happening?

 When I read about this, I have these in mind:

 void someFunc()
 {
 	// foo is very likely to get stack allocated
 	scope foo = new SomeClass();
 	foo.use();
 	// ~foo is called.
 }

It's inherently unsafe. What happens if you returned a reference to foo from someFunc? Or if you assigned a reference to foo to anything and then tried to use it after someFunc has returned? You get undefined behavior, because foo doesn't exist anymore. If you really need foo to be on the stack, then maybe you should make it a struct. However, if you really do need scope for some reason, then you can use std.typecons.scoped, and it'll do the same thing.

OK, so std.typecons.scoped will completely replace the use-case for the scope keyword. That makes it OK ;) Just making things structs isn't always sufficient because the data type in question might be in a 3rd party's code and cannot be simply redesigned. The scope keyword gave us a way to force stack-allocation in cases that would be otherwise inaccessible. But it seems like std.typecons.scoped can be used for this, so 'scope' isn't need anymore. And it simplifies the compiler. Cool. Erm, yeah I'm sure you've probably seen this discussed to death already. I know how these things go ;)
 scope on local variables is going away for pretty much the same reason that
 delete is. They're unsafe, and the fact that they're in the core language
 encourages their use. So, they're being removed and put into the standard
 library instead.

 - Jonathan M Davis

Alright. Thanks for the good explanation!
Jul 26 2012
prev sibling parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Jonathan M Davis wrote:
 On Thursday, July 26, 2012 21:09:09 Chad J wrote:
 I keep hearing that scope variables are going away.  I missed the
 discussion on it.  Why is this happening?

 When I read about this, I have these in mind:

 void someFunc()
 {
 	// foo is very likely to get stack allocated
 	scope foo = new SomeClass();
 	foo.use();
 	// ~foo is called.
 }

It's inherently unsafe. What happens if you returned a reference to foo from someFunc? Or if you assigned a reference to foo to anything and then tried to use it after someFunc has returned?

Why scope parameters are not deprecated then? It's the same situation.
 You get undefined behavior, because foo
 doesn't exist anymore.

Excuse me, but no, compiler should prevent escaping scope references just like it does with scope parameters (I know it's currently implemented just for delegates).
 If you really need foo to be on the stack, then maybe
 you should make it a struct.

Then you lose some useful class features.
 scope on local variables is going away for pretty much the same reason that
 delete is. They're unsafe, and the fact that they're in the core language
 encourages their use.

That's not convincing for me. Pointers are also unsafe, and they're in the core language.
 However, if you really do need scope for some
 reason, then you can use std.typecons.scoped, and it'll do the same 

scoped is more dangerous than language solution. See: class A { } __gshared A globalA; static this() { auto a = scoped!A; globalA = a; } and this compiles (http://dpaste.dzfl.pl/6c078e66). With scope storage class compiler would prevent this escaping assignment. It seems that we ended up with a solution that was meant to fix a language builtin but appears to be worse than that.
Oct 10 2012
next sibling parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
bearophile wrote:
 Piotr Szturmaj:

 It seems that we ended up with a solution that was meant to fix a
 language builtin but appears to be worse than that.

This is true, currently the library solution is worse (more dangerous and more broken) than the precedent built-in feature. But there is hope to have a good solution someday (mixing library code and some kind of built-support), while a broken built-in is not good. Andrei did the right thing: if you don't have a feature it's kind of easy to add something, while fixing some bad built-in is rather harder.

Wasn't it broken because preventing escaping of scoped references was not implemented?
Oct 10 2012
prev sibling parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Jonathan M Davis wrote:
 On Wednesday, October 10, 2012 17:04:41 Piotr Szturmaj wrote:
 Jonathan M Davis wrote:
 On Thursday, July 26, 2012 21:09:09 Chad J wrote:
 I keep hearing that scope variables are going away. I missed the
 discussion on it. Why is this happening?

 When I read about this, I have these in mind:

 void someFunc()
 {

 // foo is very likely to get stack allocated
 scope foo = new SomeClass();
 foo.use();
 // ~foo is called.

 }

It's inherently unsafe. What happens if you returned a reference to foo from someFunc? Or if you assigned a reference to foo to anything and then tried to use it after someFunc has returned?

Why scope parameters are not deprecated then? It's the same situation.

No. scope on parameters is completely different from scope on local variables. scope on local variables puts the variable on the stack - even if it's a class.

Yes, I know the difference between scope parameters and variables, but I thought that they both can be considered "scope references" which can't be escaped. I don't support restoring scope variables in their previous state. But I think I know a way to make scope variables safe by default.
 scope on function parameters is supposed to make it so that the compiler
 prevents any references escaping (which potentially really restricts how you
 can use the parameter). The only case where that would affect where a variable
 is placed is that it makes it so that a closure isn't created for delegates
 (which is the one place that scope on parameters actually works semi-
 properly). So, the two uses of scope do completely different things.

Could you give me an example of preventing closure allocation? I think I knew one but I don't remember now... With regards to escaping scope reference parameters, I hope that eventually they all will be blocked by the compiler, not only delegate/closure case.
 If you really need foo to be on the stack, then maybe
 you should make it a struct.

Then you lose some useful class features.

What you lose is polymorphism, which doesn't work on the stack anyway. Polymorphism is only applicable when you have a reference which could be of a base class type rather than the derived type that the object actually is. Objects on the stack must be their exact type.

I know, class on the stack really become a "value" type. But it's still useful. You can use non-scope classes with polymorhism as usual, but when needed you can allocate one concrete class on the stack. You can't assign subclass reference to scope class variable, but you still can assign scope class reference to non-scope ancestor class references. This may or may _not_ escape. I'm proposing that escaping assignments should be blocked.
 scope on local variables is going away for pretty much the same reason
 that
 delete is. They're unsafe, and the fact that they're in the core language
 encourages their use.

That's not convincing for me. Pointers are also unsafe, and they're in the core language.

Pointers aren't unsafe. Certain operations are unsafe. Note that pointers are perfectly legal in safe code. It's pointer arithmetic which isn't.

OK.
 and this compiles (http://dpaste.dzfl.pl/6c078e66). With scope storage
 class compiler would prevent this escaping assignment. It seems that we
 ended up with a solution that was meant to fix a language builtin but
 appears to be worse than that.

It may very well be more dangerous, and that may or may not be fixable, but if it's at the language level, then a lot more people are likely to use it, and it's dangerous no matter where it is and shouldn't be used under normal circumstances. Providing the feature is one thing. Making it easy to use is another. It's like delete. It's dangerous and shouldn't be used normally, so having it in the language where everyone will use it is too dangerous, so a library solution is used instead. It therefore becomes more of a power user feature (as it should be).

I agree about delete operator, but as I wrote above, I'm not sure, but I might know a way to make scope variables safe. I need to think about this :)
 But regardless of the various pros and cons, it was decided ages ago that
 it was not worth have scope on local variable be part of the language any
 more. So, it's definitely going away.

I see, but scope might be also used in other scenarios, like emplacing classes inside other classes.
Oct 10 2012
parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
Jonathan M Davis wrote:
 On Thursday, October 11, 2012 01:24:40 Piotr Szturmaj wrote:
 Could you give me an example of preventing closure allocation? I think I
 knew one but I don't remember now...

Any time that a delegate parameter is marked as scope, the compiler will skip allocating a closure. Otherwise, it has to copy the stack from the caller onto the heap to create a closure so that the delegate will continue to work once the caller has completed (e.g. if the delegate were saved for a callback and then called way later in the program). Otherwise, it would refer to an invalid stack and really nasty things would happen when the delegate was called later. By marking the delegate as scope, you're telling the compiler that it will not escape the function that it's being passed to, so the compiler then knows that the stack that it refers to will be valid for the duration of that delegate's existence, so it knows that a closure is not required, so it doesn't allocate it, gaining you efficiency.

Thanks, that's clear now, but I found a bug: __gshared void delegate() global; void dgtest(scope void delegate() dg) { global = dg; // compiles } void dguse() { int i; dgtest({ writeln(i++); }); } I guess it's a known one.
Oct 10 2012
parent Don Clugston <dac nospam.com> writes:
On 11/10/12 02:30, Piotr Szturmaj wrote:
 Jonathan M Davis wrote:
 On Thursday, October 11, 2012 01:24:40 Piotr Szturmaj wrote:
 Could you give me an example of preventing closure allocation? I think I
 knew one but I don't remember now...

Any time that a delegate parameter is marked as scope, the compiler will skip allocating a closure. Otherwise, it has to copy the stack from the caller onto the heap to create a closure so that the delegate will continue to work once the caller has completed (e.g. if the delegate were saved for a callback and then called way later in the program). Otherwise, it would refer to an invalid stack and really nasty things would happen when the delegate was called later.

 By marking the delegate as scope, you're telling the compiler that it
 will not
 escape the function that it's being passed to, so the compiler then
 knows that
 the stack that it refers to will be valid for the duration of that
 delegate's
 existence, so it knows that a closure is not required, so it doesn't
 allocate
 it, gaining you efficiency.

Thanks, that's clear now, but I found a bug: __gshared void delegate() global; void dgtest(scope void delegate() dg) { global = dg; // compiles } void dguse() { int i; dgtest({ writeln(i++); }); } I guess it's a known one.

Looks like bug 5270?
Oct 11 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Piotr Szturmaj:

 It seems that we ended up with a solution that was meant to fix 
 a language builtin but appears to be worse than that.

This is true, currently the library solution is worse (more dangerous and more broken) than the precedent built-in feature. But there is hope to have a good solution someday (mixing library code and some kind of built-support), while a broken built-in is not good. Andrei did the right thing: if you don't have a feature it's kind of easy to add something, while fixing some bad built-in is rather harder. Bye, bearophile
Oct 10 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, October 10, 2012 17:04:41 Piotr Szturmaj wrote:
 Jonathan M Davis wrote:
 On Thursday, July 26, 2012 21:09:09 Chad J wrote:
 I keep hearing that scope variables are going away. I missed the
 discussion on it. Why is this happening?
 
 When I read about this, I have these in mind:
 
 void someFunc()
 {
 
 // foo is very likely to get stack allocated
 scope foo = new SomeClass();
 foo.use();
 // ~foo is called.
 
 }

It's inherently unsafe. What happens if you returned a reference to foo from someFunc? Or if you assigned a reference to foo to anything and then tried to use it after someFunc has returned?

Why scope parameters are not deprecated then? It's the same situation.

No. scope on parameters is completely different from scope on local variables. scope on local variables puts the variable on the stack - even if it's a class. scope on function parameters is supposed to make it so that the compiler prevents any references escaping (which potentially really restricts how you can use the parameter). The only case where that would affect where a variable is placed is that it makes it so that a closure isn't created for delegates (which is the one place that scope on parameters actually works semi- properly). So, the two uses of scope do completely different things.
 If you really need foo to be on the stack, then maybe
 you should make it a struct.

Then you lose some useful class features.

What you lose is polymorphism, which doesn't work on the stack anyway. Polymorphism is only applicable when you have a reference which could be of a base class type rather than the derived type that the object actually is. Objects on the stack must be their exact type.
 scope on local variables is going away for pretty much the same reason
 that
 delete is. They're unsafe, and the fact that they're in the core language
 encourages their use.

That's not convincing for me. Pointers are also unsafe, and they're in the core language.

Pointers aren't unsafe. Certain operations are unsafe. Note that pointers are perfectly legal in safe code. It's pointer arithmetic which isn't.
 and this compiles (http://dpaste.dzfl.pl/6c078e66). With scope storage
 class compiler would prevent this escaping assignment. It seems that we
 ended up with a solution that was meant to fix a language builtin but
 appears to be worse than that.

It may very well be more dangerous, and that may or may not be fixable, but if it's at the language level, then a lot more people are likely to use it, and it's dangerous no matter where it is and shouldn't be used under normal circumstances. Providing the feature is one thing. Making it easy to use is another. It's like delete. It's dangerous and shouldn't be used normally, so having it in the language where everyone will use it is too dangerous, so a library solution is used instead. It therefore becomes more of a power user feature (as it should be). But regardless of the various pros and cons, it was decided ages ago that it was not worth have scope on local variable be part of the language any more. So, it's definitely going away. - Jonathan M Davis
Oct 10 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, October 11, 2012 01:24:40 Piotr Szturmaj wrote:
 Could you give me an example of preventing closure allocation? I think I
 knew one but I don't remember now...

Any time that a delegate parameter is marked as scope, the compiler will skip allocating a closure. Otherwise, it has to copy the stack from the caller onto the heap to create a closure so that the delegate will continue to work once the caller has completed (e.g. if the delegate were saved for a callback and then called way later in the program). Otherwise, it would refer to an invalid stack and really nasty things would happen when the delegate was called later. By marking the delegate as scope, you're telling the compiler that it will not escape the function that it's being passed to, so the compiler then knows that the stack that it refers to will be valid for the duration of that delegate's existence, so it knows that a closure is not required, so it doesn't allocate it, gaining you efficiency. - Jonathan M Davis
Oct 10 2012