www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - scope for array parameters

reply Benjamin Thaut <code benjamin-thaut.de> writes:
To make functions definitions more expressive and give the compiler more 
information to optimize I propose to make array function parameters 
extendable with 'scope' such as follows:

size_t find(scope float[] haystack, float needle){ ... }

This would give the compiler the information that the array haystack 
only has to be valid as long as the scope of the function find is valid. 
As soon as the function scope is left, the array does not need to be 
valid any more. This would greatly help when programming without a GC to 
know, if the array one passes to the function will be saved internally 
or if it only needs to be valid for the duration in which the function 
is executed. Also it would allow the compiler to optimize array literals 
such as:

find([1,2,3,4,5], 5)

The function call would not allocate on the heap, but would allocate the 
array literal on the stack, as the compiler knows that is only has to be 
valid for the scope of find.

Passing a scope array to another function which does not have the scope 
annotation should be an error. Assining a a scope array to a non local 
variable should be an error too. You could go really fancy on checking 
scoped function parameters but for a start simple rules should be enough.

The same could be done with references and pointers. The purpose of this 
would be to allow the compiler to do more static checking on the usage 
of the passed arguments and make the function definitions contain 
information if the passed references will be saved internally or not.

I do understand that non-gc programming is not considered often for the 
D-Programming Language, but it would be nice if this idea does not get 
ignored completely.

What do you think about this idea?

Kind Regards
Benjamin Thaut
Sep 04 2012
next sibling parent reply =?ISO-8859-15?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 04-09-2012 22:13, Benjamin Thaut wrote:
 To make functions definitions more expressive and give the compiler more
 information to optimize I propose to make array function parameters
 extendable with 'scope' such as follows:

 size_t find(scope float[] haystack, float needle){ ... }

 This would give the compiler the information that the array haystack
 only has to be valid as long as the scope of the function find is valid.
 As soon as the function scope is left, the array does not need to be
 valid any more. This would greatly help when programming without a GC to
 know, if the array one passes to the function will be saved internally
 or if it only needs to be valid for the duration in which the function
 is executed. Also it would allow the compiler to optimize array literals
 such as:

 find([1,2,3,4,5], 5)

 The function call would not allocate on the heap, but would allocate the
 array literal on the stack, as the compiler knows that is only has to be
 valid for the scope of find.

 Passing a scope array to another function which does not have the scope
 annotation should be an error. Assining a a scope array to a non local
 variable should be an error too. You could go really fancy on checking
 scoped function parameters but for a start simple rules should be enough.

 The same could be done with references and pointers. The purpose of this
 would be to allow the compiler to do more static checking on the usage
 of the passed arguments and make the function definitions contain
 information if the passed references will be saved internally or not.

 I do understand that non-gc programming is not considered often for the
 D-Programming Language, but it would be nice if this idea does not get
 ignored completely.

 What do you think about this idea?

 Kind Regards
 Benjamin Thaut

This is already what scope does today. See http://dlang.org/function.html: "references in the parameter cannot be escaped (e.g. assigned to a global variable)" It's just that the compiler doesn't actually enforce it fully. -- Alex Rønne Petersen alex lycus.org http://lycus.org
Sep 04 2012
next sibling parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 04.09.2012 22:18, schrieb Alex Rønne Petersen:
 On 04-09-2012 22:13, Benjamin Thaut wrote:
 This is already what scope does today. See http://dlang.org/function.html:

 "references in the parameter cannot be escaped (e.g. assigned to a
 global variable)"

 It's just that the compiler doesn't actually enforce it fully.

Does the compiler already use this information to optimize array literals? Kind Regards Benjamin Thaut
Sep 04 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 22:19:55 Benjamin Thaut wrote:
 Am 04.09.2012 22:18, schrieb Alex Rønne Petersen:
 On 04-09-2012 22:13, Benjamin Thaut wrote:
 This is already what scope does today. See http://dlang.org/function.html:
 
 "references in the parameter cannot be escaped (e.g. assigned to a
 global variable)"
 
 It's just that the compiler doesn't actually enforce it fully.

Does the compiler already use this information to optimize array literals?

It had better not (I doubt that it does though). Unless scope is actually properly enforced, then you're going to have major bugs if you use scope and then escape references to that data anyway. And without the compiler checks, that _will_ happen (especially when some folks use in all over the place). Once scope is properly enforced, then optimizing based on it would be great, but until it is, it's a _bad_ idea. - Jonathan M Davis
Sep 04 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/05/2012 01:50 AM, bearophile wrote:
 Jonathan M Davis:

 That's part of why I keep saying not to use in whenever it comes up.
 scope is
 very broken, so in is very broken. And honestly, given how often
 arrays are
 used in structs, I suspect that it's not at all uncommon for in to be
 used incorrectly.

The situation with "in"/"scope" is worse than just deprecated stuff like "delete" or "typedef". I know those things are going away, so I don't use them, and this avoids the problem.
 I believe that the only case that
 has _any_ protection at all with scope right now is delegates, which
 almost never should be const.

Do you mean code like this? What's bad about this? My delegate arguments /function pointer arguments are usually const. void foo(const int delegate(int) dg) {} void main() { foo((int x) => x); } Bye, bearophile

Similar code is the main reason for the hole in the const system. Some people want that fixed, ergo it might break.
Sep 05 2012
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 Once scope is properly enforced, then optimizing based on it 
 would be great, but until it is, it's a _bad_ idea.

Today we use "in" often for function arguments, and it implies "scope". Surely some of those programs use the arguments wrongly, this means they probably sometime escape. So are those programs someday going to become compilation errors? For the "-property" and for the arguments of delegate/function pointers the idea of turning some code currently used in errors is failing... Such things need to be implemented early in a language. Bye, bearophile
Sep 04 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/05/2012 02:10 AM, Jonathan M Davis wrote:
 On Wednesday, September 05, 2012 01:13:00 ixid wrote:
 What does -property supposedly solve?

It's supposed to make it so that property functions are used as variables and non-property functions are used as functions, since the point of property functions is to emulate variables. It's very buggy at the moment though, so it doesn't really enforce everything that it's supposed to enforce. IIRC, it only enforces that non-property functions are called with parens and not that property functions are called without them.
 It creates a horrid mess of
 brackets that ruin the elegance of UFCS code.

You mean parens? Regardless, it means that you using functions as if there were variables, which I'm very much against. They're functions and should be called as such. But not everyone agrees. I think that it's pretty much a guarantee that we're eventually going to end up with property functions having to be called without parens (I don't think that very many people disagree on that). The bigger question is whether we're going to follow TDPL and also make it so that non-property functions must be called with them (as -property currently checks for).

TDPL does not prescribe this.
 The number of parens
 required with UFCS (particularly with functions requiring template arguments)
 is one of the reasons that some people give that they think that non-property
 functions shouldn't be forced to be called with parens.

 - Jonathan M Davis

Sep 05 2012
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 4 September 2012 at 21:41:58 UTC, bearophile wrote:
 Jonathan M Davis:

 Once scope is properly enforced, then optimizing based on it 
 would be great, but until it is, it's a _bad_ idea.

Today we use "in" often for function arguments, and it implies "scope". Surely some of those programs use the arguments wrongly, this means they probably sometime escape. So are those programs someday going to become compilation errors?

Yep.
 Such things need to be implemented early in a language.

Yep. It's very unfortunate because it puts us in a lose-lose situations where we can't fix the language without breaking people's code. I think the best course of action is to fix these things as soon as possible and get the problem out of the way.
Sep 04 2012
prev sibling parent "ixid" <nuaccount gmail.com> writes:
What does -property supposedly solve? It creates a horrid mess of 
brackets that ruin the elegance of UFCS code.
Sep 04 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 22:13:44 Benjamin Thaut wrote:
 What do you think about this idea?

scope on function parameters is supposed to prevent any reference to that data escaping from the function. e.g. scope on a delegate would make it an error to assign that delegate to anything which could escape the function. It is my understanding that this is also supposed to include arrays such that if you mark an array parameter with scope, it should be illegal for any references to it (including slices of it or references to its elements) to escape the function, which _would_ allow compiler optimizations. But the compiler doesn't seem to properly check for much with regards to scope right now. A _lot_ of code is going to break once it does, particularly since some people seem to think that it's a good idea to use in (which is the same as const scope) everywhere. - Jonathan M Davis
Sep 04 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 23:57:24 Peter Alexander wrote:
 It's very unfortunate because it puts us in a lose-lose
 situations where we can't fix the language without breaking
 people's code. I think the best course of action is to fix these
 things as soon as possible and get the problem out of the way.

Indeed. The same goes for stuff like deprecating things that we've long said was going to be deprecated but hasn't yet been deprecated (e.g. delete). Lots of code is going to break when it happens, and the longer that we wait, the worse it's going to be. But until someone takes the time to fix it (and Walter is constantly busy on other things), it's not going to happen, which is obviously a big problem. - Jonathan M Davis
Sep 04 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, September 04, 2012 23:42:33 bearophile wrote:
 Jonathan M Davis:
 Once scope is properly enforced, then optimizing based on it
 would be great, but until it is, it's a _bad_ idea.

Today we use "in" often for function arguments, and it implies "scope". Surely some of those programs use the arguments wrongly, this means they probably sometime escape. So are those programs someday going to become compilation errors? For the "-property" and for the arguments of delegate/function pointers the idea of turning some code currently used in errors is failing... Such things need to be implemented early in a language.

That's part of why I keep saying not to use in whenever it comes up. scope is very broken, so in is very broken. And honestly, given how often arrays are used in structs, I suspect that it's not at all uncommon for in to be used incorrectly. I'd actually argue to _never_ to use in at this point. If you want const, then use const. If you want scope, then use scope. But using scope on anything which isn't checked properly by the compiler is just asking for it. You _will_ have code that breaks when scope is fixed. I believe that the only case that has _any_ protection at all with scope right now is delegates, which almost never should be const. So, using in makes little to no sense to me. - Jonathan M Davis
Sep 04 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 That's part of why I keep saying not to use in whenever it 
 comes up. scope is
 very broken, so in is very broken. And honestly, given how 
 often arrays are
 used in structs, I suspect that it's not at all uncommon for in 
 to be used incorrectly.

The situation with "in"/"scope" is worse than just deprecated stuff like "delete" or "typedef". I know those things are going away, so I don't use them, and this avoids the problem.
 I believe that the only case that
 has _any_ protection at all with scope right now is delegates, 
 which almost never should be const.

Do you mean code like this? What's bad about this? My delegate arguments /function pointer arguments are usually const. void foo(const int delegate(int) dg) {} void main() { foo((int x) => x); } Bye, bearophile
Sep 04 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, September 05, 2012 01:13:00 ixid wrote:
 What does -property supposedly solve?

It's supposed to make it so that property functions are used as variables and non-property functions are used as functions, since the point of property functions is to emulate variables. It's very buggy at the moment though, so it doesn't really enforce everything that it's supposed to enforce. IIRC, it only enforces that non-property functions are called with parens and not that property functions are called without them.
 It creates a horrid mess of
 brackets that ruin the elegance of UFCS code.

You mean parens? Regardless, it means that you using functions as if there were variables, which I'm very much against. They're functions and should be called as such. But not everyone agrees. I think that it's pretty much a guarantee that we're eventually going to end up with property functions having to be called without parens (I don't think that very many people disagree on that). The bigger question is whether we're going to follow TDPL and also make it so that non-property functions must be called with them (as -property currently checks for). The number of parens required with UFCS (particularly with functions requiring template arguments) is one of the reasons that some people give that they think that non-property functions shouldn't be forced to be called with parens. - Jonathan M Davis
Sep 04 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, September 05, 2012 01:50:12 bearophile wrote:
 I believe that the only case that
 has _any_ protection at all with scope right now is delegates,
 which almost never should be const.

Do you mean code like this? What's bad about this? My delegate arguments /function pointer arguments are usually const. void foo(const int delegate(int) dg) {} void main() { foo((int x) => x); }

If you have delegates that work with const, then great, but in my experience (which frequently involves storing the delegate as a member variable), const doesn't play nicely at all with delegates. - Jonathan M Davis
Sep 04 2012
prev sibling parent "Chris Nicholson-Sauls" <ibisbasenji gmail.com> writes:
On Wednesday, 5 September 2012 at 12:05:40 UTC, Timon Gehr wrote:
 TDPL does not prescribe this.

Truth; I just checked this and indeed TDPL only mentions property functions disavowing themselves of parentheses. Although I've always had the distinct impression that strict property syntax was the intended future, and the current situation was just due to inheriting D1's clumsier 'property' concept. I don't even mind all those "ugly" parens in UFCS chains, but I'm kinda crazy like that.
Sep 05 2012