www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Array literals MUST be immutable.

reply Don <nospam nospam.com> writes:
This is for me the last remaining D2 issue.
I've mentioned this several times before. Mutable array 'literals' are 
wrong on so many levels. Here are some of the issues, listed in 
decreasing importance.

(1) The language has no syntax for immutable array literals.
   [1,2,3] is mutable. You can created an immutable literal using the 
awful 'static const' syntax, but that only works for named literals.

(2) Concurrency issues make (1) even more important. It ought to 
possible to pass an array (defined at compile time) as a message.

(3) Performance of the existing array 'literals' is absolutely 
appalling. I doubled the speed of my entire app by changing ONE array 
declaration from 'immutable [] xxx' into 'static const [] xxx'!!!
Right now, if an array literal appears in your code, chances are it's a 
performance bug. It's deceptive that there's a hidden heap allocation 
happening in every array literal. It's got no business being in a 
systems language IMHO.

(4) It is unintuitive. "hello" is immutable, but ['h','e','l','l','o'] 
is not???

(5) It's trivial (a one-liner) to make a function to create a mutable 
array literal. We don't need it in the language.

Mutable arrays literals are common in toy examples, but they've got very 
little real-world relevance. By contrast, immutable array literals are 
used in lookup tables all the time. They are extremely common. We need 
to be able to express them.

This is the only remaining thing in D2 which I'm certain is a big 
mistake. Of course there's ugliness in various places, but it's possible 
to ignore them or work around them. This is a primitive, 
bread-and-butter issue with no workaround.
Feb 17 2010
next sibling parent BCS <none anon.com> writes:
Hello Don,

 This is the only remaining thing in D2 which I'm certain is a big
 mistake. Of course there's ugliness in various places, but it's
 possible to ignore them or work around them. This is a primitive,
 bread-and-butter issue with no workaround.
Sounds like you have a good point. Vote++; -- ... <IXOYE><
Feb 17 2010
prev sibling next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Don <nospam nospam.com> wrote:

 This is for me the last remaining D2 issue.
 I've mentioned this several times before. Mutable array 'literals' are  
 wrong on so many levels. Here are some of the issues, listed in  
 decreasing importance.

 (1) The language has no syntax for immutable array literals.
    [1,2,3] is mutable. You can created an immutable literal using the  
 awful 'static const' syntax, but that only works for named literals.

 (2) Concurrency issues make (1) even more important. It ought to  
 possible to pass an array (defined at compile time) as a message.

 (3) Performance of the existing array 'literals' is absolutely  
 appalling. I doubled the speed of my entire app by changing ONE array  
 declaration from 'immutable [] xxx' into 'static const [] xxx'!!!
 Right now, if an array literal appears in your code, chances are it's a  
 performance bug. It's deceptive that there's a hidden heap allocation  
 happening in every array literal. It's got no business being in a  
 systems language IMHO.

 (4) It is unintuitive. "hello" is immutable, but ['h','e','l','l','o']  
 is not???

 (5) It's trivial (a one-liner) to make a function to create a mutable  
 array literal. We don't need it in the language.

 Mutable arrays literals are common in toy examples, but they've got very  
 little real-world relevance. By contrast, immutable array literals are  
 used in lookup tables all the time. They are extremely common. We need  
 to be able to express them.

 This is the only remaining thing in D2 which I'm certain is a big  
 mistake. Of course there's ugliness in various places, but it's possible  
 to ignore them or work around them. This is a primitive,  
 bread-and-butter issue with no workaround.
Absolutely. This needs fixin'. -- Simen
Feb 17 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 17 Feb 2010 11:47:00 +0300, Don <nospam nospam.com> wrote:

 This is for me the last remaining D2 issue.
 I've mentioned this several times before. Mutable array 'literals' are  
 wrong on so many levels. Here are some of the issues, listed in  
 decreasing importance.

 (1) The language has no syntax for immutable array literals.
    [1,2,3] is mutable. You can created an immutable literal using the  
 awful 'static const' syntax, but that only works for named literals.

 (2) Concurrency issues make (1) even more important. It ought to  
 possible to pass an array (defined at compile time) as a message.

 (3) Performance of the existing array 'literals' is absolutely  
 appalling. I doubled the speed of my entire app by changing ONE array  
 declaration from 'immutable [] xxx' into 'static const [] xxx'!!!
 Right now, if an array literal appears in your code, chances are it's a  
 performance bug. It's deceptive that there's a hidden heap allocation  
 happening in every array literal. It's got no business being in a  
 systems language IMHO.

 (4) It is unintuitive. "hello" is immutable, but ['h','e','l','l','o']  
 is not???

 (5) It's trivial (a one-liner) to make a function to create a mutable  
 array literal. We don't need it in the language.

 Mutable arrays literals are common in toy examples, but they've got very  
 little real-world relevance. By contrast, immutable array literals are  
 used in lookup tables all the time. They are extremely common. We need  
 to be able to express them.

 This is the only remaining thing in D2 which I'm certain is a big  
 mistake. Of course there's ugliness in various places, but it's possible  
 to ignore them or work around them. This is a primitive,  
 bread-and-butter issue with no workaround.
Absolutely agree!
Feb 17 2010
prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Don wrote:
 This is for me the last remaining D2 issue.
That would make it difficult to do things like: int*[] foo(int *p) { return [p, p + 1]; } as all the elements of the literal would also have to be immutable. I think you've made a good case, but there is also this issue.
Feb 17 2010
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Walter Bright wrote:
 Don wrote:
 This is for me the last remaining D2 issue.
That would make it difficult to do things like: int*[] foo(int *p) { return [p, p + 1]; } as all the elements of the literal would also have to be immutable. I think you've made a good case, but there is also this issue.
Note his point (5). It's trivial to add a function for creating mutable arrays to std.array. import std.array; int*[] foo(int* p) { return array(p, p+1); } I think Don makes an excellent case. This needs to be fixed. -Lars
Feb 17 2010
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 17 Feb 2010 12:17:09 +0300, Walter Bright  
<newshound1 digitalmars.com> wrote:

 Don wrote:
 This is for me the last remaining D2 issue.
That would make it difficult to do things like: int*[] foo(int *p) { return [p, p + 1]; } as all the elements of the literal would also have to be immutable. I think you've made a good case, but there is also this issue.
IIRC, it was discussed before and the following solution was suggested: T[] toArray(T)(T[] values...) { return values.dup; // not sure why dup is needed here, but DMD2.039 complains about escaping reference } int main() { int* p = null; int*[] array = toArray(p, p + 1); assert(array.length == 2); assert(array[0] is p); assert(array[1] is (p + 1)); return 0; } In my opinion, this is a minor use case that has graceful solution, in no way it is a show stopper.
Feb 17 2010
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Denis Koroskin (2korden gmail.com)'s article
 IIRC, it was discussed before and the following solution was suggested:
 T[] toArray(T)(T[] values...)
 {
      return values.dup; // not sure why dup is needed here, but DMD2.039
 complains about escaping reference
When using this kind of variadics, an implementation is allowed to allocate the array on the stack for performance. Therefore, you were trying to escape references to a stack-allocated array.
Feb 17 2010
parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 17 Feb 2010 16:58:35 +0300, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Denis Koroskin (2korden gmail.com)'s article
 IIRC, it was discussed before and the following solution was suggested:
 T[] toArray(T)(T[] values...)
 {
      return values.dup; // not sure why dup is needed here, but DMD2.039
 complains about escaping reference
When using this kind of variadics, an implementation is allowed to allocate the array on the stack for performance. Therefore, you were trying to escape references to a stack-allocated array.
Oh, great! Didn't know the fact, thanks!
Feb 17 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-02-17 04:17:09 -0500, Walter Bright <newshound1 digitalmars.com> said:

 Don wrote:
 This is for me the last remaining D2 issue.
That would make it difficult to do things like: int*[] foo(int *p) { return [p, p + 1]; } as all the elements of the literal would also have to be immutable. I think you've made a good case, but there is also this issue.
One thing I like to do with array literals is use them for appending several elements at once: array ~= [1, a+1, b+1]; This way I avoid multiple array appends so it should be faster, but if that temporary array literal gets allocated on the heap then it's rather counterproductive for me to use array literals for this. Could cases like this, where the array never escape, be allocated on the stack? Just like local variables get allocated on the heap when they escape through a closure, array literals could be allocated on the heap when they escape through a return or some other mean. They could also be part of the program's static data when the literal expression is constant and assigned to an immutable(T)[]. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 17 2010
parent Walter Bright <newshound1 digitalmars.com> writes:
Michel Fortin wrote:
 Could cases like this, where the array never escape, be allocated on the 
 stack?
Yes, the compiler can do that.
Feb 17 2010
prev sibling next sibling parent reply grauzone <none example.net> writes:
IMHO array literals should be static arrays, which are value types. No 
issues with heap allocation or immutability requirements.

Also, even immutable values can change at runtime:

void foo(immutable int x) {
	auto array = [x];
}

array would have needed to be heap allocated even if they are changed to 
be immutable.

Don wrote:
 (2) Concurrency issues make (1) even more important. It ought to 
 possible to pass an array (defined at compile time) as a message.
This sounds very special. Any example where you'd pass an array known at compile time as message?
 (3) Performance of the existing array 'literals' is absolutely 
 appalling. I doubled the speed of my entire app by changing ONE array 
 declaration from 'immutable [] xxx' into 'static const [] xxx'!!!
 Right now, if an array literal appears in your code, chances are it's a 
 performance bug. It's deceptive that there's a hidden heap allocation 
 happening in every array literal. It's got no business being in a 
 systems language IMHO.
Delegates also can have hidden allocation (basically, you have to guess the compiler's guess whether a delegate is a real closure). There are several language elements that may or may not trigger additional memory allocations (like setting .length or the ~= operator). And the underlying problem is the bad GC that is torturing us all, isn't it?
Feb 17 2010
parent reply Don <nospam nospam.com> writes:
grauzone wrote:
 IMHO array literals should be static arrays, which are value types. No 
 issues with heap allocation or immutability requirements.
That still wouldn't solve the performance problems.
 Don wrote:
 (2) Concurrency issues make (1) even more important. It ought to 
 possible to pass an array (defined at compile time) as a message.
This sounds very special. Any example where you'd pass an array known at compile time as message?
There's one in Andrei's concurrency chapter in TDPL!
 (3) Performance of the existing array 'literals' is absolutely 
 appalling. I doubled the speed of my entire app by changing ONE array 
 declaration from 'immutable [] xxx' into 'static const [] xxx'!!!
 Right now, if an array literal appears in your code, chances are it's 
 a performance bug. It's deceptive that there's a hidden heap 
 allocation happening in every array literal. It's got no business 
 being in a systems language IMHO.
Delegates also can have hidden allocation (basically, you have to guess the compiler's guess whether a delegate is a real closure). There are several language elements that may or may not trigger additional memory allocations (like setting .length or the ~= operator). And the underlying problem is the bad GC that is torturing us all, isn't it?
No, this one has nothing to do with the GC. The problem is that there's no way of saying 'this array literal is not a variable'.
Feb 17 2010
parent grauzone <none example.net> writes:
Don wrote:
 grauzone wrote:
 IMHO array literals should be static arrays, which are value types. No 
 issues with heap allocation or immutability requirements.
That still wouldn't solve the performance problems.
You mean even for a static array on the stack, the time to construct it would impose a performance problem? If that's the case, I'd say the optimizer should be able to tell that the array data actually is constant, and compile the code as if the array was immutable. I assume the array being a value type on the stack makes it much easier for the optimizer than a heap allocated array with unknown length. Is that impossible?
Feb 17 2010
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Don wrote:
 This is for me the last remaining D2 issue.
 I've mentioned this several times before. Mutable array 'literals' are 
 wrong on so many levels. Here are some of the issues, listed in 
 decreasing importance.
[snip] I agree. How do you see we should change the language to fix things? Andrei
Feb 17 2010
parent reply Michael Rynn <michaelrynn optusnet.com.au> writes:
On Wed, 17 Feb 2010 07:49:09 -0600, Andrei Alexandrescu wrote:

 Don wrote:
 This is for me the last remaining D2 issue. I've mentioned this several
 times before. Mutable array 'literals' are wrong on so many levels.
 Here are some of the issues, listed in decreasing importance.
[snip] I agree. How do you see we should change the language to fix things? Andrei
I am trying to improve my understanding here by reasoning some of it out a bit, and trying to figure out what is everyone elses probably implicitly understands. Start looking at DPL C heritage, what is the behaviour of array literals in C? What does a good C compiler do? Whenever we put something like this in code, Elem* = [ Piece of literal data ] The compiler has created a representation of the data which of these? 1. pointer to protected read - only memory? (Probably would declare const) 2. returned a pointer to a writeable static memory location in the code- data image, which is start of the array data. If so then the data can be changed but not extended in size without reallocation. 3. allocated a copy of the original data, location known only to compiler, to heap or stack allocated memory block, and assigned to the pointer variable. How does this vary with static and stack context? I would like to (wishfully) think that what happens is that it looks like the code text in my editor. The value of the literal in my source code after being assigned is not destroyed by further manipulation of Elem*. It still exists in the original source, so it should still exist in the binary image, and can be recalled again. So thats what I would define as the "immutable source". This happens now for string type, they are source immutable. Changes can be only done to a mutable copy. I have got used to that. So what would really break if this was true of all array literals? How hard is it to do in the current D2 compiler? How does it affect the runtime typeinfo system? (I am still getting used to that). Has anyone tried this in an experimental version of D2? I suppose we can always write code that can copy from immutable source where necessary? Is it possible to be able make mutable copies more easily? Is there anything that is really awkward in D2 working with immutable string type, that cannot be done without resorting to mutable character literals? If not, then the principle can be applied to all "immutable source". But for those who would like mutable static memory initialised for some , might there be a way of telling the compiler that we wish to abuse the memory image? char[] = cast(mutable(char)[]) "I can overwrite this initialization string for all time"; // points to original binary image char[] = "I can overwrite this source string".dup; The example given :- int*[] foo(int *p) { return [p, p + 1]; } Why would you make such a function? Why not make it locally without the function call overhead? int *p = &something; ... int*[] mypointers = [p, p+1]; // this should not be a literal, but memory allocated like a if its on the stack filled in dynamically at point of execution, and not immutable? p, p+1 are the initialisation values. int*[] mypointers = [&static1, &static2]; // this looks like a literal, known at compile time. but could also be memory filled in dynamically at point of execution. If mypointers is static then it is immutable?. How does it work for struct? Michael Rynn...
Feb 17 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-02-17 20:07:20 -0500, Michael Rynn <michaelrynn optusnet.com.au> said:

 Is there anything that is really awkward in D2 working with immutable
 string type, that cannot be done without resorting to mutable character
 literals?   If not, then the principle can be applied to all "immutable
 source".
Well, array literals are not always constants. Take this for instance: int[] array = [x, y, z]; This array literal does not have an "immutable source" as you call it. Should it really require a ".dup" to get a mutable array? How efficient would it be? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 17 2010
parent reply Don <nospam nospam.com> writes:
Michel Fortin wrote:
 On 2010-02-17 20:07:20 -0500, Michael Rynn <michaelrynn optusnet.com.au> 
 said:
 
 Is there anything that is really awkward in D2 working with immutable
 string type, that cannot be done without resorting to mutable character
 literals?   If not, then the principle can be applied to all "immutable
 source".
Well, array literals are not always constants. Take this for instance: int[] array = [x, y, z]; This array literal does not have an "immutable source" as you call it.
This is the issue. The syntax sugar you get from not requiring an "immutable source" comes at a very high price.
 Should it really require a ".dup" to get a mutable array? How efficient 
 would it be?
What it does at the moment is insert a hidden .dup at all times. And the performance is terrible. Really, really terrible. Roughly 100 times worse than you expect. At the moment, [1, 2, 3] is basically transformed into something like makeArray(1,2,3);
Feb 18 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-02-18 03:10:47 -0500, Don <nospam nospam.com> said:

 Well, array literals are not always constants. Take this for instance:
 
     int[] array = [x, y, z];
 
 This array literal does not have an "immutable source" as you call it.
This is the issue. The syntax sugar you get from not requiring an "immutable source" comes at a very high price.
 Should it really require a ".dup" to get a mutable array? How efficient 
 would it be?
What it does at the moment is insert a hidden .dup at all times. And the performance is terrible. Really, really terrible. Roughly 100 times worse than you expect. At the moment, [1, 2, 3] is basically transformed into something like makeArray(1,2,3);
I agree the performance is terrible because of the often unnecessary heap allocation. But assigning a literal to an immutable array should be easy to special-case in the compiler so that it doesn't do an unnecessary copy. In fact, forcing literals to be assigned to immutable arrays only is the same special case, except it disallows the other case where it needs to do a copy (assigning to mutable). Non-constant array literals can also be optimized in some situations: when they don't escape the function's scope, they can be allocated on the stack. Better optimization could take advantage of that. Also, they're quite useful. How would you write the code above without them? int[] array; array.length = 3; array[0] = x; array[1] = y; array[2] = z; Not very attractive isn't it? And it wouldn't work when assigning to an array of immutable(int)[]. Also, I expect this'll make it harder for the compiler to optimize and allocate on the stack when it can. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 18 2010
parent reply Don <nospam nospam.com> writes:
Michel Fortin wrote:
 On 2010-02-18 03:10:47 -0500, Don <nospam nospam.com> said:
 
 Well, array literals are not always constants. Take this for instance:

     int[] array = [x, y, z];

 This array literal does not have an "immutable source" as you call it.
This is the issue. The syntax sugar you get from not requiring an "immutable source" comes at a very high price.
 Should it really require a ".dup" to get a mutable array? How 
 efficient would it be?
What it does at the moment is insert a hidden .dup at all times. And the performance is terrible. Really, really terrible. Roughly 100 times worse than you expect. At the moment, [1, 2, 3] is basically transformed into something like makeArray(1,2,3);
I agree the performance is terrible because of the often unnecessary heap allocation.
It's not just the heap allocation. The values are also recalculated.
 But assigning a literal to an immutable array should be 
 easy to special-case in the compiler so that it doesn't do an 
 unnecessary copy.
Yes, but that doesn't work in the general case. It cannot be done for anonymous literals. Also you can't force an array literal to be CTFEd.
 Also, they're quite useful. How would you write the code above without 
 them?
 
     int[] array;
     array.length = 3;
     array[0] = x;
     array[1] = y;
     array[2] = z;
Including the library implementation: T[] toArray(T)(T[] values...) { return values.dup; } int[] array = toArray(x,y,z); Mutable array literals achieve almost nothing.
Feb 18 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Feb 2010 08:00:55 -0500, Don <nospam nospam.com> wrote:

 Including the library implementation:

 T[] toArray(T)(T[] values...)
 {
      return values.dup;
 }

 int[] array = toArray(x,y,z);

 Mutable array literals achieve almost nothing.
I don't think this will work: toArray(1, 2L, 3.0) I'm not saying that this is a reason not to consider the change, but it does note a difference. You can always simply do toArray!double(...) to get around the limitation. -Steve
Feb 18 2010
parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Steven Schveighoffer wrote:
 On Thu, 18 Feb 2010 08:00:55 -0500, Don <nospam nospam.com> wrote:
 
 Including the library implementation:

 T[] toArray(T)(T[] values...)
 {
      return values.dup;
 }

 int[] array = toArray(x,y,z);

 Mutable array literals achieve almost nothing.
I don't think this will work: toArray(1, 2L, 3.0) I'm not saying that this is a reason not to consider the change, but it does note a difference. You can always simply do toArray!double(...) to get around the limitation. -Steve
import std.traits: CommonType; CommonType!(T)[] toArray(T...)(T elements) { ... } -Lars
Feb 18 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-02-18 08:00:55 -0500, Don <nospam nospam.com> said:

 I agree the performance is terrible because of the often unnecessary 
 heap allocation.
It's not just the heap allocation. The values are also recalculated.
Well, in the case of a constant literal expression they don't need to be recalculated, they could exist in the static data segment. If a constant array literal is assigned to an immutable array they don't even need to be copied. I trust the compiler will eventually do that (and sooner the better).
 But assigning a literal to an immutable array should be easy to 
 special-case in the compiler so that it doesn't do an unnecessary copy.
Yes, but that doesn't work in the general case. It cannot be done for anonymous literals.
I'm not sure what you mean by "anonymous literals". Aren't all literals anonymous?
 Also you can't force an array literal to be CTFEd.
I think that's a more general problem with CTFE. But you can force CTFE using an enum: enum value = ctfeFunction(); Wrap that in a template if you want more convenience: template ctfe(alias ctfeValue) { enum ctfe = ctfeValue; } auto variable = ctfe!(ctfeFunction());
 Also, they're quite useful. How would you write the code above without them?
 
     int[] array;
     array.length = 3;
     array[0] = x;
     array[1] = y;
     array[2] = z;
Including the library implementation: T[] toArray(T)(T[] values...) { return values.dup; } int[] array = toArray(x,y,z); Mutable array literals achieve almost nothing.
But now, how can the compiler optimize this and stack-allocate your array when it detect doesn't escape your function? You're explicitly telling it to duplicate it, even the compiler could decide it's not necessary. (I know stack-allocation isn't implemented, but Walter said it could be done.) Consider this case: int a, b, c; int[] array; array ~= [a, b, c]; array ~= toArray(a, b, c); Does it make sense to heap-allocate the mutable array? Hardly. With the literal, the compiler is free to optimize away the heap allocation, not so with toArray. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 18 2010
next sibling parent Don <nospam nospam.com> writes:
Michel Fortin wrote:

 But assigning a literal to an immutable array should be easy to 
 special-case in the compiler so that it doesn't do an unnecessary copy.
Yes, but that doesn't work in the general case. It cannot be done for anonymous literals.
I'm not sure what you mean by "anonymous literals". Aren't all literals anonymous?
No. If they are function parameters, they are anonymous. foo([1,2,3]); As opposed to: immutable int [] x = [1,2,3]; where they have a name, a storage class, and a chance to put an 'immutable' in.
 Also you can't force an array literal to be CTFEd.
I think that's a more general problem with CTFE. But you can force CTFE using an enum: enum value = ctfeFunction();
Unfortunately, that doesn't work. (1) Enum arrays aren't stored in the executable. You can't index them. (currently you can due, to a compiler bug, and it generates wrong code). (2) Using 'enum' doesn't work for the anonymous case.
 
 Wrap that in a template if you want more convenience:
 
     template ctfe(alias ctfeValue) {
         enum ctfe = ctfeValue;
     }
Expressions can be aliased. So I doubt that will work.
     auto variable = ctfe!(ctfeFunction());
 
 
 Also, they're quite useful. How would you write the code above 
 without them?

     int[] array;
     array.length = 3;
     array[0] = x;
     array[1] = y;
     array[2] = z;
Including the library implementation: T[] toArray(T)(T[] values...) { return values.dup; } int[] array = toArray(x,y,z); Mutable array literals achieve almost nothing.
But now, how can the compiler optimize this and stack-allocate your array when it detect doesn't escape your function?
That's *exactly* the way it works now. Really, there's no difference. The toArray function is just a single return statement, so it gets inlined perfectly. And .dup is a compiler built-in. So it's easy for the compiler. The compiler sees: int [] array = [x, y, z].dup; (where [x, y, z] is the existing mutable array syntax).
Feb 18 2010
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 18 Feb 2010 18:42:16 +0300, Michel Fortin
<michel.fortin michelf.com> wrote:

 On 2010-02-18 08:00:55 -0500, Don <nospam nospam.com> said:

 I agree the performance is terrible because of the often unnecessary  
 heap allocation.
It's not just the heap allocation. The values are also recalculated.
Well, in the case of a constant literal expression they don't need to be recalculated, they could exist in the static data segment. If a constant array literal is assigned to an immutable array they don't even need to be copied. I trust the compiler will eventually do that (and sooner the better).
 But assigning a literal to an immutable array should be easy to  
 special-case in the compiler so that it doesn't do an unnecessary copy.
Yes, but that doesn't work in the general case. It cannot be done for anonymous literals.
I'm not sure what you mean by "anonymous literals". Aren't all literals anonymous?
 Also you can't force an array literal to be CTFEd.
I think that's a more general problem with CTFE. But you can force CTFE using an enum: enum value = ctfeFunction(); Wrap that in a template if you want more convenience: template ctfe(alias ctfeValue) { enum ctfe = ctfeValue; } auto variable = ctfe!(ctfeFunction());
 Also, they're quite useful. How would you write the code above without  
 them?
      int[] array;
     array.length = 3;
     array[0] = x;
     array[1] = y;
     array[2] = z;
Including the library implementation: T[] toArray(T)(T[] values...) { return values.dup; } int[] array = toArray(x,y,z); Mutable array literals achieve almost nothing.
But now, how can the compiler optimize this and stack-allocate your array when it detect doesn't escape your function? You're explicitly telling it to duplicate it, even the compiler could decide it's not necessary. (I know stack-allocation isn't implemented, but Walter said it could be done.) Consider this case: int a, b, c; int[] array; array ~= [a, b, c]; array ~= toArray(a, b, c); Does it make sense to heap-allocate the mutable array? Hardly. With the literal, the compiler is free to optimize away the heap allocation, not so with toArray.
[a, b, c] could result in a static array. Then there wouldn't even be a need for toArray, just use more natural int[] arr = [a, b, c].dup; syntax
Feb 18 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Feb 2010 16:17:50 -0500, Denis Koroskin <2korden gmail.com>  
wrote:

 On Thu, 18 Feb 2010 18:42:16 +0300, Michel Fortin
 <michel.fortin michelf.com> wrote:

 Consider this case:

 	int a, b, c;
 	int[] array;
 	array ~= [a, b, c];
 	array ~= toArray(a, b, c);

 Does it make sense to heap-allocate the mutable array? Hardly. With the  
 literal, the compiler is free to optimize away the heap allocation, not  
 so with toArray.
[a, b, c] could result in a static array. Then there wouldn't even be a need for toArray, just use more natural int[] arr = [a, b, c].dup; syntax
That would be bad, T[] is implicitly casted from T[N]. Consider that you could easily escape stack data using this. In other words, the type system would allow the assignment without the dup. -Steve
Feb 18 2010
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Fri, 19 Feb 2010 00:46:05 +0300, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Thu, 18 Feb 2010 16:17:50 -0500, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Thu, 18 Feb 2010 18:42:16 +0300, Michel Fortin
 <michel.fortin michelf.com> wrote:

 Consider this case:

 	int a, b, c;
 	int[] array;
 	array ~= [a, b, c];
 	array ~= toArray(a, b, c);

 Does it make sense to heap-allocate the mutable array? Hardly. With  
 the literal, the compiler is free to optimize away the heap  
 allocation, not so with toArray.
[a, b, c] could result in a static array. Then there wouldn't even be a need for toArray, just use more natural int[] arr = [a, b, c].dup; syntax
That would be bad, T[] is implicitly casted from T[N]. Consider that you could easily escape stack data using this. In other words, the type system would allow the assignment without the dup. -Steve
I don't think so. First of all, it is consistent with current behavior: int[] foo() { int[3] staticArray = [0, 1, 2]; int[] dynamicArray = staticArray; return dynamicArray; } int[] a = foo(); writeln(a); // prints garbage Second, it's very useful to prevent unnecessary heap allocations. I would prefer static arrays not to cast to dynamic ones implicitly, but using opSlice syntax instead: int[] arr = [0, 1, 2]; // error int[] arr = [0, 1, 2][]; // fine, use on your own risk
Feb 18 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Feb 2010 20:47:16 -0500, Denis Koroskin <2korden gmail.com>  
wrote:

 On Fri, 19 Feb 2010 00:46:05 +0300, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:
 That would be bad,  T[] is implicitly casted from T[N].  Consider that  
 you could easily escape stack data using this.  In other words, the  
 type system would allow the assignment without the dup.

 -Steve
I don't think so. First of all, it is consistent with current behavior: int[] foo() { int[3] staticArray = [0, 1, 2]; int[] dynamicArray = staticArray; return dynamicArray; }
Here is the case I'd say looks bad to me: int[] foo() { int[] dynamicArray = [0, 1, 2]; return dynamicArray; } If [0, 1, 2] is a static array, then it escapes its scope and corruption ensues. The implicit static array is the problem, not the assigning of a dynamic array from a static array. The literal doesn't look like it's a static array.
 Second, it's very useful to prevent unnecessary heap allocations.
This is already available by using a static array explicitly. The question is, what should be the default? I think making it immutable and non-static is the right choice because it can always escape scope, and does not require an allocation. From there you can do either a static array if you know it won't escape scope, or a dynamic array (via dup) if you need to modify it and it will escape scope.
 I would prefer static arrays not to cast to dynamic ones implicitly, but  
 using opSlice syntax instead:

 int[] arr = [0, 1, 2]; // error
 int[] arr = [0, 1, 2][]; // fine, use on your own risk
This is just an annoyance error. What will happen is people will constantly just use the brackets to shut up the compiler without thinking about what it really means. -Steve
Feb 19 2010
next sibling parent reply grauzone <none example.net> writes:
Steven Schveighoffer wrote:
 On Thu, 18 Feb 2010 20:47:16 -0500, Denis Koroskin <2korden gmail.com> 
 wrote:
 
 On Fri, 19 Feb 2010 00:46:05 +0300, Steven Schveighoffer 
 <schveiguy yahoo.com> wrote:
 That would be bad,  T[] is implicitly casted from T[N].  Consider 
 that you could easily escape stack data using this.  In other words, 
 the type system would allow the assignment without the dup.

 -Steve
I don't think so. First of all, it is consistent with current behavior: int[] foo() { int[3] staticArray = [0, 1, 2]; int[] dynamicArray = staticArray; return dynamicArray; }
Here is the case I'd say looks bad to me: int[] foo() { int[] dynamicArray = [0, 1, 2]; return dynamicArray; } If [0, 1, 2] is a static array, then it escapes its scope and corruption ensues. The implicit static array is the problem, not the assigning of a dynamic array from a static array. The literal doesn't look like it's a static array.
SafeD will take care of it. As the compiler has to support SafeD anyway, the same mechanisms can be used to create warnings or even errors in normal D code. Don wants to make array literals immutable; that means you couldn't just assign them to int[] anyway. In both cases, you'd have to use .dup to use them as normal, mutable arrays. Array literals being static arrays would be very handy in other places. Wouldn't it be nice if you could just use them for small fixed size vectors (wasn't that one of the reasons to make static arrays value types)? Then you could write assert([1, 2] + [3, 4] == [4, 6]). No silly struct vector "classes" needed.
 Second, it's very useful to prevent unnecessary heap allocations.
This is already available by using a static array explicitly. The
Static arrays still suck a bit, although making them value types improved the situation. Inside functions, this still allocates an array on the heap and copies it into the static array: int[2] array = [1, 2]; I don't quite know why it can't just behaves as if "array" was declared on module level (or as, um, the other "static").
 question is, what should be the default?  I think making it immutable 
 and non-static is the right choice because it can always escape scope, 
 and does not require an allocation.  From there you can do either a 
 static array if you know it won't escape scope, or a dynamic array (via 
 dup) if you need to modify it and it will escape scope.
How would you "do" a static array from an immutable array? As I understood, immutable array literals would still require heap allocations sometimes. Because even runtime dependent values can be immutable. Actually, if array literals only worked for compile time values, array literals could be both immutable and static arrays allocated in the data segment.
 I would prefer static arrays not to cast to dynamic ones implicitly, 
 but using opSlice syntax instead:

 int[] arr = [0, 1, 2]; // error
 int[] arr = [0, 1, 2][]; // fine, use on your own risk
I agree. I wonder how static arrays will be handled in SafeD. Maybe just disallow using them as normal arrays? Require an explicit .dup? Anyway, it will be safe.
 This is just an annoyance error.  What will happen is people will 
 constantly just use the brackets to shut up the compiler without 
 thinking about what it really means.
Not really an argument, when array literals will be immutable, you could as well say: //the cast is only there to shut up the stupid dmd! int[] arr = cast(int[])([0, 1, 2]);
 -Steve
Feb 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 19 Feb 2010 10:22:15 -0500, grauzone <none example.net> wrote:

 Steven Schveighoffer wrote:
 On Thu, 18 Feb 2010 20:47:16 -0500, Denis Koroskin <2korden gmail.com>  
 wrote:

 On Fri, 19 Feb 2010 00:46:05 +0300, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:
 That would be bad,  T[] is implicitly casted from T[N].  Consider  
 that you could easily escape stack data using this.  In other words,  
 the type system would allow the assignment without the dup.

 -Steve
I don't think so. First of all, it is consistent with current behavior: int[] foo() { int[3] staticArray = [0, 1, 2]; int[] dynamicArray = staticArray; return dynamicArray; }
Here is the case I'd say looks bad to me: int[] foo() { int[] dynamicArray = [0, 1, 2]; return dynamicArray; } If [0, 1, 2] is a static array, then it escapes its scope and corruption ensues. The implicit static array is the problem, not the assigning of a dynamic array from a static array. The literal doesn't look like it's a static array.
SafeD will take care of it. As the compiler has to support SafeD anyway, the same mechanisms can be used to create warnings or even errors in normal D code.
I'm not sure where Andrei will draw the line for SafeD. SafeD still has to be UsefulD ;) But I think slicing a static array makes the compiler lose all knowledge that it is a scoped entity. Until scope (or something equivalent) is a proper type constructor, we will not have full escape analysis, just simple tricks and hacks that prevent certain errors. The other option is to disallow slicing static arrays in SafeD, which IMO is a mistake, but I can see validity in the argument behind it. But besides this, I want non-safe D still to be as safe as it can be!
 Don wants to make array literals immutable; that means you couldn't just  
 assign them to int[] anyway. In both cases, you'd have to use .dup to  
 use them as normal, mutable arrays.
That's fine, if I don't specify dup by accident, the compiler would complain. In Denis' proposal, the compiler doesn't complain it just silently compiles.
 Array literals being static arrays would be very handy in other places.  
 Wouldn't it be nice if you could just use them for small fixed size  
 vectors (wasn't that one of the reasons to make static arrays value  
 types)? Then you could write assert([1, 2] + [3, 4] == [4, 6]). No silly  
 struct vector "classes" needed.
Does this work with static arrays? I thought you needed to use the slicing notation. In any case, using literals in array operations would be nifty. I don't think you need to have array literals be statically sized to accomplish this.
 Second, it's very useful to prevent unnecessary heap allocations.
This is already available by using a static array explicitly. The
Static arrays still suck a bit, although making them value types improved the situation. Inside functions, this still allocates an array on the heap and copies it into the static array: int[2] array = [1, 2];
The allocation goes away if the literal is immutable. I think we all agree the current situation is sucky.
 I don't quite know why it can't just behaves as if "array" was declared  
 on module level (or as, um, the other "static").
Essentially, that is what an immutable array is. The difference is in type inference.
 question is, what should be the default?  I think making it immutable  
 and non-static is the right choice because it can always escape scope,  
 and does not require an allocation.  From there you can do either a  
 static array if you know it won't escape scope, or a dynamic array (via  
 dup) if you need to modify it and it will escape scope.
How would you "do" a static array from an immutable array? As I understood, immutable array literals would still require heap allocations sometimes. Because even runtime dependent values can be immutable.
Requiring array literals to be immutable prevents you from including runtime values. This would work fine and not allocate: int[3] arr = [1, 2, 3]; But your point is well taken. I remember you brought it up before, and I argued a while with someone about it to no avail (can't remember what or with whom I was arguing :) If array literals become immutable, being able to initialize static arrays easily from runtime values *without* heap allocation would be a very nice feature to have. Perhaps you could use variables in an array literal only if they are used to initialize statically sized arrays. This problem needs to be discussed if we are to get immutable array literals.
 I would prefer static arrays not to cast to dynamic ones implicitly,  
 but using opSlice syntax instead:

 int[] arr = [0, 1, 2]; // error
 int[] arr = [0, 1, 2][]; // fine, use on your own risk
I agree. I wonder how static arrays will be handled in SafeD. Maybe just disallow using them as normal arrays? Require an explicit .dup? Anyway, it will be safe.
I have a feeling that safeD will disallow taking the address of a local, which includes slicing a static array, but IMO this would be a huge deterrent from using safeD. Might as well call it slowD at that point :)
 This is just an annoyance error.  What will happen is people will  
 constantly just use the brackets to shut up the compiler without  
 thinking about what it really means.
Not really an argument, when array literals will be immutable, you could as well say: //the cast is only there to shut up the stupid dmd! int[] arr = cast(int[])([0, 1, 2]);
Yes, but a cast is a red flag. A simple appending of brackets is not. It is well established that casts are to let you do things that in most contexts are not a good idea. Brackets are not the same. Beside all these points, if array literals are static, then obtaining an immutable array literal requires a dup or a global. I don't think that's a good strategy. Another point is IFTI and auto type detection -- what is most useful as an automatic type for array literals? I'd say immutable arrays are more useful because they can be easily converted to all other types of arrays, and it is consistent with string literals. The only caveat as you brought up is using runtime values inside a literal to initialize a static array. I think this can be special-cased by the compiler, but I'm not sure. -Steve
Feb 19 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:
 On Fri, 19 Feb 2010 10:22:15 -0500, grauzone <none example.net> wrote:
 SafeD will take care of it. As the compiler has to support SafeD 
 anyway, the same mechanisms can be used to create warnings or even 
 errors in normal D code.
I'm not sure where Andrei will draw the line for SafeD. SafeD still has to be UsefulD ;)
Good point, but safe means safe. It's can't be a hint. Perhaps the biggest crisis between Walter and me consumed when he mentioned he wanted to make "safe" just a hint and still disable bounds checking in release mode, safe or not. He was kind enough to back off, which was very nice of him - I see no way in which I could accept a compromise on that particular matter. (By the way, much of the very fruitful collaboration between us has been caused by our ability to work out good compromises even though we have sometimes different viewpoints.) Statically-allocated arrays are not a difficult safety issue. We have two solutions - (a) have the compiler just disable conversions for stack-allocated T[N] to T[] in safe mode, (b) have the compiler perform its own flow analysis and transparently switch to dynamic allocation when needed, just like in the case of closures.
 But I think slicing a static array makes the compiler lose all knowledge 
 that it is a scoped entity.
I thought the same, but then I figured this works: int[10] a; int[5] b = a[0 .. 5]; Well it's off-topic, I know. I agree that the current situation needs addressing. Andrei
Feb 19 2010
prev sibling next sibling parent grauzone <none example.net> writes:
Steven Schveighoffer wrote:
 Don wants to make array literals immutable; that means you couldn't 
 just assign them to int[] anyway. In both cases, you'd have to use 
 .dup to use them as normal, mutable arrays.
That's fine, if I don't specify dup by accident, the compiler would complain. In Denis' proposal, the compiler doesn't complain it just silently compiles.
True enough, but you have this problem _anyway_, as Denis pointed out. And the solution will remove those safety problems for array literals as well. I don't know if the compiler has enough magic to detect the escaping reference by itself, but it would be great if that worked.
 Array literals being static arrays would be very handy in other 
 places. Wouldn't it be nice if you could just use them for small fixed 
 size vectors (wasn't that one of the reasons to make static arrays 
 value types)? Then you could write assert([1, 2] + [3, 4] == [4, 6]). 
 No silly struct vector "classes" needed.
Does this work with static arrays? I thought you needed to use the slicing notation. In any case, using literals in array operations would be nifty. I don't think you need to have array literals be statically sized to accomplish this.
Yes, right now you need []. I would hope some day it will work without. Array operations on static arrays would enable the compiler to emit a single SSE instruction for an operation. With dynamic arrays, the compiler has to insert additional instructions for boundary checks, unpacking the array elements, etc.; and it couldn't just allocate the full vector in SSE registers either.
 I don't quite know why it can't just behaves as if "array" was 
 declared on module level (or as, um, the other "static").
Essentially, that is what an immutable array is. The difference is in type inference.
Even today, the compiler could just do the right thing and allow "static" initializers even for stack allocated values. By the way, this is probably also the reason why nobody likes struct initializers. They work as static initializers only.
 If array literals become immutable, being able to initialize static 
 arrays easily from runtime values *without* heap allocation would be a 
 very nice feature to have.  Perhaps you could use variables in an array 
 literal only if they are used to initialize statically sized arrays.  
 This problem needs to be discussed if we are to get immutable array 
 literals.
Yes, but the compiler doesn't do that even today. Was it too hard to implement? Didn't Walter care enough? Or didn't he know of this special case?
 Beside all these points, if array literals are static, then obtaining an 
 immutable array literal requires a dup or a global.  I don't think 
The immutable array could be on the stack as well (it would have the type immutable(T[X]) for some integer X).
 that's a good strategy.  Another point is IFTI and auto type detection 
 -- what is most useful as an automatic type for array literals?  I'd say 
 immutable arrays are more useful because they can be easily converted to 
 all other types of arrays, and it is consistent with string literals.  
That's a good point.
 The only caveat as you brought up is using runtime values inside a 
 literal to initialize a static array.  I think this can be special-cased 
 by the compiler, but I'm not sure.
At least it wouldn't conflict with the semantics. But Don had this argument with efficiency and unobvious heal allocations. Immutable array literals don't solve this. Of course you could still force the arguments for array literals to be compile time values.
Feb 19 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-02-19 11:03:30 -0500, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 //the cast is only there to shut up the stupid dmd!
 int[] arr = cast(int[])([0, 1, 2]);
Yes, but a cast is a red flag. A simple appending of brackets is not. It is well established that casts are to let you do things that in most contexts are not a good idea. Brackets are not the same.
That's the theory. It doesn't quite work in practice because you sometime have to cast to convert from one type to another (float to int, base class to derived), or to make a literal of the type you want (especially with array literals). Many usages of cast are safe, many aren't. So seeing cast as a red flag doesn't really work, unfortunately. Even SafeD allows cast, it just restricts it to safe usages. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 19 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 19 Feb 2010 12:59:36 -0500, Michel Fortin  
<michel.fortin michelf.com> wrote:

 On 2010-02-19 11:03:30 -0500, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:

 //the cast is only there to shut up the stupid dmd!
 int[] arr = cast(int[])([0, 1, 2]);
Yes, but a cast is a red flag. A simple appending of brackets is not. It is well established that casts are to let you do things that in most contexts are not a good idea. Brackets are not the same.
That's the theory. It doesn't quite work in practice because you sometime have to cast to convert from one type to another (float to int, base class to derived), or to make a literal of the type you want (especially with array literals). Many usages of cast are safe, many aren't. So seeing cast as a red flag doesn't really work, unfortunately. Even SafeD allows cast, it just restricts it to safe usages.
There are some valid uses of casting, but cast is much more of a red flag than brackets. Brackets are used everywhere. Search your code for [] and I'm sure you'll find a lot more instances than casting. But I think aside from this, dynamic casting should really be a separate function, I wouldn't mind if it's functionality moved into the library. -Steve
Feb 19 2010
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Fri, 19 Feb 2010 17:51:26 +0300, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 The implicit static array is the problem, not the assigning of a dynamic  
 array from a static array.
Not quite true. Here is your example slightly modified: int[] foo() { int[3] staticArray; int[] dynamicArray = staticArray; return dynamicArray; } It doesn't involve any array literals, but the problem is still there.
 The literal doesn't look like it's a static array.
It really does to me. [a, b, c] is essentially 3 stack pushes and nothing else (which is the same as allocating stack space for a static array of 3 elements).
 This is already available by using a static array explicitly.
How would you initialize such a static array with 3 given elements (a, b and c). I certainly wouldn't love verbosity of int[3] array = void; array[0] = a; array[1] = b; array[2] = c;
 The question is, what should be the default?  I think making it  
 immutable and non-static is the right choice because it can always  
 escape scope, and does not require an allocation.
You can't make [a, b, c] immutable unless all of the a, b and c are immutable (or primitive types). For example, what type would that be: Object a = new Object(); Object b = new Object(); Object c = new Object(); auto array = [a, b, c]; It certainly can't be immutable(Object)[]. I believe it shouldn't be Object[] either (you can always make a dynamic array out of static but not vice versa). What bonuses would does Object[] provide over Object[3]? Object[] doesn't prevent but rather *encourages* escaping reference, because D returns static array by values: // Your preferred way: Object[] toArray(Object a, Object b, Object c) { return [a, b, c]; // Bug, escaping reference to stack data } // My suggestion Object[3] toArray(Object a, Object b, Object c) { return [a, b, c]; // Correct }
 I would prefer static arrays not to cast to dynamic ones implicitly,  
 but using opSlice syntax instead:

 int[] arr = [0, 1, 2]; // error
 int[] arr = [0, 1, 2][]; // fine, use on your own risk
This is just an annoyance error. What will happen is people will constantly just use the brackets to shut up the compiler without thinking about what it really means.
<sarcasm>Sure, just hide the bug without compiler worrying you about presence of one. </sarcasm> You suggestion encourages unsafe by default, which is bad bad bad. It just shouldn't compile. People should just use .dup if they need a dynamic array unless they *really* know what they are doing, and use []. It should never happen in SafeD, and probably not even allowed.
Feb 19 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 19 Feb 2010 10:32:29 -0500, Denis Koroskin <2korden gmail.com>  
wrote:

 On Fri, 19 Feb 2010 17:51:26 +0300, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 The implicit static array is the problem, not the assigning of a  
 dynamic array from a static array.
Not quite true. Here is your example slightly modified: int[] foo() { int[3] staticArray; int[] dynamicArray = staticArray; return dynamicArray; } It doesn't involve any array literals, but the problem is still there.
Yes, but my point was that initializing a dynamic array with a literal should not implicitly promote unsafe code. When someone uses a literal, they are expecting the type to match. A literal should be the most useful type for the range of types it initializes. I think immutable is better than a static because an implicit casting of a static to a dynamic array promotes memory corruption. An implicit conversion from a dynamic to a static does not.
 The literal doesn't look like it's a static array.
It really does to me. [a, b, c] is essentially 3 stack pushes and nothing else (which is the same as allocating stack space for a static array of 3 elements).
It looks like data to me, not stack data. When you do: int i = 1; does the 1 look like stack data? My point is, where it's stored is not implicit in its usage, so if it's stack data, it could be easily interpreted as not stack data. Storing it in the immutable data segment and typing it as immutable is the safest option.
 This is already available by using a static array explicitly.
How would you initialize such a static array with 3 given elements (a, b and c). I certainly wouldn't love verbosity of int[3] array = void; array[0] = a; array[1] = b; array[2] = c;
What I meant was compile-time values could be stored in the array. I agree storing variables in a stack array needs to be solved. A possible solution is to allow variables in array literals only when being assigned to or used as a static array. How do you solve this? immutable(int)[] i = [1, 2, 3]; // add an idup? The best thing here is to avoid a heap allocation, because immutable data made entirely from literals requiring a heap allocation is just as bad as the current situation. With literals being immutable, going to static array is possible. With literals being a static array, it is not possible.
 The question is, what should be the default?  I think making it  
 immutable and non-static is the right choice because it can always  
 escape scope, and does not require an allocation.
You can't make [a, b, c] immutable unless all of the a, b and c are immutable (or primitive types). For example, what type would that be: Object a = new Object(); Object b = new Object(); Object c = new Object(); auto array = [a, b, c];
Yes, it's a problem that needs to be solved. I'd say that literals that contain a runtime variable should be typed as static arrays, but I'm pretty sure Don doesn't like that. I'm going to start a new thread about this. I don't think this particular nuance is getting enough attention.
 It certainly can't be immutable(Object)[]. I believe it shouldn't be  
 Object[] either (you can always make a dynamic array out of static but  
 not vice versa). What bonuses would does Object[] provide over  
 Object[3]? Object[] doesn't prevent but rather *encourages* escaping  
 reference, because D returns static array by values:

 // Your preferred way:
 Object[] toArray(Object a, Object b, Object c)
 {
      return [a, b, c]; // Bug, escaping reference to stack data
 }
I didn't say I prefer it. I meant it shouldn't be this easy, even in unsafe D. To anticipate your next argument, I don't think requiring slicing brackets is difficult enough to prevent these problems also.
 // My suggestion
 Object[3] toArray(Object a, Object b, Object c)
 {
      return [a, b, c]; // Correct
 }

 I would prefer static arrays not to cast to dynamic ones implicitly,  
 but using opSlice syntax instead:

 int[] arr = [0, 1, 2]; // error
 int[] arr = [0, 1, 2][]; // fine, use on your own risk
This is just an annoyance error. What will happen is people will constantly just use the brackets to shut up the compiler without thinking about what it really means.
<sarcasm>Sure, just hide the bug without compiler worrying you about presence of one. </sarcasm>
It's not an imagined problem. Look at Java's checked exceptions for an example.
 You suggestion encourages unsafe by default, which is bad bad bad. It  
 just shouldn't compile. People should just use .dup if they need a  
 dynamic array unless they *really* know what they are doing, and use [].  
 It should never happen in SafeD, and probably not even allowed.
I think you miss out on a whole class of performance gains by preventing slicing of static arrays, but this really is an argument for what to do in SafeD. -Steve
Feb 19 2010
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
Michel Fortin Wrote:

 On 2010-02-17 04:17:09 -0500, Walter Bright <newshound1 digitalmars.com> said:
 
 Don wrote:
 This is for me the last remaining D2 issue.
That would make it difficult to do things like: int*[] foo(int *p) { return [p, p + 1]; } as all the elements of the literal would also have to be immutable. I think you've made a good case, but there is also this issue.
One thing I like to do with array literals is use them for appending several elements at once: array ~= [1, a+1, b+1]; This way I avoid multiple array appends so it should be faster, but if that temporary array literal gets allocated on the heap then it's rather counterproductive for me to use array literals for this. Could cases like this, where the array never escape, be allocated on the stack?
Doesn't this do exactly that now? array ~= 1 ~ (a+1) ~ (b+1); -Steve
Feb 17 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-02-17 18:38:55 -0500, Steven Schveighoffer <schveiguy yahoo.com> said:

 One thing I like to do with array literals is use them for appending
 several elements at once:
 
 	array ~= [1, a+1, b+1];
 
 This way I avoid multiple array appends so it should be faster, but if
 that temporary array literal gets allocated on the heap then it's
 rather counterproductive for me to use array literals for this.
 
 Could cases like this, where the array never escape, be allocated on the stack?
Doesn't this do exactly that now? array ~= 1 ~ (a+1) ~ (b+1);
It doesn't compile: Error: Can only concatenate arrays, not (int ~ int). This works fine: array ~= [1] ~ (a+1) ~ (b+1); but now we have added back an array literal. :-) -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Feb 17 2010
prev sibling parent Daniel Murphy <yebbliesnospam gmail.com> writes:
Don Wrote:

 This is for me the last remaining D2 issue.
 I've mentioned this several times before. Mutable array 'literals' are 
 wrong on so many levels. Here are some of the issues, listed in 
 decreasing importance.
 
 (1) The language has no syntax for immutable array literals.
    [1,2,3] is mutable. You can created an immutable literal using the 
 awful 'static const' syntax, but that only works for named literals.
 
 (2) Concurrency issues make (1) even more important. It ought to 
 possible to pass an array (defined at compile time) as a message.
 
 (3) Performance of the existing array 'literals' is absolutely 
 appalling. I doubled the speed of my entire app by changing ONE array 
 declaration from 'immutable [] xxx' into 'static const [] xxx'!!!
 Right now, if an array literal appears in your code, chances are it's a 
 performance bug. It's deceptive that there's a hidden heap allocation 
 happening in every array literal. It's got no business being in a 
 systems language IMHO.
 
 (4) It is unintuitive. "hello" is immutable, but ['h','e','l','l','o'] 
 is not???
 
 (5) It's trivial (a one-liner) to make a function to create a mutable 
 array literal. We don't need it in the language.
 
 Mutable arrays literals are common in toy examples, but they've got very 
 little real-world relevance. By contrast, immutable array literals are 
 used in lookup tables all the time. They are extremely common. We need 
 to be able to express them.
 
 This is the only remaining thing in D2 which I'm certain is a big 
 mistake. Of course there's ugliness in various places, but it's possible 
 to ignore them or work around them. This is a primitive, 
 bread-and-butter issue with no workaround.
I agree. This has caused huge performance problems that were fixed by changing all my 'immutable x = []' to 'static immutable x = []'. Can we keep the current behaviour for when some of the values are not known at compile time? I know it's just sugar but it's very sweet. eg. auto y = [1, 2, x, y];
Feb 18 2010