www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - static array literal syntax request: auto x=[1,2,3]S;

reply "timotheecour" <thelastmammoth gmail.com> writes:
(apologies for cross-posting here, I feel this is a better place 
to ask than in my original post where I only received 1 answer 
that seemed in favor of this:
http://d.puremagic.com/issues/show_bug.cgi?id=8008 which was 2 
months ago).

Please see to the original post above to see the proposal for the 
new syntax for static array litterals
(but please reply here).

Running some simple tests show that this is not just cosmetics 
and ease of use but would also lead to significantly reduced 
overhead when dealing with static array litterals vs raw C array 
litterals, as currently D static array litterals perform costly 
heap allocation as intermediate step (as shown in the resulting 
assembly).

Here's 2 identical programs in C and D, which, compiled with all 
optimization flags, result in 20x speedup for the C version (D 
takes 17 sec with dmd -O -release -inline -noboundscheck main).

-----
//main.d
import std.stdio,std.conv;
void main(){
	size_t n=100000000,z=0;i=0,j=0;
	for(i=0;i<n;i++){
		size_t[10] a=[i,i+1,i+2,i+3,i+4,i+5,i+6,i+7,i+8,i+9];
		for(j=0;j<9;j++){z+=a[j];}
	}
	writeln(z);
}
-----
//main.c
#include <stdio.h>
int main(){
	size_t n=100000000,z=0;i=0,j=0;
	for(i=0;i<n;i++){
		size_t a[10]={i,i+1,i+2,i+3,i+4,i+5,i+6,i+7,i+8,i+9};
		for(j=0;j<9;j++){z+=a[j];}
	}
	printf("%lu\n",z);
	return 0;
}
-----
Note, the same D program modified as follows runs about as fast 
as the C program:
size_t[10] a=void; a[0]=i; a[1]=i+1; 
a[2]=i+2;a[3]=i+3;a[4]=i+4;a[5]=i+5;a[6]=i+6;a[7]=i+7;a[8]=i+8;a[9]=i+9;

Having the new syntax auto 
a=[i,i+1,i+2,i+3,i+4,i+5,i+6,i+7,i+8,i+9]S would prevent the 
intermediate heap allocation and should be at least as fast as 
the C version, possibly faster if SSE instructions are used.

Finally, someone suggested somewhere else 
(http://www.digitalmars.com/d/archives/digitalmars/D/bugs/Issue_8008_New_static_array_literal_syntax_request_auto_x
1_2_3_S_39709.html) 
that this could be done in library code:
auto a = [1, 2, 3].toStatic;
however I don't see how that could be achieved, as by the time 
the array reaches the toStatic function, it is a dynamic array 
whose length is not known at compile time.
Also, his suggested syntax auto[$] a=[1,2,3] is not as good in my 
opinion as it prevents one from directly passing an (anonymous) 
static array to a function, eg my_fun([1,2,3]S) vs: auto[$] 
a=[1,2,3]; my_fun(a);

Thanks for your comments!
Jun 09 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/10/2012 12:05 AM, timotheecour wrote:
 (apologies for cross-posting here, I feel this is a better place to ask
 than in my original post where I only received 1 answer that seemed in
 favor of this:
 http://d.puremagic.com/issues/show_bug.cgi?id=8008 which was 2 months ago).

 Please see to the original post above to see the proposal for the new
 syntax for static array litterals
 (but please reply here).

 Running some simple tests show that this is not just cosmetics and ease
 of use but would also lead to significantly reduced overhead when
 dealing with static array litterals vs raw C array litterals, as
 currently D static array litterals perform costly heap allocation as
 intermediate step (as shown in the resulting assembly).
 ...
D static array literals don't perform a costly heap allocation. It is simply a bug in the implementation. This is not a compelling reason to add new syntax.
Jun 09 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D doesn't _have_ static array literals. It only has dynamic array literals. int[5] a = [1, 2, 3, 4, 5]; should definitely be optimized such that it doesn't do any heap allocations, but the array literal itself is dynamic regardless, as typeof indicates. The other place that this causes problems is templated functions. e.g. void func(A)(A array) if(isStaticArray!A) {} func([1, 2, 3, 4, 5]); will fail to compile. You're forced to create one on the stack first. int[5] array = [1, 2, 3, 4, 5]; func(array); That _might_ merit adding a syntax for indicating that an array literal is static. However, in general, it's definitely true that the additional heap allocations that we currently see should just be optimized out by the compiler, and if that's all that we're trying to solve, then that's a matter of fixing the optimizer, not the language. - Jonathan M Davis
Jun 09 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.

 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap allocations,
 but the array literal itself is dynamic regardless, as typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
      if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array literal is
 static.
Yes, certainly. I was simply pointing out that arguing about performance makes a poor case here.
 However, in general, it's definitely true that the additional heap allocations
 that we currently see should just be optimized out by the compiler, and if
 that's all that we're trying to solve, then that's a matter of fixing the
 optimizer, not the language.
This is not about optimization. Allocating is simply incorrect. It is a 'wrong code' bug.
Jun 09 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 10-06-2012 01:02, Timon Gehr wrote:
 On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.

 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap
 allocations,
 but the array literal itself is dynamic regardless, as typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
 if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
Yes, certainly. I was simply pointing out that arguing about performance makes a poor case here.
 However, in general, it's definitely true that the additional heap
 allocations
 that we currently see should just be optimized out by the compiler,
 and if
 that's all that we're trying to solve, then that's a matter of fixing the
 optimizer, not the language.
This is not about optimization. Allocating is simply incorrect. It is a 'wrong code' bug.
I'd have to agree. There needs to be strong guarantees about this, not just "the compiler can/might optimize this". -- Alex Rønne Petersen alex lycus.org http://lycus.org
Jun 09 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 10, 2012 01:02:40 Timon Gehr wrote:
 On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.
 
 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap
 allocations, but the array literal itself is dynamic regardless, as
 typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
Just do writlen(typeof([1, 2, 3, 4, 5]).stringof); It'll be int[], not int[5]. I don't believe that there is _any_ case where an array literal is considered static. The closest that you get is that it will implicitly conert them to static arrays in statements such as int[5] a = [1, 2, 3, 4, 5]; but as evidenced by the extra heap allocation, it still creates the dynamic array before doing the conversion, and that exact same assignment works if the dynamic array isn't a literal. int[] a = [1, 2, 3, 4, 5]; int[5] b = a; I think that it's quite clear that the type system always considers array literals to be dynamic, which is why we're getting that extra heap allocation, and why in no case are they inferred to be static.
 This is not about optimization. Allocating is simply incorrect. It is a
 'wrong code' bug.
As far as I can tell, there's nothing wrong about it as far as the type system is concerned. Certainly, the compiler considers array literals to be dynamic, and that's why we're getting these issues. This behavior is _not_ desirable from the standpoint of efficiency, but as for as the type system goes, I don't see anything incorrect about it. - Jonathan M Davis
Jun 09 2012
parent "d1rulz" <d772767 rtrtr.com> writes:
writefln(typeof([1, 2, 3, 4, 5]).stringof) is int[5u] in D1 haha.
Jun 10 2012
prev sibling parent reply mta`chrono <chrono mta-international.net> writes:
Am 10.06.2012 01:02, schrieb Timon Gehr:
 On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.

 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap
 allocations,
 but the array literal itself is dynamic regardless, as typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
      if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
Yes, certainly. I was simply pointing out that arguing about performance makes a poor case here.
 However, in general, it's definitely true that the additional heap
 allocations
 that we currently see should just be optimized out by the compiler,
 and if
 that's all that we're trying to solve, then that's a matter of fixing the
 optimizer, not the language.
This is not about optimization. Allocating is simply incorrect. It is a 'wrong code' bug.
Yes, you're right. And you've also shown that we don't need a new syntax to accomplish this.
Jun 10 2012
parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 06/10/12 10:47, mta`chrono wrote:
 Am 10.06.2012 01:02, schrieb Timon Gehr:
 On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.

 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap
 allocations,
 but the array literal itself is dynamic regardless, as typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
An array literal is a dynamic array with a known constant length that implicitly converts to a static array of the same size and element type. That's a sane definition, and AFAICT this is how it's currently handled.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
      if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
I wasn't kidding about overloading 'static' for this; it would certainly be better that inventing yet another literal syntax.
 Yes, certainly. I was simply pointing out that arguing about performance
 makes a poor case here.

 However, in general, it's definitely true that the additional heap
 allocations
 that we currently see should just be optimized out by the compiler,
 and if
 that's all that we're trying to solve, then that's a matter of fixing the
 optimizer, not the language.
This is not about optimization. Allocating is simply incorrect. It is a 'wrong code' bug.
Yes, you're right. And you've also shown that we don't need a new syntax to accomplish this.
When you want to avoid the heap allocation, you can always use hacks such as: template static_array(alias da) { typeof(da[0])[da.length] static_array = da; } f(...) { int[3] a = static_array!([1,2,3]); ... } But the compiler should do this automatically in these cases, right now it doesn't. This hack won't of course work for the program mentioned in this thread. The isStaticArray!A problem is partially related to how the compiler handles arrays. auto f(T)(T a) { enum l = a.length; return l; } works with int[3] a = [1,2,3]; f(a); but fails with f([1,2,3]); And cases like auto f(E)(E[] a) { enum l = a.length; return l; } don't work at all. Making these things work, which should be possible, would help when dealing with static arrays, and also allow more optimization opportunities. artur
Jun 10 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/10/2012 04:54 PM, Artur Skawina wrote:
 On 06/10/12 10:47, mta`chrono wrote:
 Am 10.06.2012 01:02, schrieb Timon Gehr:
 On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.

 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap
 allocations,
 but the array literal itself is dynamic regardless, as typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
An array literal is a dynamic array with a known constant length that implicitly converts to a static array of the same size and element type. That's a sane definition,
It is not. It does not specify if/when allocations take place, for example.
 and AFAICT this is how it's currently handled.
It is not. void main(){ int[0] x = [1]; }
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
       if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
I wasn't kidding about overloading 'static' for this; it would certainly be better that inventing yet another literal syntax.
Using 'static' is "inventing yet another literal syntax" as well.
 Yes, certainly. I was simply pointing out that arguing about performance
 makes a poor case here.

 However, in general, it's definitely true that the additional heap
 allocations
 that we currently see should just be optimized out by the compiler,
 and if
 that's all that we're trying to solve, then that's a matter of fixing the
 optimizer, not the language.
This is not about optimization. Allocating is simply incorrect. It is a 'wrong code' bug.
Yes, you're right. And you've also shown that we don't need a new syntax to accomplish this.
When you want to avoid the heap allocation, you can always use hacks such as: template static_array(alias da) { typeof(da[0])[da.length] static_array = da; } f(...) { int[3] a = static_array!([1,2,3]); ... } But the compiler should do this automatically in these cases,
The compiler should do the right thing, not automatically apply a hack that only works for CTFEable right hand sides.
 right now it
 doesn't. This hack won't of course work for the program mentioned in this
 thread.

 The isStaticArray!A problem is partially related to how the compiler handles
 arrays.

     auto f(T)(T a) { enum l = a.length; return l; }

 works with

     int[3] a = [1,2,3];
     f(a);

 but fails with

     f([1,2,3]);

 And cases like

     auto f(E)(E[] a) { enum l = a.length; return l; }

 don't work at all.

 Making these things work, which should be possible,
f([1,2,3]); // f!(int[]) f([1,2,3,4]); // f!(int[]) int[3] a = [1,2,3]; f(a); // f!(int[3]) f(cast(int[3])[1,2,3]); // f!(int[3]) f(cast(int[4])[1,2,3,4]); // f!(int[4])
 would help when dealing
 with static arrays, and also allow more optimization opportunities.

 artur
I don't see it helping optimization.
Jun 10 2012
parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 06/10/12 18:02, Timon Gehr wrote:
 On 06/10/2012 04:54 PM, Artur Skawina wrote:
 On 06/10/12 10:47, mta`chrono wrote:
 Am 10.06.2012 01:02, schrieb Timon Gehr:
 On 06/10/2012 12:34 AM, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 00:15:01 Timon Gehr wrote:
 D static array literals don't perform a costly heap allocation. It is
 simply a bug in the implementation. This is not a compelling reason to
 add new syntax.
D
DMD
 doesn't _have_ static array literals. It only has dynamic array literals.

 int[5] a = [1, 2, 3, 4, 5];
This is a static array literal.
 should definitely be optimized such that it doesn't do any heap
 allocations,
 but the array literal itself is dynamic regardless, as typeof indicates.
I don't see a typeof there. The array literal is a static array literal if it is assigned/converted to a static array.
An array literal is a dynamic array with a known constant length that implicitly converts to a static array of the same size and element type. That's a sane definition,
It is not. It does not specify if/when allocations take place, for example.
Implementation details. Would you like to forbid "unnecessary" allocations? Not doing this should be harmless as it only affects performance, but maybe the hidden heap allocations should indeed be disallowed. Any decent compiler won't be emitting them anyway.
 and AFAICT this is how it's currently handled.
It is not. void main(){ int[0] x = [1]; }
Try running that, in non-release mode. :) The fact that the compiler accepts it even in the cases where it should be able to statically figure out that it will fail at runtime isn't ideal, but this is just a quality of implementation issue.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
       if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
I wasn't kidding about overloading 'static' for this; it would certainly be better that inventing yet another literal syntax.
Using 'static' is "inventing yet another literal syntax" as well.
In a way - yes. But i think "static[...]" would be much better than "[...]S".
 Yes, certainly. I was simply pointing out that arguing about performance
 makes a poor case here.

 However, in general, it's definitely true that the additional heap
 allocations
 that we currently see should just be optimized out by the compiler,
 and if
 that's all that we're trying to solve, then that's a matter of fixing the
 optimizer, not the language.
This is not about optimization. Allocating is simply incorrect. It is a 'wrong code' bug.
Yes, you're right. And you've also shown that we don't need a new syntax to accomplish this.
When you want to avoid the heap allocation, you can always use hacks such as: template static_array(alias da) { typeof(da[0])[da.length] static_array = da; } f(...) { int[3] a = static_array!([1,2,3]); ... } But the compiler should do this automatically in these cases,
The compiler should do the right thing, not automatically apply a hack that only works for CTFEable right hand sides.
Obviously. I'm just showing a workaround that lets you avoid the useless heap allocation in certain cases, until the compiler learns to do the right thing.
 right now it
 doesn't. This hack won't of course work for the program mentioned in this
 thread.

 The isStaticArray!A problem is partially related to how the compiler handles
 arrays.

     auto f(T)(T a) { enum l = a.length; return l; }

 works with

     int[3] a = [1,2,3];
     f(a);

 but fails with

     f([1,2,3]);

 And cases like

     auto f(E)(E[] a) { enum l = a.length; return l; }

 don't work at all.

 Making these things work, which should be possible,
f([1,2,3]); // f!(int[]) f([1,2,3,4]); // f!(int[]) int[3] a = [1,2,3]; f(a); // f!(int[3]) f(cast(int[3])[1,2,3]); // f!(int[3]) f(cast(int[4])[1,2,3,4]); // f!(int[4])
I'm not sure what your point is here. Mine is that 'length' is known in all these cases at compile time. Yet it's only possible to retrieve it when the argument is a real static array. It should also be possible when the function is called with a literal. Yes, it's just for templates and would need (hidden) specialization/cloning. Which sounds worse than it is - because it can happen after inlining.
 would help when dealing
 with static arrays, and also allow more optimization opportunities.
 I don't see it helping optimization.
See above. artur
Jun 10 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/10/2012 07:00 PM, Artur Skawina wrote:
 On 06/10/12 18:02, Timon Gehr wrote:
 On 06/10/2012 04:54 PM, Artur Skawina wrote:
 ...
 An array literal is a dynamic array with a known constant length that
 implicitly converts to a static array of the same size and element type.
 That's a sane definition,
It is not. It does not specify if/when allocations take place, for example.
Implementation details. Would you like to forbid "unnecessary" allocations? Not doing this should be harmless as it only affects performance,
Wrong. class S{ ~this() { int[3] a = [1,2,3]; } }
 but maybe
 the hidden heap allocations should indeed be disallowed. Any decent compiler
 won't be emitting them anyway.

 and AFAICT this is how it's currently handled.
It is not. void main(){ int[0] x = [1]; }
Try running that, in non-release mode. :)
(I never post valid D code that I haven't run.)
 The fact that the compiler accepts it even in the cases where it should
 be able to statically figure out that it will fail at runtime isn't ideal,
 but this is just a quality of implementation issue.
It is a bug. A static type system is pointless if it is not used for validating certain code properties.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
        if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
I wasn't kidding about overloading 'static' for this; it would certainly be better that inventing yet another literal syntax.
Using 'static' is "inventing yet another literal syntax" as well.
In a way - yes. But i think "static[...]" would be much better than "[...]S".
Obviously, none is much better than the other.
 ...
 The isStaticArray!A problem is partially related to how the compiler handles
 arrays.

      auto f(T)(T a) { enum l = a.length; return l; }

 works with

      int[3] a = [1,2,3];
      f(a);

 but fails with

      f([1,2,3]);

 And cases like

      auto f(E)(E[] a) { enum l = a.length; return l; }

 don't work at all.

 Making these things work, which should be possible,
f([1,2,3]); // f!(int[]) f([1,2,3,4]); // f!(int[]) int[3] a = [1,2,3]; f(a); // f!(int[3]) f(cast(int[3])[1,2,3]); // f!(int[3]) f(cast(int[4])[1,2,3,4]); // f!(int[4])
I'm not sure what your point is here.
A statically known length cannot be different in two identical instantiations.
 Mine is that 'length' is
 known in all these cases at compile time. Yet it's only possible
 to retrieve it when the argument is a real static array. It should
 also be possible when the function is called with a literal.
That is a case for introducing a dedicated static array literal syntax, not for butchering the template instantiation semantics. (if at all)
 Yes,
 it's just for templates and would need (hidden) specialization/cloning.
 Which sounds worse than it is - because it can happen after inlining.
It sounds bad because it is bad.
 would help when dealing
 with static arrays, and also allow more optimization opportunities.
 I don't see it helping optimization.
See above. artur
The optimizer can inline the call and use the additional information gained without language changes. What you want to achieve is better left to macros.
Jun 10 2012
parent Artur Skawina <art.08.09 gmail.com> writes:
On 06/10/12 19:32, Timon Gehr wrote:
 On 06/10/2012 07:00 PM, Artur Skawina wrote:
 On 06/10/12 18:02, Timon Gehr wrote:
 On 06/10/2012 04:54 PM, Artur Skawina wrote:
 ...
 An array literal is a dynamic array with a known constant length that
 implicitly converts to a static array of the same size and element type.
 That's a sane definition,
It is not. It does not specify if/when allocations take place, for example.
Implementation details. Would you like to forbid "unnecessary" allocations? Not doing this should be harmless as it only affects performance,
Wrong. class S{ ~this() { int[3] a = [1,2,3]; } }
I'm sure we agree that no allocation should take place here, just a memcpy, which would often be optimized to only a few stores.
 but maybe
 the hidden heap allocations should indeed be disallowed. Any decent compiler
 won't be emitting them anyway.
Your example is a good case for explicitly banning them, I agree.
 and AFAICT this is how it's currently handled.
It is not. void main(){ int[0] x = [1]; }
Try running that, in non-release mode. :)
(I never post valid D code that I haven't run.)
Then I'm not sure what you're trying to say; that program will throw an exception at runtime. That the compiler is buggy by compiling it?
 The fact that the compiler accepts it even in the cases where it should
 be able to statically figure out that it will fail at runtime isn't ideal,
 but this is just a quality of implementation issue.
It is a bug. A static type system is pointless if it is not used for validating certain code properties.
True. The way i see this -- there are so many large problems and holes in both the language and compiler that worrying about cases like this one not being caught at compile-time is premature. Sure, eventually it needs to be handled better, but right now there are bigger issues. And my comment wrt the compiler wasn't necessarily fair - it seems to do a very decent job in most cases, front-end wise; most issues are not with the implementation, but with the definition/specification.
 The
 other place that this causes problems is templated functions. e.g.

 void func(A)(A array)
        if(isStaticArray!A)
 {}

 func([1, 2, 3, 4, 5]);

 will fail to compile. You're forced to create one on the stack first.

 int[5] array = [1, 2, 3, 4, 5];
 func(array);
func(cast(int[5])[1,2,3,4,5]);
 That _might_ merit adding a syntax for indicating that an array
 literal is
 static.
I wasn't kidding about overloading 'static' for this; it would certainly be better that inventing yet another literal syntax.
Using 'static' is "inventing yet another literal syntax" as well.
In a way - yes. But i think "static[...]" would be much better than "[...]S".
Obviously, none is much better than the other.
 ...
 The isStaticArray!A problem is partially related to how the compiler handles
 arrays.

      auto f(T)(T a) { enum l = a.length; return l; }

 works with

      int[3] a = [1,2,3];
      f(a);

 but fails with

      f([1,2,3]);

 And cases like

      auto f(E)(E[] a) { enum l = a.length; return l; }

 don't work at all.

 Making these things work, which should be possible,
f([1,2,3]); // f!(int[]) f([1,2,3,4]); // f!(int[]) int[3] a = [1,2,3]; f(a); // f!(int[3]) f(cast(int[3])[1,2,3]); // f!(int[3]) f(cast(int[4])[1,2,3,4]); // f!(int[4])
I'm not sure what your point is here.
A statically known length cannot be different in two identical instantiations.
Obviously. That's why I mentioned cloning them. This is done for non-templated functions that are called with known arguments already; all i'm suggesting is using a similar approach here. The 'length' property of an array literal can be viewed as known parameter. (and, yes, my solution requires for this be done at an earlier stage)
 Mine is that 'length' is
 known in all these cases at compile time. Yet it's only possible
 to retrieve it when the argument is a real static array. It should
 also be possible when the function is called with a literal.
That is a case for introducing a dedicated static array literal syntax, not for butchering the template instantiation semantics. (if at all)
A solution that requires the programmer to always remember to mark literals in a special way is not the best approach. It should be possible to call a library functions with both a "real" array and a literal and the syntax shouldn't be different. Especially as that "array" could be an enum. Or immutable variable set at compile time. Another approach would be using __traits, but the implications wouldn't really be much different...
 The optimizer can inline the call and use the additional information gained
without language changes.
What if i'd like to use a different algorithm for a certain array size? Right now, I can, but have to assign the literal to a temporary static array, before calling the function. This is the main issue, everything else is just a consequence of my proposed solution, more of a side-effect actually. The naive approach would be to convert the dynamic array literal to a static array, but that won't work, because static arrays are passed by value. Hmm, special-casing for const args and literals might work. Anyway, can you think of a better solution? Ie one, that does not involve introducing a new literal syntax?
 What you want to achieve is better left to macros.
I'd love to have macros, they are necessary for things like attributes and pragmas. But if macros are required for generic programming then something is wrong. artur
Jun 10 2012
prev sibling parent reply "Mehrdad" <wfunction hotmail.com> writes:
I honestly don't see the POINT of having a "dynamic array 
literal".

What's the point of making the literals dynamic?

They should all be static, and only converted to dynamic if 
necessary from the context.

But I really don't see the benefit of allocating them on the heap 
just because we can... perhaps someone can enlighten me?
Jun 10 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 10, 2012 23:23:57 Mehrdad wrote:
 I honestly don't see the POINT of having a "dynamic array
 literal".
 
 What's the point of making the literals dynamic?
 
 They should all be static, and only converted to dynamic if
 necessary from the context.
 
 But I really don't see the benefit of allocating them on the heap
 just because we can... perhaps someone can enlighten me?
In the vast majority of cases where an array literal is used, it's assigned to a dynamic array. And if you do something like auto arr = [1, 2, 3, 4, 5]; the expectation is that you'll get a dynamic array. So, it's quite natural to make array literals dynamic. Almost all arrays are dynamic. As annoying is it may be upon occasion to not be able to simply pass a literal to a templated function which you want to be taking a static array, it would be _far_ more annoying if it assumed that the literal were static. Almost all functions (templated or otherwise) which take arrays take dynamic ones, not static ones. And if an array literal were static, what would this do? int[] func(int[] a) { return a; } func([1, 2, 3, 4, 5]); would it slice the static array literal and therefore return a slice of a now invalid temporary, or would it allocate the array on the heap as it would now? With all other static arrays, you'd get a slice, which would be very much the _wrong_ thing to do here. So, making the literal static and then slicing it wuold be very bad, but making it static and then converting it to a dynamic array would be completely inconsistent with the normal case. About the only case where it's obvious what it should be doing is int[5] a = [1, 2, 3, 4, 5]; Pretty much the only reason that this causes any problems is the case where you want to initialize a static array with a literal. In virtually _all_ other cases, what you want is a dynamic array, and making it a static one would just cause problems in many of them. As such, this particular case should just be handled specially and optimize out the heap allocation rather than trying to alter how the literals themselves work. We already get that with strings. A string literal defaults to string, but you can pass it to a function taking wstring or dstring or assign it to a wstring or dstring variable. The compiler takes care of the conversion for you. So, assigning a static array with a literal would just be another case like that. But since what you almost always want is a dynamic array, making the literal itself static would be asking for a heap of trouble. - Jonathan M Davis
Jun 10 2012
next sibling parent reply "Mehrdad" <wfunction hotmail.com> writes:
Ugh... you keep on saying "on occasion" and "particular case", 
making it seem like it's such a rarity that it's not worth 
mentioning.



Regarding your examples: the rule is quite simple:

- Literals are static by default
- If they are to be assigned to a dynamic array, then make them 
dynamic instead



But I really don't understand what benefit you get by making them 
dynamic by DEFAULT... and your answer didn't tell me anything, 
other than "most of the time you're assigning them to dynamic 
arrays" (which tells me nothing about the benefits).
Jun 10 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/11/2012 12:28 AM, Mehrdad wrote:
 Ugh... you keep on saying "on occasion" and "particular case", making it
 seem like it's such a rarity that it's not worth mentioning.



 Regarding your examples: the rule is quite simple:

 - Literals are static by default
 - If they are to be assigned to a dynamic array, then make them dynamic
 instead



 But I really don't understand what benefit you get by making them
 dynamic by DEFAULT... and your answer didn't tell me anything, other
 than "most of the time you're assigning them to dynamic arrays" (which
 tells me nothing about the benefits).
Type deduction.
Jun 10 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 11, 2012 01:35:41 Timon Gehr wrote:
 On 06/11/2012 12:28 AM, Mehrdad wrote:
 Ugh... you keep on saying "on occasion" and "particular case", making it
 seem like it's such a rarity that it's not worth mentioning.
 
 
 
 Regarding your examples: the rule is quite simple:
 
 - Literals are static by default
 - If they are to be assigned to a dynamic array, then make them dynamic
 instead
 
 
 
 But I really don't understand what benefit you get by making them
 dynamic by DEFAULT... and your answer didn't tell me anything, other
 than "most of the time you're assigning them to dynamic arrays" (which
 tells me nothing about the benefits).
Type deduction.
Exactly. And if they need to be assigned to a static array, then the compiler can automatically do what it needs to do to avoid the extra heap allocation. - Jonathan M Davis
Jun 10 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
 Type deduction.
Exactly. And if they need to be assigned to a static array, then the compiler can automatically do what it needs to do to avoid the extra heap allocation. - Jonathan M Davis
"Type deduction"? o.O I don't understand... could someone give me an example of what would break if we used the rule I suggested?
Jun 10 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 11, 2012 02:08:53 Mehrdad wrote:
 Type deduction.
Exactly. And if they need to be assigned to a static array, then the compiler can automatically do what it needs to do to avoid the extra heap allocation. - Jonathan M Davis
"Type deduction"? o.O I don't understand... could someone give me an example of what would break if we used the rule I suggested?
Pretty much anything involving templates would break. For instance, auto found = find(arr, [1, 2]); wouldn't compile, because [1, 2] would be considered a static array, which isn't a range. - Jonathan M Davis
Jun 10 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Monday, 11 June 2012 at 00:16:48 UTC, Jonathan M Davis wrote:

 auto found = find(arr, [1, 2]);

 wouldn't compile, because [1, 2] would be considered a static 
 array, which
 isn't a range.

 - Jonathan M Davis
Uh... that's pretty much just saying, "code that explicitly depends on the current behavior would now break". Which is true, except that IMHO it's the code's fault. The problem is there right now /anyway/: int[2] items = [1, 2]; int[] arr = ...; find(arr, items); // "broken"... but, really, it _should_ work So if that's a problem, then it's obviously a problem with find() -- it should slice the input -- not with this proposal. Any other examples?
Jun 10 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 11, 2012 02:45:00 Mehrdad wrote:
 On Monday, 11 June 2012 at 00:16:48 UTC, Jonathan M Davis wrote:
 auto found = find(arr, [1, 2]);
 
 wouldn't compile, because [1, 2] would be considered a static
 array, which
 isn't a range.
 
 - Jonathan M Davis
Uh... that's pretty much just saying, "code that explicitly depends on the current behavior would now break". Which is true, except that IMHO it's the code's fault. The problem is there right now /anyway/: int[2] items = [1, 2]; int[] arr = ...; find(arr, items); // "broken"... but, really, it _should_ work So if that's a problem, then it's obviously a problem with find() -- it should slice the input -- not with this proposal.
Actually, having it slice the input is arguably a _bad_ idea. You risk stuff like int[] foo(int[] a) { return a; } int[] bar() { int[5] a = [1, 2, 3, 4, 5]; return foo(a); } b now refers to garbage. Honestly, I think that the fact that static arrays _ever_ slice implicitly was a _bad_ design decision and is just asking for trouble. If the programmer has to state it explicitly, then they always know that they're slicing it rather than copying it, and you reduce the risk of having slices to static arrays which don't exist anymore. IMHO, the fact that range-based functions don't work with static arrays without slicing them automatically is _good_ thing. And if all range-based functions were altered to work with static arrays, and we made array literals static, then this example would _still_ be broken auto found = find(arr, [1, 2]); But instead of giving a compilation error like it would if we just made array literal state, instead you're operating on garbage, and you get undefined behavior. If the compiler is changed so that it elides the heap allocation when assigning an array literal to a dynamic array, then that solves the problem quite cleanl. I don't see how changing array literals to be static would help anything. Instead of getting unnecessary heap allocations, you get functions that don't work with array literals and/or you end up operating on garbage. - Jonathan M Davis
Jun 10 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Monday, 11 June 2012 at 00:57:45 UTC, Jonathan M Davis wrote:
 And if all range-based functions were altered to work with 
 static arrays, and we made array literals static, then this 
 example would _still_ be broken
 auto found = find(arr, [1, 2]);
Sorry, I don't quite seem to follow. :\ Would you mind explaining why it's broken?
Jun 10 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 11, 2012 04:13:42 Mehrdad wrote:
 On Monday, 11 June 2012 at 00:57:45 UTC, Jonathan M Davis wrote:
 And if all range-based functions were altered to work with
 static arrays, and we made array literals static, then this
 example would _still_ be broken
 auto found = find(arr, [1, 2]);
Sorry, I don't quite seem to follow. :\ Would you mind explaining why it's broken?
Because then find would have to slice a static array, and that means that it risks returning a slice to a static array, which since the static array was a temporary guarantees that you would be returning a slice which pointed to garbage. The result is undefined behavior. Now, since in this particular case, what find returns won't ever include anything from any arguments other than the first, it'll be fine. But for any templated function which returns a portion of its argument, having it take a static array for that argument would be horribly broken. For instance, auto found = find([1, 2, 3, 4, 5], 3); _would_ definitely return a slice which referred to garbage were array literals static and find accepted them. - Jonathan M Davis
Jun 10 2012
next sibling parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Monday, 11 June 2012 at 02:25:54 UTC, Jonathan M Davis wrote:
 Now, since in this particular case, what find returns won't 
 ever include anything from any arguments other than the first, 
 it'll be fine.
Yes, that's precisely why your statement didn't make sense to me. :)
 But for any templated function which returns a portion of its 
 argument, having it take a static array for that argument would 
 be horribly broken. For instance,

 auto found = find([1, 2, 3, 4, 5], 3);

 _would_ definitely return a slice which referred to garbage 
 were array literals
static and find accepted them. Ah, that's a different example. :) Hm... could you automatically convert the static arrays to dynamic if the argument isn't 'scope'?
Jun 10 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 11, 2012 04:47:36 Mehrdad wrote:
 On Monday, 11 June 2012 at 02:25:54 UTC, Jonathan M Davis wrote:
 Now, since in this particular case, what find returns won't
 ever include anything from any arguments other than the first,
 it'll be fine.
Yes, that's precisely why your statement didn't make sense to me. :) :
 But for any templated function which returns a portion of its
 argument, having it take a static array for that argument would
 be horribly broken. For instance,
 
 auto found = find([1, 2, 3, 4, 5], 3);
 
 _would_ definitely return a slice which referred to garbage
 were array literals
static and find accepted them. Ah, that's a different example. :) Hm... could you automatically convert the static arrays to dynamic if the argument isn't 'scope'?
That would be at odds with how passing static arrays to functions works normally. Normally, they get sliced. But I don't get why you're so hung up on making array literals static. Having them be dynamic works great in every single case except when you specifically want to be assigning them to a static array. In that case, you get an unnecessary array allocation, which we definitely don't want. However, all it takes is for the compiler to see that you're assigning an array literal to a static array and skip the allocation. That particular use case is then fixed, and all of the rest of them continue to work just fine. What are you trying to fix beyond assignments to static arrays with array literals? - Jonathan M Davis
Jun 10 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Monday, 11 June 2012 at 03:05:36 UTC, Jonathan M Davis wrote:
 What are you trying to fix beyond assignments to static arrays 
 with array literals?
Just the fact that it would reduce unnecessary heap allocations. I'm not trying to 'fix' it though, as it's not broken. (Then again, neither is it right now.)
Jun 10 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 11, 2012 05:09:49 Mehrdad wrote:
 On Monday, 11 June 2012 at 03:05:36 UTC, Jonathan M Davis wrote:
 What are you trying to fix beyond assignments to static arrays
 with array literals?
Just the fact that it would reduce unnecessary heap allocations. I'm not trying to 'fix' it though, as it's not broken. (Then again, neither is it right now.)
Well, the unnecessary allocations are eleminated if the compiler starts eliding them like it should when assigning an array literal to a static array. - Jonathan M Davis
Jun 10 2012
prev sibling parent reply travert phare.normalesup.org (Christophe Travert) writes:
Jonathan M Davis , dans le message (digitalmars.D:169705), a écrit :
 auto found = find([1, 2, 3, 4, 5], 3);
No problem if the rule could be the following: - array literals are static by default - array literals are copied to the heap when assigned to a dynamic array. - the former rule applies even if the array literal is assigned to a dynamic array via a function call, like it is the case in this example. - if a function can take both static and dynamic array as parameter, the static version takes precedence (it is always possible to call the dynamic version via slicing, at your own risk since the array is no longer valid at the end of the function call, or, much more wisely, by explicitely using the .dup property).
Jun 11 2012
parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 06/11/12 12:06, Christophe Travert wrote:
 Jonathan M Davis , dans le message (digitalmars.D:169705), a écrit :
 auto found = find([1, 2, 3, 4, 5], 3);
No problem if the rule could be the following: - array literals are static by default - array literals are copied to the heap when assigned to a dynamic array. - the former rule applies even if the array literal is assigned to a dynamic array via a function call, like it is the case in this example. - if a function can take both static and dynamic array as parameter, the static version takes precedence (it is always possible to call the dynamic version via slicing, at your own risk since the array is no longer valid at the end of the function call, or, much more wisely, by explicitely using the .dup property).
T f(T)(T a) {} artur
Jun 11 2012
parent travert phare.normalesup.org (Christophe Travert) writes:
Artur Skawina , dans le message (digitalmars.D:169717), a écrit :
 On 06/11/12 12:06, Christophe Travert wrote:
 Jonathan M Davis , dans le message (digitalmars.D:169705), a écrit :
 auto found = find([1, 2, 3, 4, 5], 3);
No problem if the rule could be the following: - array literals are static by default - array literals are copied to the heap when assigned to a dynamic array. - the former rule applies even if the array literal is assigned to a dynamic array via a function call, like it is the case in this example. - if a function can take both static and dynamic array as parameter, the static version takes precedence (it is always possible to call the dynamic version via slicing, at your own risk since the array is no longer valid at the end of the function call, or, much more wisely, by explicitely using the .dup property).
T f(T)(T a) {} artur
I don't see the problem. If T compiles with a static array, it returns a static array, there is no invalid pointer issue, since the returned value is not a slice, but a static array. -- Christophe
Jun 14 2012
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 10 Jun 2012 20:08:53 -0400, Mehrdad <wfunction hotmail.com> wrote:

 Type deduction.
Exactly. And if they need to be assigned to a static array, then the compiler can automatically do what it needs to do to avoid the extra heap allocation. - Jonathan M Davis
"Type deduction"? o.O I don't understand... could someone give me an example of what would break if we used the rule I suggested?
Anything that uses type deduction? Most of the time, you do *not* want a static array, especially for template functions that use IFTI. Note that D1 was like this, [1,2,3] was auto-typed to int[3u]. It was a constant source of pain that I would not like to revisit. Especially since static arrays are now passed by value. -Steve
Jun 11 2012
parent travert phare.normalesup.org (Christophe Travert) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:169718), a
 écrit :
 Note that D1 was like this, [1,2,3] was auto-typed to int[3u].  It was a  
 constant source of pain that I would not like to revisit.  Especially  
 since static arrays are now passed by value.
 
 -Steve
Since static are passed by value, static arrays are now safe to use. This should decrease the pain a lot.
Jun 14 2012
prev sibling parent reply Don Clugston <dac nospam.com> writes:
On 10/06/12 23:43, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 23:23:57 Mehrdad wrote:
 I honestly don't see the POINT of having a "dynamic array
 literal".

 What's the point of making the literals dynamic?

 They should all be static, and only converted to dynamic if
 necessary from the context.

 But I really don't see the benefit of allocating them on the heap
 just because we can... perhaps someone can enlighten me?
In the vast majority of cases where an array literal is used, it's assigned to a dynamic array.
I doubt that very much. I know it's not true in my code, I use array literals almost exclusively for immutable values. Usually if you are initializing an array, where you will modify the elements later, you want all values to be the same. I argued that array literals should be immutable, just as string literals are. But I lost.
Jun 15 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, June 15, 2012 09:09:48 Don Clugston wrote:
 On 10/06/12 23:43, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 23:23:57 Mehrdad wrote:
 I honestly don't see the POINT of having a "dynamic array
 literal".
 
 What's the point of making the literals dynamic?
 
 They should all be static, and only converted to dynamic if
 necessary from the context.
 
 But I really don't see the benefit of allocating them on the heap
 just because we can... perhaps someone can enlighten me?
In the vast majority of cases where an array literal is used, it's assigned to a dynamic array.
I doubt that very much. I know it's not true in my code, I use array literals almost exclusively for immutable values. Usually if you are initializing an array, where you will modify the elements later, you want all values to be the same. I argued that array literals should be immutable, just as string literals are. But I lost.
What does immutability have to do with static vs dynamic? immutable a = [0, 1, 2, 3]; results in an immutable(int[]), not immutable(int[4]). - Jonathan M Davis
Jun 15 2012
parent "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 15 June 2012 at 07:15:50 UTC, Jonathan M Davis wrote:
 On Friday, June 15, 2012 09:09:48 Don Clugston wrote:
 On 10/06/12 23:43, Jonathan M Davis wrote:
 On Sunday, June 10, 2012 23:23:57 Mehrdad wrote:
 I honestly don't see the POINT of having a "dynamic array
 literal".
 
 What's the point of making the literals dynamic?
 
 They should all be static, and only converted to dynamic if
 necessary from the context.
 
 But I really don't see the benefit of allocating them on 
 the heap
 just because we can... perhaps someone can enlighten me?
In the vast majority of cases where an array literal is used, it's assigned to a dynamic array.
I doubt that very much. I know it's not true in my code, I use array literals almost exclusively for immutable values. Usually if you are initializing an array, where you will modify the elements later, you want all values to be the same. I argued that array literals should be immutable, just as string literals are. But I lost.
What does immutability have to do with static vs dynamic? immutable a = [0, 1, 2, 3]; results in an immutable(int[]), not immutable(int[4]). - Jonathan M Davis
Just a guess here, but I think what Don meant was this: An immutable _literal_ is quite often "static __gshared" (it's pretty pointless to make it an instance member...), which means it's always in memory, which means it should be static (as opposed to dynamic), since slicing it wouldn't cause problems.
Jun 15 2012