www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Static arrays, typeof and IFTI?

reply Norbert Nemec <Norbert Nemec-online.de> writes:
Hi there,

after a very busy and eventful year in my personal life, I have now
finally found some time to play with D2. I am very impressed by the
progress!

One thing I noticed was that static arrays are somehow strangely limited:

It is possible to overload based on the length:

--------
void f(int[3] x) {
   writeln("int[3]: ",x);
}

void f(int[4] x) {
   writeln("int[4]: ",x);
}

int main(string argv[]) {
   f([1,2,3]);
   f([1,2,3,4]);
   return 0;
}
--------

However, used as function template argument, a static array is casted to
a dynamic array:

-----------
void g(T)(T x) {
   static assert (__traits(isStaticArray,T));
   enum N = T.init.length;
   writeln(N,": ",x);
}

int main(string argv[]) {
   g([1,2,3]);
   return 0;
}
------------

gives the error message:

|  Error: static assert  (__traits(isStaticArray,int[])) is false
|         instantiated from here: g!(int[])

Without the assertion, N is defined to 0.

Further investigation shows:

-------
	g!(int[3])([1,2,3]);  // passes a static array
-------
	int[3] x3 = [1,2,3];
	g(x3);                // passes a static array
-------
	auto z3 = [1,2,3];    // defines z3 as dynamic array
	g(y3);                // passes a dynamic array
-------

So it seems, the problem is that array literals on their own turned into
dynamic arrays unless you explicitly state a static array type in some way.

Wouldn't it make more sense the other way around? After all, turning a
static array into a dynamic array is easy, the other way around is
prohibited by the compiler. If array literals simply had a static array
type, they could be implicitly casted to dynamic arrays when necessary
but stay static if possible.

Greetings,
Norbert
Oct 09 2011
next sibling parent kennytm <kennytm gmail.com> writes:
Norbert Nemec <Norbert Nemec-online.de> wrote:
 Hi there,
 
 after a very busy and eventful year in my personal life, I have now
 finally found some time to play with D2. I am very impressed by the
 progress!
 
 One thing I noticed was that static arrays are somehow strangely limited:
 
 It is possible to overload based on the length:
 
 --------
 void f(int[3] x) {
    writeln("int[3]: ",x);
 }
 
 void f(int[4] x) {
    writeln("int[4]: ",x);
 }
 
 int main(string argv[]) {
    f([1,2,3]);
    f([1,2,3,4]);
    return 0;
 }
 --------
 
 However, used as function template argument, a static array is casted to
 a dynamic array:
 
 -----------
 void g(T)(T x) {
    static assert (__traits(isStaticArray,T));
    enum N = T.init.length;
    writeln(N,": ",x);
 }
 
 int main(string argv[]) {
    g([1,2,3]);
    return 0;
 }
 ------------
 
 gives the error message:
 
 |  Error: static assert  (__traits(isStaticArray,int[])) is false
 |         instantiated from here: g!(int[])
 
 Without the assertion, N is defined to 0.
 
 Further investigation shows:
 
 -------
 	g!(int[3])([1,2,3]);  // passes a static array
 -------
 	int[3] x3 = [1,2,3];
 	g(x3);                // passes a static array
 -------
 	auto z3 = [1,2,3];    // defines z3 as dynamic array
 	g(y3);                // passes a dynamic array
 -------
 
 So it seems, the problem is that array literals on their own turned into
 dynamic arrays unless you explicitly state a static array type in some way.
 
 Wouldn't it make more sense the other way around? After all, turning a
 static array into a dynamic array is easy, the other way around is
 prohibited by the compiler. If array literals simply had a static array
 type, they could be implicitly casted to dynamic arrays when necessary
 but stay static if possible.
 
 Greetings,
 Norbert

I think you could use something like void f(T, size_t n)(T[n] param) { ... } (not tested)
Oct 09 2011
prev sibling next sibling parent reply Gor Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
There is a huge difference between a static and a dynamic array.
Dynamic array is an indirected type (objects of that type are pointers
to the actual data), while static arrays are PODs (directly refers to
the data).
Dynamic arrays are always size_t.sizeof * 2 bytes long, while static
arrays are typeof(T[0]).sizeof * T.length bytes long. That makes
static arrays expensive to move around. Making the array literals
static by default would mean unnecessary copying most of the time. In
those few cases, where it is necessary, you can explicitly make them
static arrays.

Cheers,
Gor.

On Mon, Oct 10, 2011 at 10:42 AM, Norbert Nemec <Norbert nemec-online.de> w=
rote:
 Hi there,

 after a very busy and eventful year in my personal life, I have now
 finally found some time to play with D2. I am very impressed by the
 progress!

 One thing I noticed was that static arrays are somehow strangely limited:

 It is possible to overload based on the length:

 --------
 void f(int[3] x) {
 =A0 writeln("int[3]: ",x);
 }

 void f(int[4] x) {
 =A0 writeln("int[4]: ",x);
 }

 int main(string argv[]) {
 =A0 f([1,2,3]);
 =A0 f([1,2,3,4]);
 =A0 return 0;
 }
 --------

 However, used as function template argument, a static array is casted to
 a dynamic array:

 -----------
 void g(T)(T x) {
 =A0 static assert (__traits(isStaticArray,T));
 =A0 enum N =3D T.init.length;
 =A0 writeln(N,": ",x);
 }

 int main(string argv[]) {
 =A0 g([1,2,3]);
 =A0 return 0;
 }
 ------------

 gives the error message:

 | =A0Error: static assert =A0(__traits(isStaticArray,int[])) is false
 | =A0 =A0 =A0 =A0 instantiated from here: g!(int[])

 Without the assertion, N is defined to 0.

 Further investigation shows:

 -------
 =A0 =A0 =A0 =A0g!(int[3])([1,2,3]); =A0// passes a static array
 -------
 =A0 =A0 =A0 =A0int[3] x3 =3D [1,2,3];
 =A0 =A0 =A0 =A0g(x3); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// passes a static a=

 -------
 =A0 =A0 =A0 =A0auto z3 =3D [1,2,3]; =A0 =A0// defines z3 as dynamic array
 =A0 =A0 =A0 =A0g(y3); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0// passes a dynamic =

 -------

 So it seems, the problem is that array literals on their own turned into
 dynamic arrays unless you explicitly state a static array type in some wa=

 Wouldn't it make more sense the other way around? After all, turning a
 static array into a dynamic array is easy, the other way around is
 prohibited by the compiler. If array literals simply had a static array
 type, they could be implicitly casted to dynamic arrays when necessary
 but stay static if possible.

 Greetings,
 Norbert

Oct 09 2011
next sibling parent Don <nospam nospam.com> writes:
On 10.10.2011 08:58, Gor Gyolchanyan wrote:
 There is a huge difference between a static and a dynamic array.
 Dynamic array is an indirected type (objects of that type are pointers
 to the actual data), while static arrays are PODs (directly refers to
 the data).
 Dynamic arrays are always size_t.sizeof * 2 bytes long, while static
 arrays are typeof(T[0]).sizeof * T.length bytes long. That makes
 static arrays expensive to move around. Making the array literals
 static by default would mean unnecessary copying most of the time. In
 those few cases, where it is necessary, you can explicitly make them
 static arrays.

If they'd been made immutable, the way they should be, this wouldn't be an issue. The efficiency argument is bogus, I think. Because they're mutable, they always have to be created on the heap. So by default, they are very slow.
 Cheers,
 Gor.

 On Mon, Oct 10, 2011 at 10:42 AM, Norbert Nemec<Norbert nemec-online.de> 
wrote:
 Hi there,

 after a very busy and eventful year in my personal life, I have now
 finally found some time to play with D2. I am very impressed by the
 progress!

 One thing I noticed was that static arrays are somehow strangely limited:

 It is possible to overload based on the length:

 --------
 void f(int[3] x) {
    writeln("int[3]: ",x);
 }

 void f(int[4] x) {
    writeln("int[4]: ",x);
 }

 int main(string argv[]) {
    f([1,2,3]);
    f([1,2,3,4]);
    return 0;
 }
 --------

 However, used as function template argument, a static array is casted to
 a dynamic array:

 -----------
 void g(T)(T x) {
    static assert (__traits(isStaticArray,T));
    enum N = T.init.length;
    writeln(N,": ",x);
 }

 int main(string argv[]) {
    g([1,2,3]);
    return 0;
 }
 ------------

 gives the error message:

 |  Error: static assert  (__traits(isStaticArray,int[])) is false
 |         instantiated from here: g!(int[])

 Without the assertion, N is defined to 0.

 Further investigation shows:

 -------
         g!(int[3])([1,2,3]);  // passes a static array
 -------
         int[3] x3 = [1,2,3];
         g(x3);                // passes a static array
 -------
         auto z3 = [1,2,3];    // defines z3 as dynamic array
         g(y3);                // passes a dynamic array
 -------

 So it seems, the problem is that array literals on their own turned into
 dynamic arrays unless you explicitly state a static array type in some way.

 Wouldn't it make more sense the other way around? After all, turning a
 static array into a dynamic array is easy, the other way around is
 prohibited by the compiler. If array literals simply had a static array
 type, they could be implicitly casted to dynamic arrays when necessary
 but stay static if possible.

 Greetings,
 Norbert


Oct 10 2011
prev sibling parent reply Norbert Nemec <Norbert Nemec-online.de> writes:
On 10.10.2011 08:58, Gor Gyolchanyan wrote:
 There is a huge difference between a static and a dynamic array.
 Dynamic array is an indirected type (objects of that type are pointers
 to the actual data), while static arrays are PODs (directly refers to
 the data).
 Dynamic arrays are always size_t.sizeof * 2 bytes long, while static
 arrays are typeof(T[0]).sizeof * T.length bytes long. That makes
 static arrays expensive to move around. Making the array literals
 static by default would mean unnecessary copying most of the time. In
 those few cases, where it is necessary, you can explicitly make them
 static arrays.

Actually, my problem is not so much the choice of the "default" but that there is no easy way to work around it: I couldn't find any way to force a variable or function argument to be of static array type without explicitly stating the length. In an assignment like int[3] myvar = [1,2,3,4] the compiler will complain. However there does not seem to be any way to state int[...whatever...] myvar = [1,2,3,4]; enum N = typeof(myvar).init.length; and let the compiler deduce that N==4. Even worse for template arguments. My first idea (inspired by C++, also suggested by kennytm) template(T,int N) myfunc(T[N] arg) does not work in D and I could not find any alternative way to allow writing myfunc([1,2,3,4]) and determine the length of the static array at compile time.
Oct 10 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Norbert Nemec:

 However there does not seem to be any way to state
 	int[...whatever...] myvar = [1,2,3,4];
 	enum N = typeof(myvar).init.length;
 and let the compiler deduce that N==4.

Since years some people (among them there is me too) are asking for this syntax: int arr[$] = [1,2,3,4]; Vote for it here (an enhancement request from 2006!): http://d.puremagic.com/issues/show_bug.cgi?id=481 Bye, bearophile
Oct 11 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/11/11 2:56 AM, bearophile wrote:
 Norbert Nemec:

 However there does not seem to be any way to state
 	int[...whatever...] myvar = [1,2,3,4];
 	enum N = typeof(myvar).init.length;
 and let the compiler deduce that N==4.

Since years some people (among them there is me too) are asking for this syntax: int arr[$] = [1,2,3,4]; Vote for it here (an enhancement request from 2006!): http://d.puremagic.com/issues/show_bug.cgi?id=481

I was more enthused about that, but later I realized there are very few situations in which one reasonably has a fixed-size array that is large enough to make counting inadequate and is also updated often enough to make the feature worthwhile. It's one of those features that are nice to have but are not really missed in real code. Andrei
Oct 11 2011
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 I was more enthused about that, but later I realized there are very few 
 situations in which one reasonably has a fixed-size array that is large 
 enough to make counting inadequate and is also updated often enough to 
 make the feature worthwhile. It's one of those features that are nice to 
 have but are not really missed in real code.

I am not going to appreciate a language/compiler that accepts and compiles this program silently. This trap has caused a bug in my code time ago: int[3] a = [1, 2]; void main() {} The usage of [$] in that case avoids that kind of bug... Bye, bearophile
Oct 11 2011
parent reply Don <nospam nospam.com> writes:
On 11.10.2011 18:54, bearophile wrote:
 Andrei Alexandrescu:

 I was more enthused about that, but later I realized there are very few
 situations in which one reasonably has a fixed-size array that is large
 enough to make counting inadequate and is also updated often enough to
 make the feature worthwhile. It's one of those features that are nice to
 have but are not really missed in real code.

I am not going to appreciate a language/compiler that accepts and compiles this program silently. This trap has caused a bug in my code time ago: int[3] a = [1, 2]; void main() {}

I've hit that bug many, many times. But...
 The usage of [$] in that case avoids that kind of bug...

How? The code above would still compile. Would be much better to just ban assigning an array literal to an static array of different length.
Oct 11 2011
next sibling parent reply simendsjo <simendsjo gmail.com> writes:
On 11.10.2011 22:31, Don wrote:
 On 11.10.2011 18:54, bearophile wrote:
 Andrei Alexandrescu:

 I was more enthused about that, but later I realized there are very few
 situations in which one reasonably has a fixed-size array that is large
 enough to make counting inadequate and is also updated often enough to
 make the feature worthwhile. It's one of those features that are nice to
 have but are not really missed in real code.

I am not going to appreciate a language/compiler that accepts and compiles this program silently. This trap has caused a bug in my code time ago: int[3] a = [1, 2]; void main() {}

I've hit that bug many, many times. But...
 The usage of [$] in that case avoids that kind of bug...

How? The code above would still compile. Would be much better to just ban assigning an array literal to an static array of different length.

// It's important that a has a length of 3, so I specify it int[3] a = [1, 2] Error: Not enough values for a. Expected 3, got 2 // I don't care what length a has as long as it has a static length int[$] a = [1, 2];
Oct 11 2011
parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 10/11/11 1:46 PM, simendsjo wrote:

 // It's important that a has a length of 3, so I specify it
 int[3] a = [1, 2]
 Error: Not enough values for a. Expected 3, got 2

 // I don't care what length a has as long as it has a static length
 int[$] a = [1, 2];

It would be nice if this were coupled with an earlier suggestion: // large buffer starting with 1, 2 and the remainder zeros int[1000] a = [1, 2, 0..];
Oct 11 2011
parent bearophile <bearophileHUGS lycos.com> writes:
David Gileadi:

 It would be nice if this were coupled with an earlier suggestion:
 
 // large buffer starting with 1, 2 and the remainder zeros
 int[1000] a = [1, 2, 0..];

We discussed this a bit, in past. That syntax gives problems. This is better:
 int[1000] a = [1, 2, ...];

This is nice, but as we said in past, the need for this code is not that common. I've never needed this so far (while as example I'd enjoy to use an array comprehension syntax about every 20 lines of D code, or tuple syntax about as often). Bye, bearophile
Oct 11 2011
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Don Wrote:

 On 11.10.2011 18:54, bearophile wrote:
 Andrei Alexandrescu:

 I was more enthused about that, but later I realized there are very few
 situations in which one reasonably has a fixed-size array that is large
 enough to make counting inadequate and is also updated often enough to
 make the feature worthwhile. It's one of those features that are nice to
 have but are not really missed in real code.

I am not going to appreciate a language/compiler that accepts and compiles this program silently. This trap has caused a bug in my code time ago: int[3] a = [1, 2]; void main() {}

I've hit that bug many, many times. But...
 The usage of [$] in that case avoids that kind of bug...

How? The code above would still compile.

What is the "correct" version of the code? The array [1,2] of length 2 or a sequence literal [1,2,x] (with x = some not yet specified) with length 3? The compiler can't know what the right code is. If you write [$] on the left you are saying to the compiler that the correct thing is the sequence literal you have written on the right of the equal sign. If you don't use [$] it means you want exactly N items (so if the sequence literal on the right contains less than N then there is a bug). And finally some people have even said that if you put a [, ...] at the end of the sequence literal on the right of the equal sign then the specified N number of items on the left is the correct information, and the not specified items in the sequence literal on the right need to be filled with .init.
 Would be much better to just 
 ban assigning an array literal to an static array of different length.

For me this is an acceptable solution, it's safe, and it's the simplest one. This was present in the the patch in Bugzilla (but because of my silly poking nose it was later removed). But it sometimes forces you to count items manually (and in some very uncommon cases forces you to add leading zeros manually, but this is not a problem). Bye, bearophile
Oct 11 2011
prev sibling parent Norbert Nemec <Norbert Nemec-online.de> writes:
On 11.10.2011 15:35, Andrei Alexandrescu wrote:
 On 10/11/11 2:56 AM, bearophile wrote:
 Since years some people (among them there is me too) are asking for
 this syntax:

 int arr[$] = [1,2,3,4];

 Vote for it here (an enhancement request from 2006!):
 http://d.puremagic.com/issues/show_bug.cgi?id=481

I was more enthused about that, but later I realized there are very few situations in which one reasonably has a fixed-size array that is large enough to make counting inadequate and is also updated often enough to make the feature worthwhile. It's one of those features that are nice to have but are not really missed in real code.

How about function templates? You can start overloading like void f(int[1] x); void f(int[2] x); void f(int[3] x); but there is no way to do it via a single template. The only way to do it would be explicit instantiation void f(int N)(int[N] x); to be called as f!(3)([1,2,3]); Of course, this can always be done, but to me it seems ugly enough to think about a solution.
Oct 11 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 10 Oct 2011 02:42:26 -0400, Norbert Nemec  
<Norbert nemec-online.de> wrote:

 Hi there,

 after a very busy and eventful year in my personal life, I have now
 finally found some time to play with D2. I am very impressed by the
 progress!

 One thing I noticed was that static arrays are somehow strangely limited:

 It is possible to overload based on the length:

 --------
 void f(int[3] x) {
    writeln("int[3]: ",x);
 }

 void f(int[4] x) {
    writeln("int[4]: ",x);
 }

 int main(string argv[]) {
    f([1,2,3]);
    f([1,2,3,4]);
    return 0;
 }
 --------

 However, used as function template argument, a static array is casted to
 a dynamic array:

 -----------
 void g(T)(T x) {
    static assert (__traits(isStaticArray,T));
    enum N = T.init.length;
    writeln(N,": ",x);
 }

 int main(string argv[]) {
    g([1,2,3]);
    return 0;
 }
 ------------

 gives the error message:

 |  Error: static assert  (__traits(isStaticArray,int[])) is false
 |         instantiated from here: g!(int[])

 Without the assertion, N is defined to 0.

 Further investigation shows:

 -------
 	g!(int[3])([1,2,3]);  // passes a static array
 -------
 	int[3] x3 = [1,2,3];
 	g(x3);                // passes a static array
 -------
 	auto z3 = [1,2,3];    // defines z3 as dynamic array
 	g(y3);                // passes a dynamic array
 -------

 So it seems, the problem is that array literals on their own turned into
 dynamic arrays unless you explicitly state a static array type in some  
 way.

In fact, all array literals are dynamic. Examine the code for g!(int[3])([1,2,3]) (comments added): push 3 push 2 push 1 push 3 mov EAX,offset FLAT:_D12TypeInfo_G3i6__initZ SYM32 push EAX call _d_arrayliteralT PC32 ; create dynamic array literal add ESP,014h mov EBX,EAX push dword ptr 8[EBX] ; copy the data from the heap-allocated literal into the fixed-sized array temporary push dword ptr 4[EBX] push dword ptr [EBX] call _D10testdynarr10__T1gTG3iZ1gFG3iZv PC32 ; call g So really, the situation is worse than you thought :)
 Wouldn't it make more sense the other way around? After all, turning a
 static array into a dynamic array is easy, the other way around is
 prohibited by the compiler. If array literals simply had a static array
 type, they could be implicitly casted to dynamic arrays when necessary
 but stay static if possible.

D1 I think works that way (array and string literals are fixed-sized array arrays). Most of the time, you don't want a static array to be passed. Remember that fixed-size arrays are passed by *value* not by reference. This means any large array will be fully pushed onto the stack to pass it, not just a slice. It also adds (most of the time unnecessarily) bloat. Think of string literals: foo(T)(T x) { writeln(x); } foo("hello"); foo("goodbye"); In D1, this creates *two* different instantiations of foo, one for char[5u] and one for char[7u]. In D2 if fixed-sized array were the default choice, both "hello" and "goodbye" would be passed by value, meaning pushing all the data on the stack, in addition to creating the bloat. I do agree writing a fixed-size array literal should be easier than it is. -Steve
Oct 11 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, October 10, 2011 22:55 Norbert Nemec wrote:
 Even worse for template arguments. My first idea (inspired by C++, also
 suggested by kennytm)
 
 template(T,int N) myfunc(T[N] arg)
 
 does not work in D and I could not find any alternative way to allow
 writing
 
 myfunc([1,2,3,4])

It does if you pass it an actual static array. For instance, std.bitmanip.bigEndianToNative has a similar declaration. The problem with your example stems from the fact that the literal is a dynamic array, and the compiler fails to take the implicit conversion into account when instantiating the template. So, for instance myfunc(cast(int[4])[1, 2, 3, 4]); works. Obviously, that's not really what you're looking for, but the templated function is just fine. It's just that template instantiation is very exact and doesn't do much (if anything) in the way of conversions. - Jonathan M Davis
Oct 11 2011