www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Mutable enums

reply bearophile <bearophileHUGS lycos.com> writes:
Do you remember if this bug is in Bugzilla?


import std.algorithm;
void main() {
    enum a = [3, 1, 2];
    enum s = sort(a);
    assert(equal(a, [3, 1, 2]));
    assert(equal(s, [1, 2, 3]));
}


Bye,
bearophile
Nov 13 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 13, 2011 17:54:14 bearophile wrote:
 Do you remember if this bug is in Bugzilla?
 
 
 import std.algorithm;
 void main() {
     enum a = [3, 1, 2];
     enum s = sort(a);
     assert(equal(a, [3, 1, 2]));
     assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]); a isn't altered at all, because a doesn't really exist. Notice that assert(equal(a, [3, 1, 2])); still passes. It's equivalent to assert(equal([3, 1, 2], [3, 1, 2])); - Jonathan M Davis
Nov 13 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 import std.algorithm;
 void main() {
     enum a = [3, 1, 2];
     enum s = sort(a);
     assert(equal(a, [3, 1, 2]));
     assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums... On the other hand this gives the error message I was looking for, until today I didn't even think about const enums: import std.algorithm; const enum a = [1, 2]; void main() { sort(a); } So I guess I'll start using "cont enum" and "immutable enum" instead of enums :-) Bye, bearophile
Nov 13 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
      enum a = [3, 1, 2];
      enum s = sort(a);
      assert(equal(a, [3, 1, 2]));
      assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal. enum Enum{ opt1, opt2, } void main(){ auto moo = Enum.opt1; moo = Enum.opt2; // who would seriously want an error here??? } enum a = [1,2,3]; void main(){ auto x = a; x = [2,1,3]; // ditto }
 On the other hand this gives the error message I was looking for, until today
I didn't even think about const enums:

 import std.algorithm;
 const enum a = [1, 2];
 void main() {
      sort(a);
 }


 So I guess I'll start using "cont enum" and "immutable enum" instead of enums
:-)

You can do that, but they are not a full replacement. How would you get a sorted version of such an enum, for instance? =)
Nov 14 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/14/2011 09:27 AM, Timon Gehr wrote:
 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal. enum Enum{ opt1, opt2, } void main(){ auto moo = Enum.opt1; moo = Enum.opt2; // who would seriously want an error here??? } enum a = [1,2,3]; void main(){ auto x = a; x = [2,1,3]; // ditto }
 On the other hand this gives the error message I was looking for,
 until today I didn't even think about const enums:

 import std.algorithm;
 const enum a = [1, 2];
 void main() {
 sort(a);
 }


 So I guess I'll start using "cont enum" and "immutable enum" instead
 of enums :-)

You can do that, but they are not a full replacement. How would you get a sorted version of such an enum, for instance? =)

(.dup, obviously.)
Nov 14 2011
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/14/2011 10:20 AM, so wrote:
 On Mon, 14 Nov 2011 10:27:21 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 It is the right design. Why should enum imply const or immutable? (or
 inout, for that matter). They are completely orthogonal.

 enum Enum{
 opt1,
 opt2,
 }

 void main(){
 auto moo = Enum.opt1;
 moo = Enum.opt2; // who would seriously want an error here???
 }

You are missing the point, nobody asked that.

I think you are missing the point. What else are you asking for?
 You are assigning it to
 auto, a runtime variable.
 Which was asked was about modifying a constant, sort(a) means sort a
 in-place. So you cant do:

 immutable a;
 sort(a);

 But with current design you can do:

 enum a;
 sort(a);

 Which is to me, quite wrong.

It is just as right or wrong as doing enum a; sort([1,2,3]); The design of enums is probably even irrelevant for this discussion.
Nov 14 2011
prev sibling next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
so wrote:

 
 immutable a;
 sort(a);
 
 But with current design you can do:
 
 enum a;
 sort(a);
 
 Which is to me, quite wrong.

It is not wrong at all.
Nov 14 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used on an array. Consider that: enum a = "hello"; foo(a); Does not allocate heap memory, even though "hello" is a reference type. However: enum a = ['h', 'e', 'l', 'l', 'o']; foo(a); Allocates heap memory every time a is *used*. This is counter-intuitive, one uses enum to define things using the compiler, not during runtime. It's used to invoke CTFE, to avoid heap allocation. It's not a glorified #define macro. The deep issue here is not that enum is used as a manifest constant, but rather the fact that enum can map to a *function call* rather than the *result* of that function call. Would you say this should be acceptable? enum a = malloc(5); foo(a); // calls malloc(5) and passes the result to foo. If the [...] form is an acceptable enum, I contend that malloc should be acceptable as well.

a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l', 'o']. enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on every use But malloc is not CTFE-able, that is why it fails.
 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

Too restrictive imho.
 and should *never* map to an
 expression that calls a function during runtime.

Well, I would not miss that at all. But being stored as enum should not imply restrictions on type qualifiers.
Nov 14 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/14/2011 08:37 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used on an array. Consider that: enum a = "hello"; foo(a); Does not allocate heap memory, even though "hello" is a reference type. However: enum a = ['h', 'e', 'l', 'l', 'o']; foo(a); Allocates heap memory every time a is *used*. This is counter-intuitive, one uses enum to define things using the compiler, not during runtime. It's used to invoke CTFE, to avoid heap allocation. It's not a glorified #define macro. The deep issue here is not that enum is used as a manifest constant, but rather the fact that enum can map to a *function call* rather than the *result* of that function call. Would you say this should be acceptable? enum a = malloc(5); foo(a); // calls malloc(5) and passes the result to foo. If the [...] form is an acceptable enum, I contend that malloc should be acceptable as well.

a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l', 'o']. enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on every use But malloc is not CTFE-able, that is why it fails.

You are comparing apples to oranges here. Whether it's CTFE able or not has nothing to do with it, since the code is executed at runtime, not compile time.

The code is executed at compile time. It is just that the value is later created by allocating at runtime. enum foo = {writeln("foo"); return [1,2,3];}(); // fails, because writeln is not ctfe-able.
 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

Too restrictive imho.

It allows the compiler to evaluate the enum at compile time, and store any referenced data in ROM, avoiding frequent heap allocations (similar to string literals). IMO, type freedom is lower on the priority list than performance. You can already define a symbol that calls arbitrary code at runtime: property int[] a() { return [3, 1, 2];} Why should we muddy enum's goals with also being able to call functions during runtime?

As I said, I would not miss the capability of enums to create mutable arrays a lot. Usually you don't want that behavior, and explicitly .dup-ing is just fine. But I think it is a bit exaggerated to say enums can call functions at runtime. It is up to the compiler how to implement the array allocation.
 and should *never* map to an
 expression that calls a function during runtime.

Well, I would not miss that at all. But being stored as enum should not imply restrictions on type qualifiers.

The restrictions are required in order to avoid calling runtime functions for enum usage. Without the restrictions, you must necessarily call runtime functions for any reference-based types (to avoid modifying the original).

Yes, I don't need that. But I don't really want compile time capabilities hampered. enum a = [2,1,4]; enum b = sort(a); // should be fine. auto c = a; // sort(c); // don't care a lot if this works
 Note that I'm not saying literals in general should not trigger heap
 allocations, I'm saying assigning such literals to enums should require
 unrestricted copying without runtime function calls.

Yes, I get that. And I think it makes sense. But I am not (yet?) convinced that the solution to make all enums non-assignable, head-mutable and tail-immutable is satisfying.
 I don't think you would miss this as much as you think. Assigning a
 non-immutable array from an immutable one is as easy as adding a .dup,
 and then the code is more clear that an allocation is taking place.

It would be somewhat odd. enum a = [2,1,4]; enum b = sort(a.dup); // what exactly is that 'a.dup' thing? enum c = a.dup; // does this implicitly convert to immutable, or what happens here? enum d = sort(c); // does not work? enum e = foo(a.dup, b.dup, c.dup, d.dup);
Nov 14 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 14:59:50 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 08:37 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used on an array. Consider that: enum a = "hello"; foo(a); Does not allocate heap memory, even though "hello" is a reference type. However: enum a = ['h', 'e', 'l', 'l', 'o']; foo(a); Allocates heap memory every time a is *used*. This is counter-intuitive, one uses enum to define things using the compiler, not during runtime. It's used to invoke CTFE, to avoid heap allocation. It's not a glorified #define macro. The deep issue here is not that enum is used as a manifest constant, but rather the fact that enum can map to a *function call* rather than the *result* of that function call. Would you say this should be acceptable? enum a = malloc(5); foo(a); // calls malloc(5) and passes the result to foo. If the [...] form is an acceptable enum, I contend that malloc should be acceptable as well.

a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l', 'o']. enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on every use But malloc is not CTFE-able, that is why it fails.

You are comparing apples to oranges here. Whether it's CTFE able or not has nothing to do with it, since the code is executed at runtime, not compile time.

The code is executed at compile time. It is just that the value is later created by allocating at runtime. enum foo = {writeln("foo"); return [1,2,3];}(); // fails, because writeln is not ctfe-able.

Look at the code generated for enum a = [1, 2, 3]. using a is replaced with a call to _d_arrayliteral. There is no CTFE going on.

There is some ctfe going on, but the compiler has to allocate the result anew every time it is used. So there is also some runtime overhead. To make my point clearer: int foo(){return 100;} enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100, 100]; void main(){ auto x = a; // this does *not* call foo. But it allocates a new array literal }
 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

Too restrictive imho.

It allows the compiler to evaluate the enum at compile time, and store any referenced data in ROM, avoiding frequent heap allocations (similar to string literals). IMO, type freedom is lower on the priority list than performance. You can already define a symbol that calls arbitrary code at runtime: property int[] a() { return [3, 1, 2];} Why should we muddy enum's goals with also being able to call functions during runtime?

As I said, I would not miss the capability of enums to create mutable arrays a lot. Usually you don't want that behavior, and explicitly .dup-ing is just fine. But I think it is a bit exaggerated to say enums can call functions at runtime. It is up to the compiler how to implement the array allocation.

The compiler has no choice. It must develop the array at runtime, or else the type allows one to modify the source value (just like in D1 how you could modify string literals). In essence, the compiler is creating a new copy for every usage (and building it from scratch).

That is a quality of implementation issue. The language semantics do not require that.
 and should *never* map to an
 expression that calls a function during runtime.

Well, I would not miss that at all. But being stored as enum should not imply restrictions on type qualifiers.

The restrictions are required in order to avoid calling runtime functions for enum usage. Without the restrictions, you must necessarily call runtime functions for any reference-based types (to avoid modifying the original).

Yes, I don't need that. But I don't really want compile time capabilities hampered. enum a = [2,1,4]; enum b = sort(a); // should be fine.

I was actually surprised that this compiles. But this should not be a problem even if a was immutable(int)[]. sort should be able to create a copy of an immutable array in order to sort it. It doesn't matter the performance hit, because this should all be done at compile time.

It does not, but explicitly calling .dup works immutable x = [3,2,1]; immutable y = sort(x.dup);
 Note that I'm not saying literals in general should not trigger heap
 allocations, I'm saying assigning such literals to enums should require
 unrestricted copying without runtime function calls.

Yes, I get that. And I think it makes sense. But I am not (yet?) convinced that the solution to make all enums non-assignable, head-mutable and tail-immutable is satisfying.

When I see an enum, I think "evaluated at compile time". No matter how complex it is to build that value, it should be built at compile-time and *used* at runtime. No complex function calls should be done at runtime, an enum is a value.

Exactly. Therefore you assign from it by copying it. Compare to static array. int[10] x = [1,2,3,4,5,6,7,8,9,0]; x still needs to be initialized at runtime.
 I did an interesting little test:

 import std.algorithm;
 import std.stdio;

 int[] foo(int[] x)
 {
 return x ~ x;
 }
 enum a = [3, 1, 2];
 enum b = sort(foo(foo(foo(a))));

 void main()
 {
 writeln(b);
 }

 Want to see the assembly generated for the writeln call?

 push 018h
 mov EAX,offset FLAT:_D11TypeInfo_Ai6__initZ SYM32
 push EAX
 call _d_arrayliteralTX PC32
 add ESP,8
 mov ECX,1
 mov [EAX],ECX
 mov 4[EAX],ECX
 mov 8[EAX],ECX
 mov 0Ch[EAX],ECX
 mov 010h[EAX],ECX
 mov 014h[EAX],ECX
 mov 018h[EAX],ECX
 mov 01Ch[EAX],ECX
 mov EDX,2
 mov 020h[EAX],EDX
 mov 024h[EAX],EDX
 mov 028h[EAX],EDX
 mov 02Ch[EAX],EDX
 mov 030h[EAX],EDX
 mov 034h[EAX],EDX
 mov 038h[EAX],EDX
 mov 03Ch[EAX],EDX
 mov EBX,3
 mov 040h[EAX],EBX
 mov 044h[EAX],EBX
 mov 048h[EAX],EBX
 mov 04Ch[EAX],EBX
 mov 050h[EAX],EBX
 mov 054h[EAX],EBX
 mov 058h[EAX],EBX
 mov 05Ch[EAX],EBX
 mov ECX,EAX
 mov EAX,018h
 mov -8[EBP],EAX
 mov -4[EBP],ECX
 mov EDX,-4[EBP]
 mov EAX,-8[EBP]
 push EDX
 push EAX
 call
 _D3std5stdio76__T7writelnTS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZ7writelnFS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZv PC32



 Really? That's a better solution than using ROM space to store the
 result of the expression as evaluated at compile time? The worst part is
 that this will be used *EVERY TIME* I use the enum b (even if I pass it
 as a const array).

That just tells us that DMD sucks at generating code for array literals. This generates identical code: import std.stdio; void main() { writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3]); } You don't need enums for that. What it actually should for both our examples is more like the following: import std.stdio; immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3]; void main() { writeln(_somewhereinrom.dup); } push %ebp mov %esp,%ebp pushl 0x8097184 pushl 0x8097180 mov $0x80975c8,%eax push %eax call 8079470 <_adDupT> add $0xc,%esp push %edx push %eax call 807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv> xor %eax,%eax pop %ebp ret If writeln would actually be const correct, the compiler could even get rid of the allocation. This is not about enums that much, it is about array literals. The fact that stack static array initialization allocates is one of DMDs bigger warts. Look at the ridiculous code generated for the following example: void main() { int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3]; writeln(x); }
 I don't think you would miss this as much as you think. Assigning a
 non-immutable array from an immutable one is as easy as adding a .dup,
 and then the code is more clear that an allocation is taking place.

It would be somewhat odd. enum a = [2,1,4]; enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

I don't think .dup should be necessary at compile time. Creating a sorted copy of an immutable array should be quite doable.

I agree, phobos won't currently do it though.
 enum c = a.dup; // does this implicitly convert to immutable, or what
 happens here?

Either a compile error (cannot store mutable reference data as an enum), or an implicit conversion back to immutable.
 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

Again, I don't think .dup would be used for dependent enums, I was rather thinking dup would be used where you need a mutable copy of an array during enum usage in normal code.

But if the type of a,b,c,d is immutable(int)[] and foo is a function that takes 4 int[]s then the .dup's are necessary to pass type checking.
Nov 14 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 16:28:52 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
 Look at the code generated for enum a = [1, 2, 3]. using a is replaced
 with a call to _d_arrayliteral. There is no CTFE going on.

There is some ctfe going on, but the compiler has to allocate the result anew every time it is used. So there is also some runtime overhead. To make my point clearer: int foo(){return 100;} enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100, 100]; void main(){ auto x = a; // this does *not* call foo. But it allocates a new array literal }

Yes, you are right. The issue is that the resulting array is initialized at runtime, not that CTFE is being avoided. After doing some of these tests, I have a better understanding of the issues.
 The compiler has no choice. It must develop the array at runtime, or
 else the type allows one to modify the source value (just like in D1 how
 you could modify string literals). In essence, the compiler is creating
 a new copy for every usage (and building it from scratch).

That is a quality of implementation issue. The language semantics do not require that.

The language semantics require that if an enum type points at mutable data, a runtime allocation *must* occur to avoid corruption of literals. I think a rule requiring an enum to be immutable or implicitly cast to immutable puts the burden of runtime allocation on the coder, making it clear what's going on. In C++, novice coders typically pass classes by value not knowing what a horrible thing this is doing. Then they are puzzled why the code is so slow, the syntax is so short! This is another case of a hidden allocation which can be either avoided or made visible.
 enum a = [2,1,4];
 enum b = sort(a); // should be fine.

I was actually surprised that this compiles. But this should not be a problem even if a was immutable(int)[]. sort should be able to create a copy of an immutable array in order to sort it. It doesn't matter the performance hit, because this should all be done at compile time.

It does not, but explicitly calling .dup works immutable x = [3,2,1]; immutable y = sort(x.dup);

I'm saying sort (or another symbol, ctfesort?) can be made to do the dup automatically for you so you don't have to have it when using ctfe. Extra allocations during CTFE cost nothing (well, with a properly GC'd compiler, that is). Update: I have a better idea, see below.
 When I see an enum, I think "evaluated at compile time". No matter how
 complex it is to build that value, it should be built at compile-time
 and *used* at runtime. No complex function calls should be done at
 runtime, an enum is a value.

Exactly. Therefore you assign from it by copying it. Compare to static array. int[10] x = [1,2,3,4,5,6,7,8,9,0]; x still needs to be initialized at runtime.

Yes, but this is spelled out because copying a static array requires moving data. However, this does *not* require a hidden allocation (even though it does do a hidden allocation currently). I'm not worried about copying data as much as I am about hidden allocations. Hidden allocations are a huge drag on performance. Every time you allocate, you need to take a global GC lock, and it's an unbounded operation (doing one allocation could run a collection cycle).

You don't actually _need_ a global GC lock. It is just how it is implemented in this case. Note that this is an explicit allocation: int[] a = [1,2,3]; // just as explicit as a NewExpression Only the enums "hide" it sometimes, but actually you should know the involved types.
 I did an interesting little test:


[snip]
 That just tells us that DMD sucks at generating code for array literals.

 This generates identical code:

 import std.stdio;

 void main() {
 writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3,
 3, 3, 3, 3]);
 }

 You don't need enums for that.


 What it actually should for both our examples is more like the following:

 import std.stdio;

 immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,
 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];

 void main() {
 writeln(_somewhereinrom.dup);
 }

 push %ebp
 mov %esp,%ebp
 pushl 0x8097184
 pushl 0x8097180
 mov $0x80975c8,%eax
 push %eax
 call 8079470 <_adDupT>
 add $0xc,%esp
 push %edx
 push %eax
 call 807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
 xor %eax,%eax
 pop %ebp
 ret


 If writeln would actually be const correct, the compiler could even
 get rid of the allocation.

That is the idea. Get rid of the hidden allocation. Writeln *is* const correct, it can certainly print immutable(int)[].

Well, there is a function called writeln that can do that. That is a different function. But the one that gets actually called is not const correct as well. This is writeln: // Most general instance void writeln(T...)(T args) if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) : const(char)[])) { stdout.write(args, '\n'); } => writeln([1,2,3]); // modulo IFTY: writeln!(int[])([1,2,3]); // const correct? writeln!(int[])([1,2,3].idup); // nope! Error: cannot implicitly convert expression (_adDupT(& _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[] Now if there was a const there, the compiler could infer that writeln will not change the .duped array. Ergo it could pass the reference to immutable data directly. It would maybe help against template code bloat a bit, but not that much because const(immutable(int)[]) and the like are distinct types to const(int[]).
 The issue is not
 writeln, it's what the type of the array literal/enum is.

 Technically, an array literal is equivalent to an enum, and should
 follow the same rules.

Remember that immutable is transitive. That can really get in your way in this case.
 This is not about enums that much, it is about array literals.

 The fact that stack static array initialization allocates is one of
 DMDs bigger warts.

 Look at the ridiculous code generated for the following example:

 void main() {
 int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3,
 3, 3, 3, 3, 3];
 writeln(x);
 }

Yes, these are all cases of the same issue.
 enum a = [2,1,4];
 enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

I don't think .dup should be necessary at compile time. Creating a sorted copy of an immutable array should be quite doable.

I agree, phobos won't currently do it though.

This is easily fixed. But maybe there is a better way (see below).
 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

Again, I don't think .dup would be used for dependent enums, I was rather thinking dup would be used where you need a mutable copy of an array during enum usage in normal code.

But if the type of a,b,c,d is immutable(int)[] and foo is a function that takes 4 int[]s then the .dup's are necessary to pass type checking.

What about this idea: At a global level, expressions that result in CTFE being triggered, can be implicitly cast from mutable to immutable and vice versa via a deep-dup. This allows you to use enums as parameters to functions accepting mutable references. Then enums that are derived from other enums do not need to follow the same rules as runtime code that uses the enums. This of course, only happens at the global-expressions level, as function internals must compile at runtime as well. What I thought of as a solution was to create CTFE only functions that wrap other functions to do a dup. But you wouldn't want to do this during runtime, because dup is expensive. During compile time, dup costs nothing. This idea essentially takes the place of that boilerplate code. -Steve

That could work, but I think this is cluttering up the semantics a bit.
Nov 15 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:

 Yes, but this is spelled out because copying a static array requires
 moving data. However, this does *not* require a hidden allocation (even
 though it does do a hidden allocation currently).

 I'm not worried about copying data as much as I am about hidden
 allocations. Hidden allocations are a huge drag on performance. Every
 time you allocate, you need to take a global GC lock, and it's an
 unbounded operation (doing one allocation could run a collection cycle).

You don't actually _need_ a global GC lock. It is just how it is implemented in this case.

This is all fine in theory. I haven't seen any implementations. But memory allocations are not cheap, even without a GC lock.
 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.
 That is the idea. Get rid of the hidden allocation. Writeln *is* const
 correct, it can certainly print immutable(int)[].

Well, there is a function called writeln that can do that. That is a different function. But the one that gets actually called is not const correct as well. This is writeln: // Most general instance void writeln(T...)(T args) if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) : const(char)[])) { stdout.write(args, '\n'); } => writeln([1,2,3]); // modulo IFTY: writeln!(int[])([1,2,3]); // const correct? writeln!(int[])([1,2,3].idup); // nope! Error: cannot implicitly convert expression (_adDupT(& _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[], it will compile. Simple test: immutable(int)[] a = [1, 2, 3]; writeln(a); works with 2.056.

That is like saying void foo(int[] a){...} // prints a void bar(immutable(int)[] a){...} // prints a int[] a = [1,2,3]; immutable(int)[] b = [1, 2, 3]; foo(a); bar(b); "=> Oh, bar is const correct because I can call foo with mutable data and bar with immutable data." foo is writeln!(int[]) bar is writeln!(immutable(int)[]) in effect, if you do: writeln("hello".dup); The compiler cannot assume that the IFTI'd writeln!(char[]) will not change the argument, ergo it cannot optimize away the .dup, even though it would be a valid transformation as long as the programmer does not make the program semantics depend on the identity relation on immutable references (doing so defeats the purpose of immutability).
 The issue is not
 writeln, it's what the type of the array literal/enum is.

 Technically, an array literal is equivalent to an enum, and should
 follow the same rules.

Remember that immutable is transitive. That can really get in your way in this case.

If the data is stored in ROM, it should be typed as immutable. If it's not, then it could be modified. The only other option is heap allocation and construction on use, which I've already said is undesirable. If we start from "all array literals and enums that contain references store the referenced data in ROM," then we will find ways to deal with any inconveniences. It's all a matter of what's more important in a system language, performance or convenience.

Both are more important. It is D. Everything you need to do is make the enum static immutable instead. D is also a scripting language and an application programming language. auto a = [new Foo, new Bar, new Qux]; // I want that to work.
Nov 16 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/16/2011 08:26 PM, Timon Gehr wrote:
 auto a = [new Foo, new Bar, new Qux]; // I want that to work.

(It currently does, if that was unclear)
Nov 16 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable.

string is not an immutable type. It is immutable(char)[] and char is a built-in value type. static assert(isMutable!string);
 Make no mistake, this doesn't cover every current instance array
 literals, such as ones which contain necessarily-heap-allocated
 entities. Those would be covered by a library function (e.g.
 array(...)). But those are not array literals anyways, they are
 constructors.

It is true that they are constructors, but they are currently also called array literals.
 Well, there is a function called writeln that can do that. That is a
 different function. But the one that gets actually called is not const
 correct as well.


 This is writeln:

 // Most general instance
 void writeln(T...)(T args)
 if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
 const(char)[]))
 {
 stdout.write(args, '\n');
 }


 =>
 writeln([1,2,3]);
 // modulo IFTY:
 writeln!(int[])([1,2,3]);
 // const correct?
 writeln!(int[])([1,2,3].idup); // nope!

 Error: cannot implicitly convert expression (_adDupT(&
 _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[], it will compile. Simple test: immutable(int)[] a = [1, 2, 3]; writeln(a); works with 2.056.

That is like saying void foo(int[] a){...} // prints a void bar(immutable(int)[] a){...} // prints a int[] a = [1,2,3]; immutable(int)[] b = [1, 2, 3]; foo(a); bar(b); "=> Oh, bar is const correct because I can call foo with mutable data and bar with immutable data." foo is writeln!(int[]) bar is writeln!(immutable(int)[]) in effect, if you do: writeln("hello".dup); The compiler cannot assume that the IFTI'd writeln!(char[]) will not change the argument, ergo it cannot optimize away the .dup, even though it would be a valid transformation as long as the programmer does not make the program semantics depend on the identity relation on immutable references (doing so defeats the purpose of immutability).

writeln is not contractually const correct, but that's only because most of D is not const correct either. For example, Object.toString is not const, so if you const-ified all writeln args, you couldn't print objects. But writeln itself does not change any data (a rogue toString might change data, but that is not common practice). Once D becomes const correct, writeln *can* apply const to all it's parameters.

Ok, I see. writeln still could be const-correct in principle. It would need to apply const to the parameters that have a const toString method. But the language is not powerful enough to do that. There is no way to intercept IFTI... I am commonly missing that feature.
 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

Both are more important. It is D. Everything you need to do is make the enum static immutable instead. D is also a scripting language and an application programming language. auto a = [new Foo, new Bar, new Qux]; // I want that to work.

auto a = array(new Foo, new Bar, new Qux);

Requiring that works modulo template bloat and breakage of existing code. Also, it does not really buy us anything imho.
 The one case which is difficult to do is initializing a fixed-size array
 with a literal that uses runtime data. I suppose we'd need a function
 that returns a fixed-sized array made of its arguments, and doing the
 init builds it in place. i.e.:

 Object[3] objs = array_fixed(new Foo, new Bar, new Qux);

 would not do any moving of references, it would construct the fixed
 sized array in-place. Initializing fixed sized arrays with array
 literals already needs attention anyway.

From the compiler side. Not necessarily from the std lib side.
Nov 16 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/16/2011 10:56 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable.

string is not an immutable type. It is immutable(char)[] and char is a built-in value type. static assert(isMutable!string);

It fits my definition of a valid enum reference type (immutable or implicitly castable to immutable). The point is that the data referenced is stored in ROM and therefore a) immutable and b) fully defined at compile-time.

Indeed. But fact is, the data that is qualified with immutable is not of reference type therefore it behaves nicely. And you don't get that in the general case.
 Make no mistake, this doesn't cover every current instance array
 literals, such as ones which contain necessarily-heap-allocated
 entities. Those would be covered by a library function (e.g.
 array(...)). But those are not array literals anyways, they are
 constructors.

It is true that they are constructors, but they are currently also called array literals.

I'm looking to redefine what a literal means in D.
 writeln is not contractually const correct, but that's only because most
 of D is not const correct either. For example, Object.toString is not
 const, so if you const-ified all writeln args, you couldn't print
 objects. But writeln itself does not change any data (a rogue toString
 might change data, but that is not common practice).

 Once D becomes const correct, writeln *can* apply const to all it's
 parameters.

Ok, I see. writeln still could be const-correct in principle. It would need to apply const to the parameters that have a const toString method. But the language is not powerful enough to do that. There is no way to intercept IFTI... I am commonly missing that feature.


I am looking for something like this: template writeln(T...)(T){ alias writelnImpl!(writelnInferConst!T) writeln; } (it is even trivial to parse, unlike normal function definitions!)
 I have an enhancement request in for intercepting IFTI (not sure if it
 applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

It has a complexity of at least 2^numparams and probably all kinds of odd implications for the compiler internals that would lead to a buggy implementation. I think this is a better solution: void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);} Then it is just a matter of applying proper value range propagation for IFTY: void bar(T: short)(T t){...} void main(){ bar(1); // ok }
 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap
 allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

Both are more important. It is D. Everything you need to do is make the enum static immutable instead. D is also a scripting language and an application programming language. auto a = [new Foo, new Bar, new Qux]; // I want that to work.

auto a = array(new Foo, new Bar, new Qux);

Requiring that works modulo template bloat and breakage of existing code. Also, it does not really buy us anything imho.

I think the bloat is a wash. Every time I use an array literal, there is a complete instantiation of what would be in an array template inline in the calling function.

With the template function you have that too because the calling function builds the arguments. It is just that you also get the template bloat and an additional function call. Unless it is inlined, of course.
 Yes, it breaks code. It's worth it. To avoid a heap allocation for [1,
 2, 3] is worth breaking code that uses runtime data in an array literal.
 The compiler can be helpful in the error message:

 Error somefile.d(345): array literals cannot contain runtime data. Maybe
 you meant:
 array(new Foo, new Bar, new Qux);

 I want to point out that currently there is *NO* way to make an
 immutable array literal that lives in ROM. This is unacceptable.

There is: void main(){ static immutable x = [1,2,3]; } And the compiler *really* should write this to ROM: void main(){ immutable(int)[] x=[1,2,3]; // should be slice to ROM, because the array literal contents are typed as immutable and known during compile time. }
Nov 16 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/16/2011 11:39 PM, Timon Gehr wrote:
 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation for
 IFTY:

 void bar(T: short)(T t){...}

 void main(){
 bar(1); // ok
 }

BTW, this already works for your use case: void foo2(ParameterTypeTuple!foo t){foo(t);}
Nov 16 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/17/2011 03:19 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 17:39:16 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 10:56 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know
 the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable.

string is not an immutable type. It is immutable(char)[] and char is a built-in value type. static assert(isMutable!string);

It fits my definition of a valid enum reference type (immutable or implicitly castable to immutable). The point is that the data referenced is stored in ROM and therefore a) immutable and b) fully defined at compile-time.

Indeed. But fact is, the data that is qualified with immutable is not of reference type therefore it behaves nicely. And you don't get that in the general case.

In the general case, there is always a library function for construction. In other words, what the compiler currently does for array literals can be done in a library. But a library cannot create ROM space. The compiler-based features (and CTFE in general) should be helping us create things at compile time, not at run time. We already have the tools to construct arrays at runtime.
 I am looking for something like this:

 template writeln(T...)(T){
 alias writelnImpl!(writelnInferConst!T) writeln;
 }

 (it is even trivial to parse, unlike normal function definitions!)

What does writelnInferConst!T do? I'm afraid I'm not getting what you are saying. I was thinking writeln should do this: void writeln(T...)(const T args) {...}

As you pointed out, this cannot print types that have a non-const toString method (caching the result could be a perfectly valid reason for that.) writelnInferConst finds out which parameters can be treated as const and still be printed so that the correct version of writeln may be called. For example: class Foo{ // can be printed if const string toString()const{return "Foo";} } class Bar{ // cannot be printed if const string cache; string toString(){return cache!is null?cache:(cache="Bar");} } template hasConstToString(T){ enum hasConstToString = is(typeof((const T t){return t.toString();})); } template writelnInferConstImpl(T...){ static if(!T.length) alias T X; else static if(hasConstToString!(T[0])){ alias T[0] _; alias TypeTuple!(const(_),writelnInferConst!(T[1..$])) X; }else alias TypeTuple!(T[0],writelnInferConst!(T[1..$])) X; } template writelnInferConst(T...){alias writelnInferConstImpl!T.X writelnInferConst;} // (bug 6966) static assert(is(writelnInferConst!(Foo,Bar,Foo,Foo,Bar)==TypeTuple!(const(Foo),Bar,const(Foo),const(Foo),Bar))); The real thing would also do stuff like actual argument immutable(Foo[])[] => formal argument const(Foo[][]). In order to get rid of bloat created by pointless instantiations of writelnImpl.
 I have an enhancement request in for intercepting IFTI (not sure if it
 applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

It has a complexity of at least 2^numparams and probably all kinds of odd implications for the compiler internals that would lead to a buggy implementation.

I'm not a compiler writer, but I don't see how this is the case.

There were/are quite a few error gagging related bugs. I guess this would be similar.
 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation
 for IFTY:

 void bar(T: short)(T t){...}

 void main(){
 bar(1); // ok
 }

The issue with all this is, IFTI doesn't work that way: void foo(T: short)(T t) {} void main() { foo(1); } testifti.d(5): Error: template testifti.foo(T : short) does not match any function template declaration testifti.d(5): Error: template testifti.foo(T : short) cannot deduce template function from argument types !()(int) IFTI decides the types of literals before trying to find a proper template to instantiate. Any possibility to intercept the decision of literal type or of instantiation would be useful. I think that it's better suited to the constraints, because there is more power there. But I'm not sure. If you can find a more straightforward way, I'm all for it. In any case, I need to update the bug report, because the general case is if foo has an overload. For instance: foo(short s); foo(wstring w); foo2 should be able to call both with 1 and "hello" without issue. My driving use case to create the enhancement was creating a wrapper type that intercepted function calls. I planned to use opDispatch, but it didn't quite work with literals.

Ok, I see the problem. My proposed IFTI template mechanism would save the day. It would look like this (static foreach would have to be replaced by a recursive mixin template because Walter encountered implementation difficulties). template OverloadsOf(alias symbol){ // should this be in std.traits? alias TypeTuple!(__traits(getOverloads, __traits(parent,symbol), __traits(identifier,symbol))) OverloadsOf; } auto wrapper(alias foo)(ParameterTypeTuple!foo args){ return foo(args); } template opDispatch(string op,T...)(T){ static foreach(foo; OverloadsOf!(mixin(op))){ alias wrapper!foo opDispatch; } static if(OverloadsOf!(mixin(op)).length==0) { // we are dealing with a template function auto opDispatch(T args){ return foo(args); } } }
 I think the bloat is a wash. Every time I use an array literal, there is
 a complete instantiation of what would be in an array template inline in
 the calling function.

With the template function you have that too because the calling function builds the arguments. It is just that you also get the template bloat and an additional function call. Unless it is inlined, of course.

You are right, I hadn't thought of that. But what about this: most array literals are actually literals (made up of CTFE-decided values). Making them ROM-stored would *eliminate* bloat as compared to the current implementation.

Yes, and the compiler should do that. That works fine with the current semantics of array literals.
 Also, given how template-centric D and phobos are, I think at some point
 we need to examine how to minimize template bloat in general, by
 coalescing identical code into one function, or not emitting functions
 that are always inlined, not to mention avoiding storing templates only
 used at compile-time in the code (e.g. isInputRange).

I agree.
 Yes, it breaks code. It's worth it. To avoid a heap allocation for [1,
 2, 3] is worth breaking code that uses runtime data in an array literal.
 The compiler can be helpful in the error message:

 Error somefile.d(345): array literals cannot contain runtime data. Maybe
 you meant:
 array(new Foo, new Bar, new Qux);

 I want to point out that currently there is *NO* way to make an
 immutable array literal that lives in ROM. This is unacceptable.

There is: void main(){ static immutable x = [1,2,3]; }

Seems rather odd you should have to jump through these hoops to get the compiler to use ROM space. But I concede that I did not know of this trick. It does not sway my opinion that CTFE should produce ROM-stored references, and library function should be used for runtime-constructed references.

CTFE should produce ROM-stored data iff it is used during run time, I agree on that. However if the enum is typed as mutable, it should create a copy of the ROM-stored data.
Nov 17 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/17/2011 07:23 PM, Steven Schveighoffer wrote:
 On Thu, 17 Nov 2011 12:31:58 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/17/2011 03:19 PM, Steven Schveighoffer wrote:

 What does writelnInferConst!T do? I'm afraid I'm not getting what you
 are saying.

 I was thinking writeln should do this:

 void writeln(T...)(const T args) {...}

As you pointed out, this cannot print types that have a non-const toString method (caching the result could be a perfectly valid reason for that.)

Caching string representation IMO is not a significant use case. Not only that, but toString should be deprecated anyways in favor of a stream-based system.
 writelnInferConst finds out which parameters can be treated as const
 and still be printed so that the correct version of writeln may be
 called.

 For example:

 class Foo{ // can be printed if const
 string toString()const{return "Foo";}
 }

 class Bar{ // cannot be printed if const
 string cache;
 string toString(){return cache!is null?cache:(cache="Bar");}
 }

 template hasConstToString(T){
 enum hasConstToString = is(typeof((const T t){return t.toString();}));
 }

 template writelnInferConstImpl(T...){
 static if(!T.length) alias T X;
 else static if(hasConstToString!(T[0])){
 alias T[0] _;
 alias TypeTuple!(const(_),writelnInferConst!(T[1..$])) X;
 }else
 alias TypeTuple!(T[0],writelnInferConst!(T[1..$])) X;
 }
 template writelnInferConst(T...){alias writelnInferConstImpl!T.X
 writelnInferConst;} // (bug 6966)


 static
 assert(is(writelnInferConst!(Foo,Bar,Foo,Foo,Bar)==TypeTuple!(const(Foo),Bar,const(Foo),const(Foo),Bar)));

If your goal is to reduce template bloat, I think this is not the solution. But also note that this still does not guarantee const. I don't really see the point of doing all these templates if you aren't going to guarantee writeln doesn't modify the data.
 The issue with all this is, IFTI doesn't work that way:

 void foo(T: short)(T t) {}

 void main()
 {
 foo(1);
 }

 testifti.d(5): Error: template testifti.foo(T : short) does not match
 any function template declaration
 testifti.d(5): Error: template testifti.foo(T : short) cannot deduce
 template function from argument types !()(int)

 IFTI decides the types of literals before trying to find a proper
 template to instantiate. Any possibility to intercept the decision of
 literal type or of instantiation would be useful. I think that it's
 better suited to the constraints, because there is more power there. But
 I'm not sure. If you can find a more straightforward way, I'm all for
 it.

 In any case, I need to update the bug report, because the general case
 is if foo has an overload. For instance:

 foo(short s);
 foo(wstring w);

 foo2 should be able to call both with 1 and "hello" without issue.

 My driving use case to create the enhancement was creating a wrapper
 type that intercepted function calls. I planned to use opDispatch, but
 it didn't quite work with literals.

Ok, I see the problem. My proposed IFTI template mechanism would save the day. It would look like this (static foreach would have to be replaced by a recursive mixin template because Walter encountered implementation difficulties). template OverloadsOf(alias symbol){ // should this be in std.traits? alias TypeTuple!(__traits(getOverloads, __traits(parent,symbol), __traits(identifier,symbol))) OverloadsOf; } auto wrapper(alias foo)(ParameterTypeTuple!foo args){ return foo(args); } template opDispatch(string op,T...)(T){ static foreach(foo; OverloadsOf!(mixin(op))){ alias wrapper!foo opDispatch; } static if(OverloadsOf!(mixin(op)).length==0) { // we are dealing with a template function auto opDispatch(T args){ return foo(args); } } }

Pardon my saying so, but this looks horrendous. Not to mention that I don't think it would work.

Oh, it would certainly work.
 IFTI instantiates templates, it does not look
 inside instantiated templates for overloads.

This works, does this solve the confusion?: void foo(int){writeln("foo!");} void bar(double){writeln("bar!");} template merge(){ alias foo qux; alias bar qux; } alias merge!().qux qux; void main(){ qux(1); // calls foo qux(1.0); // calls bar }
 BTW, your proposed IFTI template mechanism, do you have it stated
 anywhere? Maybe it fixes the problem I mentioned.

Not yet, I will file a bugzilla enhancement request and post a link here. What it does is quite simple: 1. Apply normal IFTI instantiation rules to the IFTI template, as if it was a function template. 2. Instantiate the IFTI template with the deduced arguments. 3. The result of the instantiation must be callable with the original arguments. Call it. This allows the function that is called to have a different (albeit compatible) signature from what IFTI would give you.
 Seems rather odd you should have to jump through these hoops to get the
 compiler to use ROM space. But I concede that I did not know of this
 trick. It does not sway my opinion that CTFE should produce ROM-stored
 references, and library function should be used for runtime-constructed
 references.

CTFE should produce ROM-stored data iff it is used during run time, I agree on that. However if the enum is typed as mutable, it should create a copy of the ROM-stored data.

Well, this is closer than I thought we were to an agreement. I would agree with this, as long as it could be implicitly cast to immutable or const. i.e.: enum foo = [1, 2, 3]; immutable(int)[] a = foo; // no allocation const(int)[] b = foo; // no allocation -Steve

Yes, that is exactly how I would imagine it to work.
Nov 17 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 13, 2011 19:02:01 bearophile wrote:
 Jonathan M Davis:
 import std.algorithm;
 void main() {
 
     enum a = [3, 1, 2];
     enum s = sort(a);
     assert(equal(a, [3, 1, 2]));
     assert(equal(s, [1, 2, 3]));
 
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums... On the other hand this gives the error message I was looking for, until today I didn't even think about const enums: import std.algorithm; const enum a = [1, 2]; void main() { sort(a); } So I guess I'll start using "cont enum" and "immutable enum" instead of enums :-)

It depends entirely on what you're trying to do. If you understand how manifest constants work, then they can be quite advantageous. What you probably really want for arrays though is not an enum but just a const or immutable module variable. immutable a = [3, 1, 2]; Otherwise, you're allocating a new array every time you use the enum. So, use a manifest constant when you want to avoid having it take up any memory but don't care about whatever allocations may occur when it's used (primitive types such as ints being a prime example), whereas if you want an actual memory location for the constant and/or want it to be allocated only once, then use variable rather than an enum. - Jonathan M Davis
Nov 13 2011
prev sibling next sibling parent so <so so.so> writes:
On Mon, 14 Nov 2011 02:09:40 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 It depends entirely on what you're trying to do. If you understand how
 manifest constants work, then they can be quite advantageous. What you
 probably really want for arrays though is not an enum but just a const or
 immutable module variable.

 immutable a = [3, 1, 2];

 Otherwise, you're allocating a new array every time you use the enum.  
 So, use
 a manifest constant when you want to avoid having it take up any memory  
 but
 don't care about whatever allocations may occur when it's used (primitive
 types such as ints being a prime example), whereas if you want an actual
 memory location for the constant and/or want it to be allocated only  
 once,
 then use variable rather than an enum.

 - Jonathan M Davis

Wait a second, it is definitely a bug. You can't modify an enum. "immutable a = [3, 1, 2];" is practically "enum a = [3, 1, 2];".
Nov 13 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 14, 2011 02:16:57 so wrote:
 On Mon, 14 Nov 2011 02:09:40 +0200, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 It depends entirely on what you're trying to do. If you understand how
 manifest constants work, then they can be quite advantageous. What you
 probably really want for arrays though is not an enum but just a const
 or
 immutable module variable.
 
 immutable a = [3, 1, 2];
 
 Otherwise, you're allocating a new array every time you use the enum.
 So, use
 a manifest constant when you want to avoid having it take up any memory
 but
 don't care about whatever allocations may occur when it's used
 (primitive
 types such as ints being a prime example), whereas if you want an actual
 memory location for the constant and/or want it to be allocated only
 once,
 then use variable rather than an enum.
 
 - Jonathan M Davis

Wait a second, it is definitely a bug. You can't modify an enum. "immutable a = [3, 1, 2];" is practically "enum a = [3, 1, 2];".

No. Those are two _very_ different things. immutable a = [3, 1, 2]; creates a variable of type immutable(int[]). You takes up memory. You can take its address - e.g. &a. It exists even if you don't use it at all. enum a = [3, 1, 2]; on the other hand, doesn't create any variable at all. It's what's called a manifest constant. It's a placeholder. It takes up no memory. You can't take its address. If you never use it, it doesn't end up in the generated program at all. Every use of a effectively copies the value of a to wherever its used. So, enum a = [3, 1, 2]; sort(a); is _identical_ to sort([3, 1, 2]); That means that if you use a in any code, it's copied in that code such that you could end up allocating a lot of memory that you didn't intend to if you heavily use manifest constants which are strings or arrays. But it also means that you get a new copy to use at each location, which can also be useful. And for types that don't end up on the heap (e.g. int), there's really no cost to it, and it actually is _more_ efficient (albeit marginally), since it avoids having to allocate any memory for the enum itself, since it's not a variable. - Jonathan M Davis
Nov 13 2011
prev sibling next sibling parent so <so so.so> writes:
On Mon, 14 Nov 2011 02:50:50 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 No. Those are two _very_ different things.

 immutable a = [3, 1, 2];

 creates a variable of type immutable(int[]). You takes up memory. You  
 can take
 its address - e.g. &a. It exists even if you don't use it at all.

 enum a = [3, 1, 2];

 on the other hand, doesn't create any variable at all. It's what's  
 called a
 manifest constant. It's a placeholder. It takes up no memory. You can't  
 take
 its address. If you never use it, it doesn't end up in the generated  
 program
 at all. Every use of a effectively copies the value of a to wherever its  
 used.
 So,

 enum a = [3, 1, 2];
 sort(a);

 is _identical_ to

 sort([3, 1, 2]);

 That means that if you use a in any code, it's copied in that code such  
 that
 you could end up allocating a lot of memory that you didn't intend to if  
 you
 heavily use manifest constants which are strings or arrays. But it also  
 means
 that you get a new copy to use at each location, which can also be  
 useful. And
 for types that don't end up on the heap (e.g. int), there's really no  
 cost to
 it, and it actually is _more_ efficient (albeit marginally), since it  
 avoids
 having to allocate any memory for the enum itself, since it's not a  
 variable.

 - Jonathan M Davis

Trying to remember the discussions on digitalmars.D regarding enum/immutable. There were contradicting opinions. You ask the same question at two different times, you get at least two different answers, especially on this matter. There was/is not a consensus on how it works or should work. While i think enum/immutable are not same thing. Because with enum you mean it is a compile time value and compile time only, but immutable might rely on runtime initializers. So it differs here. This just means enum gives you more guaranties, not less. enum a = 5; immutable b = 6; a = 3; b = 3; If the above code is invalid for both enum and immutable, so should the "in-place" sort case. I understand how it works, but it doesn't make any sense why it should work that way.
Nov 13 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 14, 2011 04:03:54 so wrote:
 Trying to remember the discussions on digitalmars.D regarding
 enum/immutable. There were contradicting opinions.
 You ask the same question at two different times, you get at least two
 different answers, especially on this matter.
 There was/is not a consensus on how it works or should work.
 
 While i think enum/immutable are not same thing. Because with enum you
 mean it is a compile time value and compile time only, but immutable might
 rely on runtime initializers. So it differs here. This just means enum
 gives you more guaranties, not less.
 
 enum a = 5;
 immutable b = 6;
 a = 3;
 b = 3;
 
 If the above code is invalid for both enum and immutable, so should the
 "in-place" sort case. I understand how it works, but it doesn't make any
 sense why it should work that way.

It works exactly as it's designed to work. There are people who disagree with how it's designed, but it works exactly as designed. The above is invalid, because you can't assign to an enum - it's not a variable - and you can't assign to an immutable variable. sort works with an enum because you're passing it a valid array which is _not_ immutable. It's a new array just as if you'd passed it the enum's value directly. That may be surprising to some, but it's exactly as designed and is not a bug. If you think that it should work differently, then you're arguing against the current design, which is a completely different discussion. - Jonathan M Davis
Nov 13 2011
prev sibling next sibling parent so <so so.so> writes:
On Mon, 14 Nov 2011 10:27:21 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 It is the right design. Why should enum imply const or immutable? (or  
 inout, for that matter). They are completely orthogonal.

 enum Enum{
      opt1,
      opt2,
 }

 void main(){
      auto moo = Enum.opt1;
      moo = Enum.opt2; // who would seriously want an error here???
 }

You are missing the point, nobody asked that. You are assigning it to auto, a runtime variable. Which was asked was about modifying a constant, sort(a) means sort a in-place. So you cant do: immutable a; sort(a); But with current design you can do: enum a; sort(a); Which is to me, quite wrong.
Nov 14 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
      enum a = [3, 1, 2];
      enum s = sort(a);
      assert(equal(a, [3, 1, 2]));
      assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used on an array. Consider that: enum a = "hello"; foo(a); Does not allocate heap memory, even though "hello" is a reference type. However: enum a = ['h', 'e', 'l', 'l', 'o']; foo(a); Allocates heap memory every time a is *used*. This is counter-intuitive, one uses enum to define things using the compiler, not during runtime. It's used to invoke CTFE, to avoid heap allocation. It's not a glorified #define macro. The deep issue here is not that enum is used as a manifest constant, but rather the fact that enum can map to a *function call* rather than the *result* of that function call. Would you say this should be acceptable? enum a = malloc(5); foo(a); // calls malloc(5) and passes the result to foo. If the [...] form is an acceptable enum, I contend that malloc should be acceptable as well. My view is that enum should only be acceptable on data that is immutable, or implicitly cast to immutable, and should *never* map to an expression that calls a function during runtime. -Steve
Nov 14 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used on an array. Consider that: enum a = "hello"; foo(a); Does not allocate heap memory, even though "hello" is a reference type. However: enum a = ['h', 'e', 'l', 'l', 'o']; foo(a); Allocates heap memory every time a is *used*. This is counter-intuitive, one uses enum to define things using the compiler, not during runtime. It's used to invoke CTFE, to avoid heap allocation. It's not a glorified #define macro. The deep issue here is not that enum is used as a manifest constant, but rather the fact that enum can map to a *function call* rather than the *result* of that function call. Would you say this should be acceptable? enum a = malloc(5); foo(a); // calls malloc(5) and passes the result to foo. If the [...] form is an acceptable enum, I contend that malloc should be acceptable as well.

a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l', 'o']. enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on every use But malloc is not CTFE-able, that is why it fails.

You are comparing apples to oranges here. Whether it's CTFE able or not has nothing to do with it, since the code is executed at runtime, not compile time.
 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

Too restrictive imho.

It allows the compiler to evaluate the enum at compile time, and store any referenced data in ROM, avoiding frequent heap allocations (similar to string literals). IMO, type freedom is lower on the priority list than performance. You can already define a symbol that calls arbitrary code at runtime: property int[] a() { return [3, 1, 2];} Why should we muddy enum's goals with also being able to call functions during runtime?
 and should *never* map to an
 expression that calls a function during runtime.

Well, I would not miss that at all. But being stored as enum should not imply restrictions on type qualifiers.

The restrictions are required in order to avoid calling runtime functions for enum usage. Without the restrictions, you must necessarily call runtime functions for any reference-based types (to avoid modifying the original). Note that I'm not saying literals in general should not trigger heap allocations, I'm saying assigning such literals to enums should require unrestricted copying without runtime function calls. I don't think you would miss this as much as you think. Assigning a non-immutable array from an immutable one is as easy as adding a .dup, and then the code is more clear that an allocation is taking place. -Steve
Nov 14 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Nov 2011 14:59:50 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 08:37 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 13:37:18 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/14/2011 02:13 PM, Steven Schveighoffer wrote:
 On Mon, 14 Nov 2011 03:27:21 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/14/2011 01:02 AM, bearophile wrote:
 Jonathan M Davis:

 import std.algorithm;
 void main() {
 enum a = [3, 1, 2];
 enum s = sort(a);
 assert(equal(a, [3, 1, 2]));
 assert(equal(s, [1, 2, 3]));
 }

It's not a bug. Those an manifest constants. They're copy-pasted into whatever code you used them in. So, enum a = [3, 1, 2]; enum s = sort(a); is equivalent to enum a = [3, 1, 2]; enum s = sort([3, 1, 2]);

You are right, there's no DMD bug here. Yet, it's a bit surprising to sort in-place a "constant". I have to stop thinking of them as constants. I don't like this design of enums...

It is the right design. Why should enum imply const or immutable? (or inout, for that matter). They are completely orthogonal.

There is definitely some debatable practice here for wherever enum is used on an array. Consider that: enum a = "hello"; foo(a); Does not allocate heap memory, even though "hello" is a reference type. However: enum a = ['h', 'e', 'l', 'l', 'o']; foo(a); Allocates heap memory every time a is *used*. This is counter-intuitive, one uses enum to define things using the compiler, not during runtime. It's used to invoke CTFE, to avoid heap allocation. It's not a glorified #define macro. The deep issue here is not that enum is used as a manifest constant, but rather the fact that enum can map to a *function call* rather than the *result* of that function call. Would you say this should be acceptable? enum a = malloc(5); foo(a); // calls malloc(5) and passes the result to foo. If the [...] form is an acceptable enum, I contend that malloc should be acceptable as well.

a indeed refers to the result of the evaluation of ['h', 'e', 'l', 'l', 'o']. enum a = {return ['h', 'e', 'l', 'l', 'o'];}(); // also allocates on every use But malloc is not CTFE-able, that is why it fails.

You are comparing apples to oranges here. Whether it's CTFE able or not has nothing to do with it, since the code is executed at runtime, not compile time.

The code is executed at compile time. It is just that the value is later created by allocating at runtime. enum foo = {writeln("foo"); return [1,2,3];}(); // fails, because writeln is not ctfe-able.

Look at the code generated for enum a = [1, 2, 3]. using a is replaced with a call to _d_arrayliteral. There is no CTFE going on.
 My view is that enum should only be acceptable on data that is
 immutable, or implicitly cast to immutable,

Too restrictive imho.

It allows the compiler to evaluate the enum at compile time, and store any referenced data in ROM, avoiding frequent heap allocations (similar to string literals). IMO, type freedom is lower on the priority list than performance. You can already define a symbol that calls arbitrary code at runtime: property int[] a() { return [3, 1, 2];} Why should we muddy enum's goals with also being able to call functions during runtime?

As I said, I would not miss the capability of enums to create mutable arrays a lot. Usually you don't want that behavior, and explicitly .dup-ing is just fine. But I think it is a bit exaggerated to say enums can call functions at runtime. It is up to the compiler how to implement the array allocation.

The compiler has no choice. It must develop the array at runtime, or else the type allows one to modify the source value (just like in D1 how you could modify string literals). In essence, the compiler is creating a new copy for every usage (and building it from scratch).
 and should *never* map to an
 expression that calls a function during runtime.

Well, I would not miss that at all. But being stored as enum should not imply restrictions on type qualifiers.

The restrictions are required in order to avoid calling runtime functions for enum usage. Without the restrictions, you must necessarily call runtime functions for any reference-based types (to avoid modifying the original).

Yes, I don't need that. But I don't really want compile time capabilities hampered. enum a = [2,1,4]; enum b = sort(a); // should be fine.

I was actually surprised that this compiles. But this should not be a problem even if a was immutable(int)[]. sort should be able to create a copy of an immutable array in order to sort it. It doesn't matter the performance hit, because this should all be done at compile time.
 Note that I'm not saying literals in general should not trigger heap
 allocations, I'm saying assigning such literals to enums should require
 unrestricted copying without runtime function calls.

Yes, I get that. And I think it makes sense. But I am not (yet?) convinced that the solution to make all enums non-assignable, head-mutable and tail-immutable is satisfying.

When I see an enum, I think "evaluated at compile time". No matter how complex it is to build that value, it should be built at compile-time and *used* at runtime. No complex function calls should be done at runtime, an enum is a value. I did an interesting little test: import std.algorithm; import std.stdio; int[] foo(int[] x) { return x ~ x; } enum a = [3, 1, 2]; enum b = sort(foo(foo(foo(a)))); void main() { writeln(b); } Want to see the assembly generated for the writeln call? push 018h mov EAX,offset FLAT:_D11TypeInfo_Ai6__initZ SYM32 push EAX call _d_arrayliteralTX PC32 add ESP,8 mov ECX,1 mov [EAX],ECX mov 4[EAX],ECX mov 8[EAX],ECX mov 0Ch[EAX],ECX mov 010h[EAX],ECX mov 014h[EAX],ECX mov 018h[EAX],ECX mov 01Ch[EAX],ECX mov EDX,2 mov 020h[EAX],EDX mov 024h[EAX],EDX mov 028h[EAX],EDX mov 02Ch[EAX],EDX mov 030h[EAX],EDX mov 034h[EAX],EDX mov 038h[EAX],EDX mov 03Ch[EAX],EDX mov EBX,3 mov 040h[EAX],EBX mov 044h[EAX],EBX mov 048h[EAX],EBX mov 04Ch[EAX],EBX mov 050h[EAX],EBX mov 054h[EAX],EBX mov 058h[EAX],EBX mov 05Ch[EAX],EBX mov ECX,EAX mov EAX,018h mov -8[EBP],EAX mov -4[EBP],ECX mov EDX,-4[EBP] mov EAX,-8[EBP] push EDX push EAX call _D3std5stdio76__T7writelnTS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZ7writelnFS3std5range37__T11SortedRangeTAiVAyaa5_61203c2062Z11SortedRangeZv PC32 Really? That's a better solution than using ROM space to store the result of the expression as evaluated at compile time? The worst part is that this will be used *EVERY TIME* I use the enum b (even if I pass it as a const array).
 I don't think you would miss this as much as you think. Assigning a
 non-immutable array from an immutable one is as easy as adding a .dup,
 and then the code is more clear that an allocation is taking place.

It would be somewhat odd. enum a = [2,1,4]; enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

I don't think .dup should be necessary at compile time. Creating a sorted copy of an immutable array should be quite doable.
 enum c = a.dup;   // does this implicitly convert to immutable, or what  
 happens here?

Either a compile error (cannot store mutable reference data as an enum), or an implicit conversion back to immutable.
 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

Again, I don't think .dup would be used for dependent enums, I was rather thinking dup would be used where you need a mutable copy of an array during enum usage in normal code. -Steve
Nov 14 2011
prev sibling next sibling parent so <so so.so> writes:
On Mon, 14 Nov 2011 15:13:20 +0200, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 There is definitely some debatable practice here for wherever enum is  
 used on an array.

 Consider that:

 enum a = "hello";

 foo(a);

 Does not allocate heap memory, even though "hello" is a reference type.

 However:

 enum a = ['h', 'e', 'l', 'l', 'o'];

 foo(a);

 Allocates heap memory every time a is *used*.  This is  
 counter-intuitive, one uses enum to define things using the compiler,  
 not during runtime.  It's used to invoke CTFE, to avoid heap  
 allocation.  It's not a glorified #define macro.

Thanks Steve, this is exactly what i was trying to say.
 The deep issue here is not that enum is used as a manifest constant, but  
 rather the fact that enum can map to a *function call* rather than the  
 *result* of that function call.

 Would you say this should be acceptable?

 enum a = malloc(5);

 foo(a); // calls malloc(5) and passes the result to foo.

 If the [...] form is an acceptable enum, I contend that malloc should be  
 acceptable as well.

 My view is that enum should only be acceptable on data that is  
 immutable, or implicitly cast to immutable, and should *never* map to an  
 expression that calls a function during runtime.

I agree, enum in its current shape implies just that. -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 14 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Nov 2011 16:28:52 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
 Look at the code generated for enum a = [1, 2, 3]. using a is replaced
 with a call to _d_arrayliteral. There is no CTFE going on.

There is some ctfe going on, but the compiler has to allocate the result anew every time it is used. So there is also some runtime overhead. To make my point clearer: int foo(){return 100;} enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100, 100]; void main(){ auto x = a; // this does *not* call foo. But it allocates a new array literal }

Yes, you are right. The issue is that the resulting array is initialized at runtime, not that CTFE is being avoided. After doing some of these tests, I have a better understanding of the issues.
 The compiler has no choice. It must develop the array at runtime, or
 else the type allows one to modify the source value (just like in D1 how
 you could modify string literals). In essence, the compiler is creating
 a new copy for every usage (and building it from scratch).

That is a quality of implementation issue. The language semantics do not require that.

The language semantics require that if an enum type points at mutable data, a runtime allocation *must* occur to avoid corruption of literals. I think a rule requiring an enum to be immutable or implicitly cast to immutable puts the burden of runtime allocation on the coder, making it clear what's going on. In C++, novice coders typically pass classes by value not knowing what a horrible thing this is doing. Then they are puzzled why the code is so slow, the syntax is so short! This is another case of a hidden allocation which can be either avoided or made visible.
 enum a = [2,1,4];
 enum b = sort(a); // should be fine.

I was actually surprised that this compiles. But this should not be a problem even if a was immutable(int)[]. sort should be able to create a copy of an immutable array in order to sort it. It doesn't matter the performance hit, because this should all be done at compile time.

It does not, but explicitly calling .dup works immutable x = [3,2,1]; immutable y = sort(x.dup);

I'm saying sort (or another symbol, ctfesort?) can be made to do the dup automatically for you so you don't have to have it when using ctfe. Extra allocations during CTFE cost nothing (well, with a properly GC'd compiler, that is). Update: I have a better idea, see below.
 When I see an enum, I think "evaluated at compile time". No matter how
 complex it is to build that value, it should be built at compile-time
 and *used* at runtime. No complex function calls should be done at
 runtime, an enum is a value.

Exactly. Therefore you assign from it by copying it. Compare to static array. int[10] x = [1,2,3,4,5,6,7,8,9,0]; x still needs to be initialized at runtime.

Yes, but this is spelled out because copying a static array requires moving data. However, this does *not* require a hidden allocation (even though it does do a hidden allocation currently). I'm not worried about copying data as much as I am about hidden allocations. Hidden allocations are a huge drag on performance. Every time you allocate, you need to take a global GC lock, and it's an unbounded operation (doing one allocation could run a collection cycle).
 I did an interesting little test:


[snip]
 That just tells us that DMD sucks at generating code for array literals.

 This generates identical code:

 import std.stdio;

 void main() {
      writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3,  
 3, 3, 3, 3, 3]);
 }

 You don't need enums for that.


 What it actually should for both our examples is more like the following:

 import std.stdio;

 immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,  
 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];

 void main() {
      writeln(_somewhereinrom.dup);
 }

 push   %ebp
 mov    %esp,%ebp
 pushl  0x8097184
 pushl  0x8097180
 mov    $0x80975c8,%eax
 push   %eax
 call   8079470 <_adDupT>
 add    $0xc,%esp
 push   %edx
 push   %eax
 call   807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
 xor    %eax,%eax
 pop    %ebp
 ret


 If writeln would actually be const correct, the compiler could even get  
 rid of the allocation.

That is the idea. Get rid of the hidden allocation. Writeln *is* const correct, it can certainly print immutable(int)[]. The issue is not writeln, it's what the type of the array literal/enum is. Technically, an array literal is equivalent to an enum, and should follow the same rules.
 This is not about enums that much, it is about array literals.

 The fact that stack static array initialization allocates is one of DMDs  
 bigger warts.

 Look at the ridiculous code generated for the following example:

 void main() {
      int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3,  
 3, 3, 3, 3, 3, 3];
      writeln(x);
 }

Yes, these are all cases of the same issue.
 enum a = [2,1,4];
 enum b = sort(a.dup); // what exactly is that 'a.dup' thing?

I don't think .dup should be necessary at compile time. Creating a sorted copy of an immutable array should be quite doable.

I agree, phobos won't currently do it though.

This is easily fixed. But maybe there is a better way (see below).
 enum d = sort(c); // does not work?

 enum e = foo(a.dup, b.dup, c.dup, d.dup);

Again, I don't think .dup would be used for dependent enums, I was rather thinking dup would be used where you need a mutable copy of an array during enum usage in normal code.

But if the type of a,b,c,d is immutable(int)[] and foo is a function that takes 4 int[]s then the .dup's are necessary to pass type checking.

What about this idea: At a global level, expressions that result in CTFE being triggered, can be implicitly cast from mutable to immutable and vice versa via a deep-dup. This allows you to use enums as parameters to functions accepting mutable references. Then enums that are derived from other enums do not need to follow the same rules as runtime code that uses the enums. This of course, only happens at the global-expressions level, as function internals must compile at runtime as well. What I thought of as a solution was to create CTFE only functions that wrap other functions to do a dup. But you wouldn't want to do this during runtime, because dup is expensive. During compile time, dup costs nothing. This idea essentially takes the place of that boilerplate code. -Steve
Nov 15 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:

 Yes, but this is spelled out because copying a static array requires
 moving data. However, this does *not* require a hidden allocation (even
 though it does do a hidden allocation currently).

 I'm not worried about copying data as much as I am about hidden
 allocations. Hidden allocations are a huge drag on performance. Every
 time you allocate, you need to take a global GC lock, and it's an
 unbounded operation (doing one allocation could run a collection cycle).

You don't actually _need_ a global GC lock. It is just how it is implemented in this case.

This is all fine in theory. I haven't seen any implementations. But memory allocations are not cheap, even without a GC lock.
 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the  
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.
 That is the idea. Get rid of the hidden allocation. Writeln *is* const
 correct, it can certainly print immutable(int)[].

Well, there is a function called writeln that can do that. That is a different function. But the one that gets actually called is not const correct as well. This is writeln: // Most general instance void writeln(T...)(T args) if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) : const(char)[])) { stdout.write(args, '\n'); } => writeln([1,2,3]); // modulo IFTY: writeln!(int[])([1,2,3]); // const correct? writeln!(int[])([1,2,3].idup); // nope! Error: cannot implicitly convert expression (_adDupT(& _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[], it will compile. Simple test: immutable(int)[] a = [1, 2, 3]; writeln(a); works with 2.056.
 The issue is not
 writeln, it's what the type of the array literal/enum is.

 Technically, an array literal is equivalent to an enum, and should
 follow the same rules.

Remember that immutable is transitive. That can really get in your way in this case.

If the data is stored in ROM, it should be typed as immutable. If it's not, then it could be modified. The only other option is heap allocation and construction on use, which I've already said is undesirable. If we start from "all array literals and enums that contain references store the referenced data in ROM," then we will find ways to deal with any inconveniences. It's all a matter of what's more important in a system language, performance or convenience.
 What about this idea:

 At a global level, expressions that result in CTFE being triggered, can
 be implicitly cast from mutable to immutable and vice versa via a
 deep-dup. This allows you to use enums as parameters to functions
 accepting mutable references. Then enums that are derived from other
 enums do not need to follow the same rules as runtime code that uses the
 enums.

 This of course, only happens at the global-expressions level, as
 function internals must compile at runtime as well.

 What I thought of as a solution was to create CTFE only functions that
 wrap other functions to do a dup. But you wouldn't want to do this
 during runtime, because dup is expensive. During compile time, dup costs
 nothing. This idea essentially takes the place of that boilerplate code.

That could work, but I think this is cluttering up the semantics a bit.

It's just a shortcut. In essence, during compile time execution, .dups are added for you because they are free. During runtime, they are not free, so you must add them. -Steve
Nov 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable. Make no mistake, this doesn't cover every current instance array literals, such as ones which contain necessarily-heap-allocated entities. Those would be covered by a library function (e.g. array(...)). But those are not array literals anyways, they are constructors.
 Well, there is a function called writeln that can do that. That is a
 different function. But the one that gets actually called is not const
 correct as well.


 This is writeln:

 // Most general instance
 void writeln(T...)(T args)
 if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
 const(char)[]))
 {
 stdout.write(args, '\n');
 }


 =>
 writeln([1,2,3]);
 // modulo IFTY:
 writeln!(int[])([1,2,3]);
 // const correct?
 writeln!(int[])([1,2,3].idup); // nope!

 Error: cannot implicitly convert expression (_adDupT(&
 _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[], it will compile. Simple test: immutable(int)[] a = [1, 2, 3]; writeln(a); works with 2.056.

That is like saying void foo(int[] a){...} // prints a void bar(immutable(int)[] a){...} // prints a int[] a = [1,2,3]; immutable(int)[] b = [1, 2, 3]; foo(a); bar(b); "=> Oh, bar is const correct because I can call foo with mutable data and bar with immutable data." foo is writeln!(int[]) bar is writeln!(immutable(int)[]) in effect, if you do: writeln("hello".dup); The compiler cannot assume that the IFTI'd writeln!(char[]) will not change the argument, ergo it cannot optimize away the .dup, even though it would be a valid transformation as long as the programmer does not make the program semantics depend on the identity relation on immutable references (doing so defeats the purpose of immutability).

writeln is not contractually const correct, but that's only because most of D is not const correct either. For example, Object.toString is not const, so if you const-ified all writeln args, you couldn't print objects. But writeln itself does not change any data (a rogue toString might change data, but that is not common practice). Once D becomes const correct, writeln *can* apply const to all it's parameters.
 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

Both are more important. It is D. Everything you need to do is make the enum static immutable instead. D is also a scripting language and an application programming language. auto a = [new Foo, new Bar, new Qux]; // I want that to work.

auto a = array(new Foo, new Bar, new Qux); The one case which is difficult to do is initializing a fixed-size array with a literal that uses runtime data. I suppose we'd need a function that returns a fixed-sized array made of its arguments, and doing the init builds it in place. i.e.: Object[3] objs = array_fixed(new Foo, new Bar, new Qux); would not do any moving of references, it would construct the fixed sized array in-place. Initializing fixed sized arrays with array literals already needs attention anyway. -Steve
Nov 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Nov 2011 15:00:16 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 The one case which is difficult to do is initializing a fixed-size array  
 with a literal that uses runtime data.  I suppose we'd need a function  
 that returns a fixed-sized array made of its arguments, and doing the  
 init builds it in place.  i.e.:

 Object[3] objs = array_fixed(new Foo, new Bar, new Qux);

 would not do any moving of references, it would construct the fixed  
 sized array in-place.  Initializing fixed sized arrays with array  
 literals already needs attention anyway.

one benefit here, we could use auto: auto objs = array_fixed(new Foo, new Bar, new Qux); -Steve
Nov 16 2011
prev sibling next sibling parent =?utf-8?Q?Simen_Kj=C3=A6r=C3=A5s?= <simen.kjaras gmail.com> writes:
On Wed, 16 Nov 2011 21:00:16 +0100, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable.

immutable(int[][]) a = [[1]]; int[][] b = a.dup; // Fails. The problem is with more than a single layer of indirection.
Nov 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable.

string is not an immutable type. It is immutable(char)[] and char is a built-in value type. static assert(isMutable!string);

It fits my definition of a valid enum reference type (immutable or implicitly castable to immutable). The point is that the data referenced is stored in ROM and therefore a) immutable and b) fully defined at compile-time.
 Make no mistake, this doesn't cover every current instance array
 literals, such as ones which contain necessarily-heap-allocated
 entities. Those would be covered by a library function (e.g.
 array(...)). But those are not array literals anyways, they are
 constructors.

It is true that they are constructors, but they are currently also called array literals.

I'm looking to redefine what a literal means in D.
 writeln is not contractually const correct, but that's only because most
 of D is not const correct either. For example, Object.toString is not
 const, so if you const-ified all writeln args, you couldn't print
 objects. But writeln itself does not change any data (a rogue toString
 might change data, but that is not common practice).

 Once D becomes const correct, writeln *can* apply const to all it's
 parameters.

Ok, I see. writeln still could be const-correct in principle. It would need to apply const to the parameters that have a const toString method. But the language is not powerful enough to do that. There is no way to intercept IFTI... I am commonly missing that feature.

I have an enhancement request in for intercepting IFTI (not sure if it applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998
 If the data is stored in ROM, it should be typed as immutable. If it's
 not, then it could be modified. The only other option is heap  
 allocation
 and construction on use, which I've already said is undesirable.

 If we start from "all array literals and enums that contain references
 store the referenced data in ROM," then we will find ways to deal with
 any inconveniences. It's all a matter of what's more important in a
 system language, performance or convenience.

Both are more important. It is D. Everything you need to do is make the enum static immutable instead. D is also a scripting language and an application programming language. auto a = [new Foo, new Bar, new Qux]; // I want that to work.

auto a = array(new Foo, new Bar, new Qux);

Requiring that works modulo template bloat and breakage of existing code. Also, it does not really buy us anything imho.

I think the bloat is a wash. Every time I use an array literal, there is a complete instantiation of what would be in an array template inline in the calling function. Yes, it breaks code. It's worth it. To avoid a heap allocation for [1, 2, 3] is worth breaking code that uses runtime data in an array literal. The compiler can be helpful in the error message: Error somefile.d(345): array literals cannot contain runtime data. Maybe you meant: array(new Foo, new Bar, new Qux); I want to point out that currently there is *NO* way to make an immutable array literal that lives in ROM. This is unacceptable. -Steve
Nov 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Nov 2011 16:47:58 -0500, Simen Kj=C3=A6r=C3=A5s <simen.kjaras=
 gmail.com>  =

wrote:

 On Wed, 16 Nov 2011 21:00:16 +0100, Steven Schveighoffer  =

 <schveiguy yahoo.com> wrote:

 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>  =


 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch> =




 wrote:

 Note that this is an explicit allocation:

 int[] a =3D [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know t=





 involved types.

As I've said, there are already ways to explicitly allocate memory.=




 suggested replacement for this is:

 int[] a =3D array(1, 2, 3);

 And you could always do:

 int[] a =3D [1, 2, 3].dup;

 Nobody complains about having to do:

 char[] a =3D "hello".dup;

 I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not =



 arbitrary reference types.

string is a reference type. We hear no complaints about strings bein=


 stored in ROM and the type of literals being immutable.

immutable(int[][]) a =3D [[1]]; int[][] b =3D a.dup; // Fails. The problem is with more than a single layer of indirection.

Solved via library: auto b =3D a.deepdup; -Steve
Nov 16 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Nov 2011 17:39:16 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 10:56 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
 On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
 On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr gmx.ch>
 wrote:

 Note that this is an explicit allocation:

 int[] a = [1,2,3]; // just as explicit as a NewExpression

 Only the enums "hide" it sometimes, but actually you should know  
 the
 involved types.

As I've said, there are already ways to explicitly allocate memory. A suggested replacement for this is: int[] a = array(1, 2, 3); And you could always do: int[] a = [1, 2, 3].dup; Nobody complains about having to do: char[] a = "hello".dup; I don't see why we couldn't do the same for all array literals.

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type. We hear no complaints about strings being stored in ROM and the type of literals being immutable.

string is not an immutable type. It is immutable(char)[] and char is a built-in value type. static assert(isMutable!string);

It fits my definition of a valid enum reference type (immutable or implicitly castable to immutable). The point is that the data referenced is stored in ROM and therefore a) immutable and b) fully defined at compile-time.

Indeed. But fact is, the data that is qualified with immutable is not of reference type therefore it behaves nicely. And you don't get that in the general case.

In the general case, there is always a library function for construction. In other words, what the compiler currently does for array literals can be done in a library. But a library cannot create ROM space. The compiler-based features (and CTFE in general) should be helping us create things at compile time, not at run time. We already have the tools to construct arrays at runtime.
 I am looking for something like this:

 template writeln(T...)(T){
      alias writelnImpl!(writelnInferConst!T) writeln;
 }

 (it is even trivial to parse, unlike normal function definitions!)

What does writelnInferConst!T do? I'm afraid I'm not getting what you are saying. I was thinking writeln should do this: void writeln(T...)(const T args) {...}
 I have an enhancement request in for intercepting IFTI (not sure if it
 applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

It has a complexity of at least 2^numparams and probably all kinds of odd implications for the compiler internals that would lead to a buggy implementation.

I'm not a compiler writer, but I don't see how this is the case.
 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation for  
 IFTY:

 void bar(T: short)(T t){...}

 void main(){
      bar(1); // ok
 }

The issue with all this is, IFTI doesn't work that way: void foo(T: short)(T t) {} void main() { foo(1); } testifti.d(5): Error: template testifti.foo(T : short) does not match any function template declaration testifti.d(5): Error: template testifti.foo(T : short) cannot deduce template function from argument types !()(int) IFTI decides the types of literals before trying to find a proper template to instantiate. Any possibility to intercept the decision of literal type or of instantiation would be useful. I think that it's better suited to the constraints, because there is more power there. But I'm not sure. If you can find a more straightforward way, I'm all for it. In any case, I need to update the bug report, because the general case is if foo has an overload. For instance: foo(short s); foo(wstring w); foo2 should be able to call both with 1 and "hello" without issue. My driving use case to create the enhancement was creating a wrapper type that intercepted function calls. I planned to use opDispatch, but it didn't quite work with literals.
 I think the bloat is a wash. Every time I use an array literal, there is
 a complete instantiation of what would be in an array template inline in
 the calling function.

With the template function you have that too because the calling function builds the arguments. It is just that you also get the template bloat and an additional function call. Unless it is inlined, of course.

You are right, I hadn't thought of that. But what about this: most array literals are actually literals (made up of CTFE-decided values). Making them ROM-stored would *eliminate* bloat as compared to the current implementation. Also, given how template-centric D and phobos are, I think at some point we need to examine how to minimize template bloat in general, by coalescing identical code into one function, or not emitting functions that are always inlined, not to mention avoiding storing templates only used at compile-time in the code (e.g. isInputRange).
 Yes, it breaks code. It's worth it. To avoid a heap allocation for [1,
 2, 3] is worth breaking code that uses runtime data in an array literal.
 The compiler can be helpful in the error message:

 Error somefile.d(345): array literals cannot contain runtime data. Maybe
 you meant:
 array(new Foo, new Bar, new Qux);

 I want to point out that currently there is *NO* way to make an
 immutable array literal that lives in ROM. This is unacceptable.

There is: void main(){ static immutable x = [1,2,3]; }

Seems rather odd you should have to jump through these hoops to get the compiler to use ROM space. But I concede that I did not know of this trick. It does not sway my opinion that CTFE should produce ROM-stored references, and library function should be used for runtime-constructed references. -Steve
Nov 17 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 Nov 2011 18:25:48 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/16/2011 11:39 PM, Timon Gehr wrote:
 I think this is a better solution:

 void foo2(T: ParameterTypeTuple!foo[0])(T t){foo(t);}

 Then it is just a matter of applying proper value range propagation for
 IFTY:

 void bar(T: short)(T t){...}

 void main(){
 bar(1); // ok
 }

BTW, this already works for your use case: void foo2(ParameterTypeTuple!foo t){foo(t);}

My use case is incomplete, I minimized it too much. I will update it. -Steve
Nov 17 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 17 Nov 2011 12:31:58 -0500, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 11/17/2011 03:19 PM, Steven Schveighoffer wrote:

 What does writelnInferConst!T do? I'm afraid I'm not getting what you
 are saying.

 I was thinking writeln should do this:

 void writeln(T...)(const T args) {...}

As you pointed out, this cannot print types that have a non-const toString method (caching the result could be a perfectly valid reason for that.)

Caching string representation IMO is not a significant use case. Not only that, but toString should be deprecated anyways in favor of a stream-based system.
 writelnInferConst finds out which parameters can be treated as const and  
 still be printed so that the correct version of writeln may be called.

 For example:

 class Foo{ // can be printed if const
      string toString()const{return "Foo";}
 }

 class Bar{ // cannot be printed if const
      string cache;
      string toString(){return cache!is null?cache:(cache="Bar");}
 }

 template hasConstToString(T){
      enum hasConstToString = is(typeof((const T t){return  
 t.toString();}));
 }

 template writelnInferConstImpl(T...){
      static if(!T.length) alias T X;
      else static if(hasConstToString!(T[0])){
              alias T[0] _;
              alias TypeTuple!(const(_),writelnInferConst!(T[1..$])) X;
      }else
          alias TypeTuple!(T[0],writelnInferConst!(T[1..$])) X;
 }
 template writelnInferConst(T...){alias writelnInferConstImpl!T.X  
 writelnInferConst;} // (bug 6966)


 static  
 assert(is(writelnInferConst!(Foo,Bar,Foo,Foo,Bar)==TypeTuple!(const(Foo),Bar,const(Foo),const(Foo),Bar)));

If your goal is to reduce template bloat, I think this is not the solution. But also note that this still does not guarantee const. I don't really see the point of doing all these templates if you aren't going to guarantee writeln doesn't modify the data.
 The issue with all this is, IFTI doesn't work that way:

 void foo(T: short)(T t) {}

 void main()
 {
 foo(1);
 }

 testifti.d(5): Error: template testifti.foo(T : short) does not match
 any function template declaration
 testifti.d(5): Error: template testifti.foo(T : short) cannot deduce
 template function from argument types !()(int)

 IFTI decides the types of literals before trying to find a proper
 template to instantiate. Any possibility to intercept the decision of
 literal type or of instantiation would be useful. I think that it's
 better suited to the constraints, because there is more power there. But
 I'm not sure. If you can find a more straightforward way, I'm all for  
 it.

 In any case, I need to update the bug report, because the general case
 is if foo has an overload. For instance:

 foo(short s);
 foo(wstring w);

 foo2 should be able to call both with 1 and "hello" without issue.

 My driving use case to create the enhancement was creating a wrapper
 type that intercepted function calls. I planned to use opDispatch, but
 it didn't quite work with literals.

Ok, I see the problem. My proposed IFTI template mechanism would save the day. It would look like this (static foreach would have to be replaced by a recursive mixin template because Walter encountered implementation difficulties). template OverloadsOf(alias symbol){ // should this be in std.traits? alias TypeTuple!(__traits(getOverloads, __traits(parent,symbol), __traits(identifier,symbol))) OverloadsOf; } auto wrapper(alias foo)(ParameterTypeTuple!foo args){ return foo(args); } template opDispatch(string op,T...)(T){ static foreach(foo; OverloadsOf!(mixin(op))){ alias wrapper!foo opDispatch; } static if(OverloadsOf!(mixin(op)).length==0) { // we are dealing with a template function auto opDispatch(T args){ return foo(args); } } }

Pardon my saying so, but this looks horrendous. Not to mention that I don't think it would work. IFTI instantiates templates, it does not look inside instantiated templates for overloads. BTW, your proposed IFTI template mechanism, do you have it stated anywhere? Maybe it fixes the problem I mentioned.
 Seems rather odd you should have to jump through these hoops to get the
 compiler to use ROM space. But I concede that I did not know of this
 trick. It does not sway my opinion that CTFE should produce ROM-stored
 references, and library function should be used for runtime-constructed
 references.

CTFE should produce ROM-stored data iff it is used during run time, I agree on that. However if the enum is typed as mutable, it should create a copy of the ROM-stored data.

Well, this is closer than I thought we were to an agreement. I would agree with this, as long as it could be implicitly cast to immutable or const. i.e.: enum foo = [1, 2, 3]; immutable(int)[] a = foo; // no allocation const(int)[] b = foo; // no allocation -Steve
Nov 17 2011