www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Why can't static arrays be sorted?

reply TheGag96 <thegag96 gmail.com> writes:
I was writing some code today and ran into this oddity that I'd 
never come across before:

     import std.algorithm : sort;
     int[10] arr = [0, 3, 4, 6, 2, 1, 1, 4, 6, 9];
     thing.sort();

This doesn't compile. Obviously the .sort property works, but 
what about static arrays makes them unable to be sorted by 
std.algorithm.sort? Thanks.
Oct 04 2016
next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 4 October 2016 at 20:05:15 UTC, TheGag96 wrote:
 I was writing some code today and ran into this oddity that I'd 
 never come across before:

     import std.algorithm : sort;
     int[10] arr = [0, 3, 4, 6, 2, 1, 1, 4, 6, 9];
     thing.sort();

 This doesn't compile. Obviously the .sort property works, but 
 what about static arrays makes them unable to be sorted by 
 std.algorithm.sort? Thanks.
Static arrays in D are value types. Try: thing[].sort(); This will pass a slice of the array to sort, holding a reference to the static array's data.
Oct 04 2016
prev sibling next sibling parent Adrian Matoga <dlang.spam matoga.info> writes:
On Tuesday, 4 October 2016 at 20:05:15 UTC, TheGag96 wrote:
 I was writing some code today and ran into this oddity that I'd 
 never come across before:

     import std.algorithm : sort;
     int[10] arr = [0, 3, 4, 6, 2, 1, 1, 4, 6, 9];
     thing.sort();

 This doesn't compile. Obviously the .sort property works, but 
 what about static arrays makes them unable to be sorted by 
 std.algorithm.sort? Thanks.
Try: arr[].sort();
Oct 04 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, October 04, 2016 20:05:15 TheGag96 via Digitalmars-d-learn wrote:
 I was writing some code today and ran into this oddity that I'd
 never come across before:

      import std.algorithm : sort;
      int[10] arr = [0, 3, 4, 6, 2, 1, 1, 4, 6, 9];
      thing.sort();

 This doesn't compile. Obviously the .sort property works, but
 what about static arrays makes them unable to be sorted by
 std.algorithm.sort? Thanks.
The problem is that static arrays aren't ranges (calling popFront on them can't work, because their length isn't mutable). However, you can slice a static array to get a dynamic array which _is_ a range. e.g. thing[].sort(); Just make sure that such a dynamic array does not outlive the static array, or it will refer to invalid memory (which would not be a problem in this case). - Jonathan M Davis
Oct 04 2016
parent reply TheGag96 <thegag96 gmail.com> writes:
On Wednesday, 5 October 2016 at 02:19:13 UTC, Jonathan M Davis 
wrote:
 The problem is that static arrays aren't ranges (calling 
 popFront on them can't work, because their length isn't 
 mutable). However, you can slice a static array to get a 
 dynamic array which _is_ a range. e.g.

 thing[].sort();

 Just make sure that such a dynamic array does not outlive the 
 static array, or it will refer to invalid memory (which would 
 not be a problem in this case).

 - Jonathan M Davis
Ah thanks guys. I think I just got used to thinking arrays would always be found to be ranges by the compiler. Good to know!
Oct 05 2016
parent reply pineapple <meapineapple gmail.com> writes:
On Wednesday, 5 October 2016 at 18:19:27 UTC, TheGag96 wrote:
 On Wednesday, 5 October 2016 at 02:19:13 UTC, Jonathan M Davis 
 wrote:
 The problem is that static arrays aren't ranges (calling 
 popFront on them can't work, because their length isn't 
 mutable). However, you can slice a static array to get a 
 dynamic array which _is_ a range. e.g.

 thing[].sort();

 Just make sure that such a dynamic array does not outlive the 
 static array, or it will refer to invalid memory (which would 
 not be a problem in this case).

 - Jonathan M Davis
Ah thanks guys. I think I just got used to thinking arrays would always be found to be ranges by the compiler. Good to know!
Would just like to point out that this is design weirdness on Phobos' part - the library I've been writing does not have this problem.
Oct 05 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, October 05, 2016 19:22:03 pineapple via Digitalmars-d-learn 
wrote:
 On Wednesday, 5 October 2016 at 18:19:27 UTC, TheGag96 wrote:
 On Wednesday, 5 October 2016 at 02:19:13 UTC, Jonathan M Davis

 wrote:
 The problem is that static arrays aren't ranges (calling
 popFront on them can't work, because their length isn't
 mutable). However, you can slice a static array to get a
 dynamic array which _is_ a range. e.g.

 thing[].sort();

 Just make sure that such a dynamic array does not outlive the
 static array, or it will refer to invalid memory (which would
 not be a problem in this case).

 - Jonathan M Davis
Ah thanks guys. I think I just got used to thinking arrays would always be found to be ranges by the compiler. Good to know!
Would just like to point out that this is design weirdness on Phobos' part - the library I've been writing does not have this problem.
It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it. - Jonathan M Davis
Oct 05 2016
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 10/05/2016 12:30 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
 On Wednesday, October 05, 2016 19:22:03 pineapple via 
Digitalmars-d-learn
 wrote:
 Would just like to point out that this is design weirdness on
 Phobos' part - the library I've been writing does not have this
 problem.
It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it.
But algorithms like sort() need not require popFront(). I guess we need another range type: NonShrinkingRandomAccessRange. :) Ali
Oct 05 2016
prev sibling next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis 
wrote:
 Would just like to point out that this is design weirdness on 
 Phobos' part - the library I've been writing does not have 
 this problem.
It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it. - Jonathan M Davis
Just because the static array itself isn't a range doesn't mean that it should be necessary to do unintuitive gymnastics with it just to pass it to functions like `sort`.
Oct 06 2016
next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Thursday, 6 October 2016 at 09:17:08 UTC, pineapple wrote:
 On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis 
 wrote:
 Would just like to point out that this is design weirdness on 
 Phobos' part - the library I've been writing does not have 
 this problem.
It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it. - Jonathan M Davis
Just because the static array itself isn't a range doesn't mean that it should be necessary to do unintuitive gymnastics with it just to pass it to functions like `sort`.
I mean, people post here how often asking why static or dynamic arrays aren't being accepted by Phobos' range functions in their code? Maybe Phobos really ought to consider another approach. Accepting things that are _valid_ as ranges and not only things that are ranges themselves has proven to be an effective strategy in mach.
Oct 06 2016
parent notna <notna.remove.this ist-einmalig.de> writes:
On Thursday, 6 October 2016 at 09:23:19 UTC, pineapple wrote:
 On Thursday, 6 October 2016 at 09:17:08 UTC, pineapple wrote:
 On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis 
 wrote:
 Would just like to point out that this is design weirdness 
 on Phobos' part - the library I've been writing does not 
 have this problem.
It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it. - Jonathan M Davis
Just because the static array itself isn't a range doesn't mean that it should be necessary to do unintuitive gymnastics with it just to pass it to functions like `sort`.
I mean, people post here how often asking why static or dynamic arrays aren't being accepted by Phobos' range functions in their code? Maybe Phobos really ought to consider another approach. Accepting things that are _valid_ as ranges and not only things that are ranges themselves has proven to be an effective strategy in mach.
+1000
Oct 08 2016
prev sibling parent Adrian Matoga <dlang.spam matoga.info> writes:
On Thursday, 6 October 2016 at 09:17:08 UTC, pineapple wrote:
 On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis 
 wrote:
 Would just like to point out that this is design weirdness on 
 Phobos' part - the library I've been writing does not have 
 this problem.
It doesn't even make conceptual sense for a static array to be a range, because you can't remove elements from it. - Jonathan M Davis
Just because the static array itself isn't a range doesn't mean that it should be necessary to do unintuitive gymnastics with it just to pass it to functions like `sort`.
`sort` takes a range. [] gets a range from anything. It's a convention, not gymnastics.
Oct 06 2016
prev sibling parent reply TheGag96 <thegag96 gmail.com> writes:
On Wednesday, 5 October 2016 at 19:30:01 UTC, Jonathan M Davis 
wrote:
 It doesn't even make conceptual sense for a static array to be 
 a range, because you can't remove elements from it.

 - Jonathan M Davis
Interestingly enough, I found that using .each() actually compiles without the [] but (as expected) creates a copy... So, these output different values: thing.each!((ref x) => writeln(&x)); thing[].each!((ref x) => writeln(&x)); Should there me a more consistent behavior here? Even if the behavior is pretty undesired, why can the compiler consider it a range here but not .sort()?
Oct 06 2016
parent reply ag0aep6g <anonymous example.com> writes:
On 10/06/2016 09:54 PM, TheGag96 wrote:
 Interestingly enough, I found that using .each() actually compiles
 without the []
[...]
 why can the compiler consider it a range here but not
 .sort()?
each is not restricted to ranges. It accepts other `foreach`-ables, too. The documentation says that it "also supports opApply-based iterators", but it's really anything that foreach accepts. Relevant piece of the source: https://github.com/dlang/phobos/blob/08c587ead2156c85c30a2bbc18028b5967010682/std/algorithm/iteration.d#L913-L914
Oct 06 2016
parent reply Jon Degenhardt <jond noreply.com> writes:
On Thursday, 6 October 2016 at 20:11:17 UTC, ag0aep6g wrote:
 On 10/06/2016 09:54 PM, TheGag96 wrote:
 Interestingly enough, I found that using .each() actually 
 compiles
 without the []
[...]
 why can the compiler consider it a range here but not
 .sort()?
each is not restricted to ranges. It accepts other `foreach`-ables, too. The documentation says that it "also supports opApply-based iterators", but it's really anything that foreach accepts. [snip]
Thanks! Explains some things. I knew each! was callable in different circumstances than other functional operations, but hadn't quite figured it out. Looks like reduce! and fold! also take iterables. There also appears to be a distinction between the iterator and range cases when a ref parameter is used. As it iterator, each! won't modify the reference. Example: void main() { import std.algorithm : each; int[] dynamicArray = [1, 2, 3, 4, 5]; int[5] staticArray = [1, 2, 3, 4, 5]; dynamicArray.each!((ref x) => x++); assert(dynamicArray == [2, 3, 4, 5, 6]); // modified staticArray.each!((ref x) => x++); assert(staticArray == [1, 2, 3, 4, 5]); // not modified staticArray[].each!((ref x) => x++); assert(staticArray == [2, 3, 4, 5, 6]); // modified } This distinction is a bit on the nuanced side. Is it behaving as it should? --Jon
Oct 08 2016
parent reply TheGag96 <thegag96 gmail.com> writes:
On Saturday, 8 October 2016 at 21:14:43 UTC, Jon Degenhardt wrote:
 This distinction is a bit on the nuanced side. Is it behaving 
 as it should?

 --Jon
I think so? It's not being modified in the second case because the array is being passed by value... "x" there is a reference to an element of the copy created to be passed to each(). I assume there's a good reason why ranges in general are passed by value into these functions -- except in this one case, the stuff inside range types copied when passed by value won't be whole arrays, I'm guessing.
Oct 10 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Monday, October 10, 2016 16:29:41 TheGag96 via Digitalmars-d-learn wrote:
 On Saturday, 8 October 2016 at 21:14:43 UTC, Jon Degenhardt wrote:
 This distinction is a bit on the nuanced side. Is it behaving
 as it should?

 --Jon
I think so? It's not being modified in the second case because the array is being passed by value... "x" there is a reference to an element of the copy created to be passed to each(). I assume there's a good reason why ranges in general are passed by value into these functions -- except in this one case, the stuff inside range types copied when passed by value won't be whole arrays, I'm guessing.
Whether it's by value depends entirely on the type of the range. They're passed around, and copying them has whatever semantics it has. In most cases, it copies the state of the range but doesn't copy all of the elements (e.g. that's what happens with a dynamic array, since it gets sliced). But if a range is a class, then it's definitely a reference type. The only way to properly save the state of a range is to call save. But passing by ref would make no sense at all with input ranges. It would completely kill chaining them. Almost all range-based functions return rvalues. - Jonathan M Davis
Oct 10 2016
parent reply Jon Degenhardt <jond noreply.com> writes:
On Monday, 10 October 2016 at 16:46:55 UTC, Jonathan M Davis 
wrote:
 On Monday, October 10, 2016 16:29:41 TheGag96 via 
 Digitalmars-d-learn wrote:
 On Saturday, 8 October 2016 at 21:14:43 UTC, Jon Degenhardt 
 wrote:
 This distinction is a bit on the nuanced side. Is it 
 behaving as it should?

 --Jon
I think so? It's not being modified in the second case because the array is being passed by value... "x" there is a reference to an element of the copy created to be passed to each(). I assume there's a good reason why ranges in general are passed by value into these functions -- except in this one case, the stuff inside range types copied when passed by value won't be whole arrays, I'm guessing.
Whether it's by value depends entirely on the type of the range. They're passed around, and copying them has whatever semantics it has. In most cases, it copies the state of the range but doesn't copy all of the elements (e.g. that's what happens with a dynamic array, since it gets sliced). But if a range is a class, then it's definitely a reference type. The only way to properly save the state of a range is to call save. But passing by ref would make no sense at all with input ranges. It would completely kill chaining them. Almost all range-based functions return rvalues. - Jonathan M Davis
The example I gave uses ref parameters. On the surface it would seem reasonable to that passing a static array by ref would allow it to be modified, without having to slice it first. The documentation says: // If the range supports it, the value can be mutated in place arr.each!((ref n) => n++); assert(arr == [1, 2, 3, 4, 5]); but, 'arr' is a dynamic array, so technically it's not describing a static array (the opApply case). Expanding the example, using foreach with ref parameters will modify the static array in place, without slicing it. I would have expected each! with a ref parameter to behave the same. At a minimum this could be better documented, but it may also be a bug. Example: T increment(T)(ref T x) { return x++; } void main() { import std.algorithm : each; int[] dynamicArray = [1, 2, 3, 4, 5]; int[5] staticArray = [1, 2, 3, 4, 5]; dynamicArray.each!(x => x++); // Dynamic array by value assert(dynamicArray == [1, 2, 3, 4, 5]); // ==> Not modified dynamicArray.each!((ref x) => x++); // Dynamic array by ref assert(dynamicArray == [2, 3, 4, 5, 6]); // ==> Modified staticArray[].each!((ref x) => x++); // Slice of static array, by ref assert(staticArray == [2, 3, 4, 5, 6]); // ==> Modified staticArray.each!((ref x) => x++); // Static array by ref assert(staticArray == [2, 3, 4, 5, 6]); // ==> Not Modified /* Similar to above, using foreach and ref params. */ foreach (ref x; dynamicArray) x.increment; assert(dynamicArray == [3, 4, 5, 6, 7]); // Dynamic array => Modified foreach (ref x; staticArray[]) x.increment; assert(staticArray == [3, 4, 5, 6, 7]); // Static array slice => Modified foreach (ref x; staticArray) x.increment; assert(staticArray == [4, 5, 6, 7, 8]); // Static array => Modified }
Oct 10 2016
parent reply ag0aep6g <anonymous example.com> writes:
On 10/11/2016 06:24 AM, Jon Degenhardt wrote:
 The example I gave uses ref parameters. On the surface it would seem
 reasonable to that passing a static array by ref would allow it to be
 modified, without having to slice it first.
Your ref parameters are only for the per-element operations. You're not passing the array as a whole by reference. And you can't, because `each` itself takes the whole range by copy. So, the by-ref increments themselves do work, but they're applied to a copy of your original static array. Question is, should `each` 1) take all inputs (ranges, arrays, other foreachables) by reference, or 2) take some inputs (like static arrays) by reference, or 3) take all inputs by value (current behavior)? #1 would break code. Would probably need some deprecating and name shuffling to be acceptable. Would also need to make sure that this is actually the most desirable behavior. #2 would probably create surprising corner cases. I don't think we can tell for sure if a range needs to be passed by reference in order to see updates to its elements. I'd be against this. #3 may be a little surprising in how it doesn't affect value types (like static arrays). However, before switching to #1, you'd need to make sure that that one doesn't have worse corner cases. I don't see any deal breakers, but that doesn't mean they're not there ;) You also have to see if changing to #1 is worth the effort. It would be an effort not only for the implementer, but also for the users who have to update all their code.
Oct 11 2016
parent reply Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 11 October 2016 at 18:18:41 UTC, ag0aep6g wrote:
 On 10/11/2016 06:24 AM, Jon Degenhardt wrote:
 The example I gave uses ref parameters. On the surface it 
 would seem
 reasonable to that passing a static array by ref would allow 
 it to be
 modified, without having to slice it first.
Your ref parameters are only for the per-element operations. You're not passing the array as a whole by reference. And you can't, because `each` itself takes the whole range by copy. So, the by-ref increments themselves do work, but they're applied to a copy of your original static array.
I see. Thanks for the explanation. I wasn't thinking it through properly. Also, I guess I had assumed that the intent was that each! be able to modify the elements, and therefore the whole array it would be pass by reference, but didn't consider it properly. I'm not going to make any suggestions about whether the behavior should be changed. At some point when I get a bit of time I'll try to submit a documentation change to make the current behavior clearer. --Jon
Oct 11 2016
parent Jon Degenhardt <jond noreply.com> writes:
On Tuesday, 11 October 2016 at 19:46:31 UTC, Jon Degenhardt wrote:
 On Tuesday, 11 October 2016 at 18:18:41 UTC, ag0aep6g wrote:
 On 10/11/2016 06:24 AM, Jon Degenhardt wrote:
 The example I gave uses ref parameters. On the surface it 
 would seem
 reasonable to that passing a static array by ref would allow 
 it to be
 modified, without having to slice it first.
Your ref parameters are only for the per-element operations. You're not passing the array as a whole by reference. And you can't, because `each` itself takes the whole range by copy. So, the by-ref increments themselves do work, but they're applied to a copy of your original static array.
I see. Thanks for the explanation. I wasn't thinking it through properly. Also, I guess I had assumed that the intent was that each! be able to modify the elements, and therefore the whole array it would be pass by reference, but didn't consider it properly.
Another perspective where the current behavior could be confusing is that it is somewhat natural to assume that 'each' is the functional equivalent of foreach, and that they can be used interchangeably. However, for static arrays they cannot be.
Oct 11 2016