www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Alias this with array can only be used once

reply "Blake Anderton" <rbanderton gmail.com> writes:
Why doesn't this work? I'm assuming I'm not fully understanding 
how "alias this" interacts with ranges (i.e. the range isn't 
being reset after count is finished with it), but I'm not sure 
how to fix this either:

import std.algorithm;

class ArrayContainer
{
     int[] values;
     this(int[] v) { values = v; }
     alias values this;
}

void main(string[] args)
{
     auto c = new ArrayContainer([1, 2, 3]);
     assert(count(c) == 3); //succeeds
     assert(c.length == 3); //FAILS - is actually zero
}
Feb 22 2012
next sibling parent =?utf-8?Q?Simen_Kj=C3=A6r=C3=A5s?= <simen.kjaras gmail.com> writes:
On Wed, 22 Feb 2012 23:16:41 +0100, Blake Anderton <rbanderton gmail.com>  
wrote:

 Why doesn't this work? I'm assuming I'm not fully understanding how  
 "alias this" interacts with ranges (i.e. the range isn't being reset  
 after count is finished with it), but I'm not sure how to fix this  
 either:

 import std.algorithm;

 class ArrayContainer
 {
      int[] values;
      this(int[] v) { values = v; }
      alias values this;
 }

 void main(string[] args)
 {
      auto c = new ArrayContainer([1, 2, 3]);
      assert(count(c) == 3); //succeeds
      assert(c.length == 3); //FAILS - is actually zero
 }

That's interesting. The thing that happens is the function popFront is callable on ArrayContainer, and popFront mutates its parameter. The solution is to use count(c[]), but I'm not so sure I agree with the way this works currently.
Feb 22 2012
prev sibling next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/22/2012 02:16 PM, Blake Anderton wrote:
 Why doesn't this work? I'm assuming I'm not fully understanding how
 "alias this" interacts with ranges (i.e. the range isn't being reset
 after count is finished with it), but I'm not sure how to fix this either:

 import std.algorithm;

 class ArrayContainer
 {
 int[] values;
 this(int[] v) { values = v; }
 alias values this;
 }

 void main(string[] args)
 {
 auto c = new ArrayContainer([1, 2, 3]);
 assert(count(c) == 3); //succeeds
 assert(c.length == 3); //FAILS - is actually zero
 }

Consumption of the range is natural for an InputRange. What I see in Phobos is that containers like ArrayContainer themselves don't behave like ranges, rather they hand out Range objects to be consumed. As a solution, it can have a member function named something like elements() that returns a separate array to be used as an InputRange. Ali
Feb 22 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, February 22, 2012 23:16:41 Blake Anderton wrote:
 Why doesn't this work? I'm assuming I'm not fully understanding
 how "alias this" interacts with ranges (i.e. the range isn't
 being reset after count is finished with it), but I'm not sure
 how to fix this either:
 
 import std.algorithm;
 
 class ArrayContainer
 {
 int[] values;
 this(int[] v) { values = v; }
 alias values this;
 }
 
 void main(string[] args)
 {
 auto c = new ArrayContainer([1, 2, 3]);
 assert(count(c) == 3); //succeeds
 assert(c.length == 3); //FAILS - is actually zero
 }

I believe that the problem stems from the fact that count is being instantiated with ArrayContainer rather than int[]. That means that when it gets processed, it ends up operating on the array directly rather than on a slice. If ArrayContainer were a struct rather than a class, then this would work. But by doing what you're doing, you've managed to create a type that effectively conflates a range and a container. So, when you pass it to range- based functions, it _will_ be consumed. You really shouldn't give direct access to the array like that. It's only going to cause you trouble. Containers should _not_ be ranges, and using alias this makes your ArrayContainer a range. Instead, you should provide an opSlice to get the array. Then you'd do assert(count(c[]) == 3); assert(c[].length == 3); The explicit [] would be required to call opSlice, since ArrayContainer would then not be passable to count (which it really shouldn't be anyway). However, if you're really married to the idea of using alias this, then the above code will work with your current implementation, since it forces the array to be sliced, and count therefore gets instantiated with int[] rather than ArrayContainer. But again, I _really_ advise against having a container which is a range. It's a _bad_ idea which is only going to cause you trouble. - Jonathan M Davis
Feb 22 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/22/2012 03:16 PM, Blake Anderton wrote:

 With an array the above tests work (I assume because each
 operation is implicitly a new slice?).

It works because the slice itself is copied to count() and count() consumes that copy internally. Now I see the issue here: Why does 'alias this' disable the parameter copying behavior? Why is the member slice not being copied? This must be clarified. Is this a bug? A hole in the spec?
 I found I can fix the above example by aliasing a function that returns
 a slice of the member array instead of aliasing the field itself,

That function is called save() on ForwardRange ranges and does exactly that.
 but
 that may still be too close a relationship of the container/range. My
 instinct (which could be completely wrong) is that it's fine since the
 container is taking responsibility for creating valid ranges, it is just
 doing so implicitly.

 User code having to explicitly take ranges (either through opSlice or a
 property/method) isn't a terrible thing, but it would be convenient to
 not have to take that extra step.

You have to decide whether this is a container or a range. If it's a range, the users should be able to freely consume it from the top and save copies of it by save() when they need to. I recommend defining the range functions directly on this type in that case. Ali
Feb 22 2012
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/22/2012 03:59 PM, Jonathan M Davis wrote:
 On Wednesday, February 22, 2012 15:40:41 Ali Çehreli wrote:
 On 02/22/2012 03:16 PM, Blake Anderton wrote:
 With an array the above tests work (I assume because each
 operation is implicitly a new slice?).

It works because the slice itself is copied to count() and count() consumes that copy internally. Now I see the issue here: Why does 'alias this' disable the parameter copying behavior? Why is the member slice not being copied? This must be clarified. Is this a bug? A hole in the spec?

I believe that it's quite clear. Think about it. How is the template instantiated? With the type that you give it. And as long as the type

 the template constraint, the function will be instantiated with the exact
 type that you gave it.

This proves that 'alias this' is something to watch out for. It makes our type to pass certain constraints but at the end it is still our object that gets passed, not the 'alias this'ed member. The problem here has been that the original type was a class (i.e. a reference type). The class variable gets copied to count() and count() consumes the one instance of the class. If the original code used 'struct' instead, this issue would not have come up, because then the member of the copy would be consumed. Sneaky! :) This code passes the asserts: import std.algorithm; struct ArrayContainer { int[] values; this(int[] v) { values = v; } alias values this; } void main(string[] args) { auto c = ArrayContainer([1, 2, 3]); assert(count(c) == 3); //succeeds assert(c.length == 3); //succeeds too //(would fail if ArrayContainer were a class) } Ali
Feb 22 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, February 22, 2012 23:28:47 Simen Kjærås wrote:
 On Wed, 22 Feb 2012 23:16:41 +0100, Blake Anderton <rbanderton gmail.com>
 
 wrote:
 Why doesn't this work? I'm assuming I'm not fully understanding how
 "alias this" interacts with ranges (i.e. the range isn't being reset
 after count is finished with it), but I'm not sure how to fix this
 either:
 
 import std.algorithm;
 
 class ArrayContainer
 {
 
 int[] values;
 this(int[] v) { values = v; }
 alias values this;
 
 }
 
 void main(string[] args)
 {
 
 auto c = new ArrayContainer([1, 2, 3]);
 assert(count(c) == 3); //succeeds
 assert(c.length == 3); //FAILS - is actually zero
 
 }

That's interesting. The thing that happens is the function popFront is callable on ArrayContainer, and popFront mutates its parameter. The solution is to use count(c[]), but I'm not so sure I agree with the way this works currently.

It's the consequence of having a container which is also a range - which is what this code is doing. It's _not_ something that's a good idea. Regardless, it's not like this code could have any other behavior than it does. The template constraint on count verifies that the argument is an input range, which it is, since you can call front, popFront, and empty on it thanks to the alias this. So, the template gets instantiated with the type that you pass it - ArrayContainer. And then ArrayContainer gets consumed by count as any range would gets consumed by count. It's just that unlike most ranges, it's a class, so it doesn't get implicitly sliced or saved when you pass it in. So, instead of the slice being consumed, it itself is consumed. This is all solved by just not declaring a container which is a range. It's just asking for trouble. You don't want your container to be consumed when you pass it to range-based functions. Rather, you should be slicing it so that the _range_ gets consumed without screwing over your container. - Jonathan M Davis
Feb 22 2012
prev sibling next sibling parent "Blake Anderton" <rbanderton gmail.com> writes:
Thanks all for the quick replies.

You're right- I am mixing the container with the range to ill 
effect. I'm just doing this for practice so I'm not tied to any 
one solution, but I was trying to make the container work as much 
like a built-in type as possible. With an array the above tests 
work (I assume because each operation is implicitly a new slice?).

I found I can fix the above example by aliasing a function that 
returns a slice of the member array instead of aliasing the field 
itself, but that may still be too close a relationship of the 
container/range. My instinct (which could be completely wrong) is 
that it's fine since the container is taking responsibility for 
creating valid ranges, it is just doing so implicitly.

User code having to explicitly take ranges (either through 
opSlice or a property/method) isn't a terrible thing, but it 
would be convenient to not have to take that extra step.
Feb 22 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, February 23, 2012 00:16:00 Blake Anderton wrote:
 Thanks all for the quick replies.
 
 You're right- I am mixing the container with the range to ill
 effect. I'm just doing this for practice so I'm not tied to any
 one solution, but I was trying to make the container work as much
 like a built-in type as possible. With an array the above tests
 work (I assume because each operation is implicitly a new slice?).
 
 I found I can fix the above example by aliasing a function that
 returns a slice of the member array instead of aliasing the field
 itself, but that may still be too close a relationship of the
 container/range. My instinct (which could be completely wrong) is
 that it's fine since the container is taking responsibility for
 creating valid ranges, it is just doing so implicitly.
 
 User code having to explicitly take ranges (either through
 opSlice or a property/method) isn't a terrible thing, but it
 would be convenient to not have to take that extra step.

That's just the way that it works with containers. Dynamic arrays are weird in that they're not really containers, but people tend to think of them as containers. But they don't own their own memory, and they're ranges. Static arrays _are_ containers and act like containers should in that they require you to slice them to pass to range-based functions, because that's what you have to do to get a range over them. In any case, a container is not a range and shouldn't be treated as one. Your solution with an aliased function sort of solves the problem and sort of not. Your container is still considered a range, but when you go to use it as a range, you end up getting a slice of the internal array, which _is_ a range. I'd advise just making a clear distinction between the container and the range and not using alias this. alias this is great when you want to treat one type like another, but that's _not_ what we want with containers and ranges. Conflating the two will cause problems, much as might be able to get away with it in some cases. The standard way to do it (as std.container's types do) is to have the container have _no_ range functions at all but to implement opSlice which returns a range over that container. When it comes to containers, you really shouldn't think of ranges as being similar to iterators. And you wouldn't make a container into an iterator would you? Conflating the two would not be pretty. Unfortunately, with ranges, the distinctious isn't as obvious, since they refer to a range of elements rather than a single element, but the problem is still essentially the same. - Jonathan M Davis
Feb 22 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, February 22, 2012 15:40:41 Ali Çehreli wrote:
 On 02/22/2012 03:16 PM, Blake Anderton wrote:
 With an array the above tests work (I assume because each
 operation is implicitly a new slice?).

It works because the slice itself is copied to count() and count() consumes that copy internally. Now I see the issue here: Why does 'alias this' disable the parameter copying behavior? Why is the member slice not being copied? This must be clarified. Is this a bug? A hole in the spec?

I believe that it's quite clear. Think about it. How is the template instantiated? With the type that you give it. And as long as the type passes the template constraint, the function will be instantiated with the exact type that you gave it. What does the template constraint do? In this case it's if(isInputRange!Range && is(typeof(binaryFun!pred(r.front, value)) == bool)) where isInputRange is template isInputRange(R) { enum bool isInputRange = is(typeof( { R r = void; // can define a range object if (r.empty) {} // can test for empty r.popFront(); // can invoke popFront() auto h = r.front; // can get the front of the range })); } Well, because of the alias, r.front will grab the front on the array (as there is no front on the container), and the second part will compile and be considered true. In the case of isInputRange, it becomes template isInputRange(R) { enum bool isInputRange = is(typeof( { ArrayContainer r = void; // can define a range object if (r.empty) {} // can test for empty r.popFront(); // can invoke popFront() auto h = r.front; // can get the front of the range })); } Naturally, the declaration of ArrayContainer succeeds, and then all of the subsequent functions just end up using alias this with the array. So, the result is true, and so isInputRange is true, and ArrayContainer passes count's template constraint. And so, count is instantiated with ArrayContainer, not int[]. I don't see anything ambiguous about that. You asked the compiler to instantiate count with ArrayContainer, not int[], and so that's what it did. You could explicitly instantiate it with int[] if you wanted to, though you might as well just slice the ArrayContainer (and thus the int[]) and get an int[] that way. - Jonathan M Davis
Feb 22 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, February 22, 2012 16:25:09 Ali Çehreli wrote:
 On 02/22/2012 03:59 PM, Jonathan M Davis wrote:
 On Wednesday, February 22, 2012 15:40:41 Ali Çehreli wrote:
 On 02/22/2012 03:16 PM, Blake Anderton wrote:
 With an array the above tests work (I assume because each
 operation is implicitly a new slice?).

It works because the slice itself is copied to count() and count() consumes that copy internally. Now I see the issue here: Why does 'alias this' disable the parameter copying behavior? Why is the member slice not being copied? This must be clarified. Is this a bug? A hole in the spec?

I believe that it's quite clear. Think about it. How is the template instantiated? With the type that you give it. And as long as the type

passes
 the template constraint, the function will be instantiated with the exact
 type that you gave it.

This proves that 'alias this' is something to watch out for. It makes our type to pass certain constraints but at the end it is still our object that gets passed, not the 'alias this'ed member. The problem here has been that the original type was a class (i.e. a reference type). The class variable gets copied to count() and count() consumes the one instance of the class. If the original code used 'struct' instead, this issue would not have come up, because then the member of the copy would be consumed. Sneaky! :)

Yes, but then you get all kinds of fun bugs due to the fact that your container is a pseudo value type. All of the instances share data as long as the array isn't caused to reallocate for some reason. So, you could end up with multiple copies affecting each other or having drastically different data, depending on what the code does. But there are a variety of problems here. Using classes for ranges makes it so that they do not work in the typically expected way. save is in ranges specifically for handling it, but it means that you have to actually _use_ it, which many algorithms should but probably don't, because they're only tested with ranges which are structs or arrays. Conflating ranges and containers is a bad idea which will cause issues. They are distinct concepts which should be kept separate. alias this makes it so that that type acts like the aliased type in a number of circumstances, but it still isn't _exactly_ that type, so it won't act exactly the same as the aliased type in all circumstances (templates being a prime example). But if you think about it, since it _is_ a different type, it _can't_ act like the aliased type in all circumstances. You just have to be well aware of what the exact effects are. But templated functions (and range- based functions in particular) are likely to cause issues, because they'll compile thanks to the alias, but they won't be instantiated with the aliased type. - Jonathan M Davis
Feb 22 2012
prev sibling parent "Blake Anderton" <rbanderton gmail.com> writes:
Good points on why an explicit range distinction is desired; I 
probably should have looked harder at how std.container worked. 
I'm also probably spoiled/unlearning from C#'s IEnumerable<T> 
syntactic sugar (foreach over enumerator, LINQ) which handles 
most of these considerations for you. You could say it encourages 
conflating the container with the iteration mechanism (range in 
D, enumerator in C#). I see the distinction now but old habits 
die hard, I guess. :)
Feb 23 2012