www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - opSlice() magic for foreach now works?

reply spir <denis.spir gmail.com> writes:
Hello,

I was trying std.container's SList and noticed foreach iteration works for it, 
while it defines neither opApply, nore a (complete) input range interface. But 
it does implement a parameter-less opSlice, which maps to "l2 = l1[];". More 
precisely, SList (as other container types) defines a 'Range' struct type which 
is the return type of opSlice().
TDPL states this method should automagically deal as standard traversal for 
foreach (when no opApply or range interface is defined on the collection to be 
traverseditself, I guess). But the magic did not work up to very recently.

Does this mean this feature is now implemented? (Note: I tried it for a custom 
collection type, and it worked fine.) If yes, I guess it deserves an announce 
on the mailing list, as many of thus were waiting for it (hurray!). And a clear 
explanation at three places:
* in the 'foreach' section of the language reference manual
* in std.range's doc
* in std.container's doc

Note that this feature fills a long-standing gap in the unification of 
collections & ranges: there have been until now 2 contradictory requirements 
upon ranges:

1. A collection should be iterable (indeed).
2. A collection should /not/ implement a range interface (be its own range), 
instead it should provide
    one or more methods returning traversal ranges. This is supposed to be more 
flexible.

The only solutions were either to provide opApply in addition to 2., or to 
force the user explicitely calling the methods, as in:
    foreach (element; myCollection.elements()) {...}
Both are unsatisfying for different reasons. Thank to the "total slice" magic, 
opSlice() now implicitely deals as /standard/ traversal for a collection. Like 
if it were a kind of 'opTraversal'. This is exactly equivalent, I guess, to 
Python's __iter__. opSlice() is thus, if I understand correctly, supposed to 
return a 'traversable' thingy, typically an input range.

I have not yet tried to provide a collection implementing opSlice() to funcs 
expecting a range. Hope this works as expected, or it will do so soon.

[See also thread about "multiple-item traversal ranges"]
Denis
-- 
_________________
vita es estrany
spir.wikidot.com
Mar 30 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 30 Mar 2011 10:06:26 -0400, spir <denis.spir gmail.com> wrote:

 Hello,

 I was trying std.container's SList and noticed foreach iteration works  
 for it, while it defines neither opApply, nore a (complete) input range  
 interface. But it does implement a parameter-less opSlice, which maps to  
 "l2 = l1[];". More precisely, SList (as other container types) defines a  
 'Range' struct type which is the return type of opSlice().
 TDPL states this method should automagically deal as standard traversal  
 for foreach (when no opApply or range interface is defined on the  
 collection to be traverseditself, I guess). But the magic did not work  
 up to very recently.

Apparently, it works for SList all the way back to at least 2.050, but I'm not sure why. It doesn't work for a basic case in 2.052: struct S { struct R { property int front() { return 0;} void popFront() {} property bool empty() const { return true;} } R opSlice() { return R();} } void main() { S s; foreach(int n; s) {} } testopslice.d(19): Error: no property 'opApply' for type 'S' testopslice.d(19): Error: opApply() function for S must return an int -Steve
Mar 30 2011
parent spir <denis.spir gmail.com> writes:
On 03/30/2011 05:34 PM, Steven Schveighoffer wrote:
 On Wed, 30 Mar 2011 11:29:02 -0400, spir <denis.spir gmail.com> wrote:

 Strange, I just tried it on a custom, trivial, collection type --precisely to
 verify that it does not work only for the standard SList-- and it worked fine
 (dmd 2.051 on ubuntu). But now I cannot reproduce this success anymore! get
 the same error as you. Will go on trying and tell you if anything changes.

 Anyway, if the feature is still unimplemented, why and how does foreach work
 on SList. ???

It may be partially/poorly implemented. I looked at the disassembly, and it is indeed calling opSlice and the range primitives inside the foreach loop. There is likely a trigger that the compiler uses to determine if it can do that, and the trigger is probably incorrectly defined. I would think (though I'm not a compiler guy) that to fix this should be minor since the code generation already works in at least one case. Walter?

Yes, i must have hit this trigger button by pure chance on my first attempt (unfortunately, since I was then sure the feature works, i did not keep the code, so i cannot compare). Denis -- _________________ vita es estrany spir.wikidot.com
Mar 30 2011
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 3/30/11 4:06 PM, spir wrote:
 Hello,

 I was trying std.container's SList and noticed foreach iteration works
 for it, while it defines neither opApply, nore a (complete) input range
 interface. But it does implement a parameter-less opSlice, which maps to
 "l2 = l1[];". More precisely, SList (as other container types) defines a
 'Range' struct type which is the return type of opSlice().
 TDPL states this method should automagically deal as standard traversal
 for foreach (when no opApply or range interface is defined on the
 collection to be traverseditself, I guess). But the magic did not work
 up to very recently.

 Does this mean this feature is now implemented? (Note: I tried it for a
 custom collection type, and it worked fine.) If yes, I guess it deserves
 an announce on the mailing list, as many of thus were waiting for it
 (hurray!). And a clear explanation at three places:
 * in the 'foreach' section of the language reference manual
 * in std.range's doc
 * in std.container's doc

 Note that this feature fills a long-standing gap in the unification of
 collections & ranges: there have been until now 2 contradictory
 requirements upon ranges:

 1. A collection should be iterable (indeed).
 2. A collection should /not/ implement a range interface (be its own
 range), instead it should provide
 one or more methods returning traversal ranges. This is supposed to be
 more flexible.

 The only solutions were either to provide opApply in addition to 2., or
 to force the user explicitely calling the methods, as in:
 foreach (element; myCollection.elements()) {...}
 Both are unsatisfying for different reasons. Thank to the "total slice"
 magic, opSlice() now implicitely deals as /standard/ traversal for a
 collection. Like if it were a kind of 'opTraversal'. This is exactly
 equivalent, I guess, to Python's __iter__. opSlice() is thus, if I
 understand correctly, supposed to return a 'traversable' thingy,
 typically an input range.

 I have not yet tried to provide a collection implementing opSlice() to
 funcs expecting a range. Hope this works as expected, or it will do so
 soon.

 [See also thread about "multiple-item traversal ranges"]
 Denis

Do we have three ways now to implement iteration, opApply, opSlice and ranges? -- /Jacob Carlborg
Mar 30 2011
parent reply David Nadlinger <see klickverbot.at> writes:
On 3/30/11 4:43 PM, Jacob Carlborg wrote:
 Do we have three ways now to implement iteration, opApply, opSlice and
 ranges?

Wouldn't opSlice only be syntatic sugar for ranges? David
Mar 30 2011
parent reply Jacob Carlborg <doob me.com> writes:
On 2011-03-30 16:47, David Nadlinger wrote:
 On 3/30/11 4:43 PM, Jacob Carlborg wrote:
 Do we have three ways now to implement iteration, opApply, opSlice and
 ranges?

Wouldn't opSlice only be syntatic sugar for ranges? David

I have no idea, that's why I'm asking. -- /Jacob Carlborg
Mar 30 2011
parent reply Mafi <mafi example.org> writes:
Am 30.03.2011 20:35, schrieb Jacob Carlborg:
 On 2011-03-30 16:47, David Nadlinger wrote:
 On 3/30/11 4:43 PM, Jacob Carlborg wrote:
 Do we have three ways now to implement iteration, opApply, opSlice and
 ranges?

Wouldn't opSlice only be syntatic sugar for ranges? David

I have no idea, that's why I'm asking.

There are anly two ways to be truly iteratable: opApplay and Ranges. Now, if the object does not implememhnt either, opSlice is checked: "Do you know somebody that can iterate over you?". If opSlice (ie a[]) gives something iterable - ie something with opApply, a Range or something which itself has a opSlice - the foreach is rewritten to use that thing. Mafi
Mar 30 2011
parent Jacob Carlborg <doob me.com> writes:
On 2011-03-30 22:35, Mafi wrote:
 Am 30.03.2011 20:35, schrieb Jacob Carlborg:
 On 2011-03-30 16:47, David Nadlinger wrote:
 On 3/30/11 4:43 PM, Jacob Carlborg wrote:
 Do we have three ways now to implement iteration, opApply, opSlice and
 ranges?

Wouldn't opSlice only be syntatic sugar for ranges? David

I have no idea, that's why I'm asking.

There are anly two ways to be truly iteratable: opApplay and Ranges. Now, if the object does not implememhnt either, opSlice is checked: "Do you know somebody that can iterate over you?". If opSlice (ie a[]) gives something iterable - ie something with opApply, a Range or something which itself has a opSlice - the foreach is rewritten to use that thing. Mafi

Ok, thanks. -- /Jacob Carlborg
Mar 31 2011
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On 03/30/2011 04:29 PM, Steven Schveighoffer wrote:
 On Wed, 30 Mar 2011 10:06:26 -0400, spir <denis.spir gmail.com> wrote:

 Hello,

 I was trying std.container's SList and noticed foreach iteration works for
 it, while it defines neither opApply, nore a (complete) input range
 interface. But it does implement a parameter-less opSlice, which maps to "l2
 = l1[];". More precisely, SList (as other container types) defines a 'Range'
 struct type which is the return type of opSlice().
 TDPL states this method should automagically deal as standard traversal for
 foreach (when no opApply or range interface is defined on the collection to
 be traverseditself, I guess). But the magic did not work up to very recently.

Apparently, it works for SList all the way back to at least 2.050, but I'm not sure why. It doesn't work for a basic case in 2.052: struct S { struct R { property int front() { return 0;} void popFront() {} property bool empty() const { return true;} } R opSlice() { return R();} } void main() { S s; foreach(int n; s) {} } testopslice.d(19): Error: no property 'opApply' for type 'S' testopslice.d(19): Error: opApply() function for S must return an int

Strange, I just tried it on a custom, trivial, collection type --precisely to verify that it does not work only for the standard SList-- and it worked fine (dmd 2.051 on ubuntu). But now I cannot reproduce this success anymore! get the same error as you. Will go on trying and tell you if anything changes. Anyway, if the feature is still unimplemented, why and how does foreach work on SList. ??? Denis -- _________________ vita es estrany spir.wikidot.com
Mar 30 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 30 Mar 2011 11:29:02 -0400, spir <denis.spir gmail.com> wrote:

 Strange, I just tried it on a custom, trivial, collection type  
 --precisely to verify that it does not work only for the standard  
 SList-- and it worked fine (dmd 2.051 on ubuntu). But now I cannot  
 reproduce this success anymore! get the same error as you. Will go on  
 trying and tell you if anything changes.

 Anyway, if the feature is still unimplemented, why and how does foreach  
 work on SList. ???

It may be partially/poorly implemented. I looked at the disassembly, and it is indeed calling opSlice and the range primitives inside the foreach loop. There is likely a trigger that the compiler uses to determine if it can do that, and the trigger is probably incorrectly defined. I would think (though I'm not a compiler guy) that to fix this should be minor since the code generation already works in at least one case. Walter? -Steve
Mar 30 2011