www.digitalmars.com         C & C++   DMDScript  

D - DMD 0.71 release

reply "Walter" <walter digitalmars.com> writes:
This includes the new foreach construct, which is the building block for
container classes and iterations. I'll work on the bug reports next!

http://www.digitalmars.com/d/changelog.html
Sep 03 2003
next sibling parent reply And <And_member pathlink.com> writes:
In article <bj5pa5$156u$1 digitaldaemon.com>, Walter says...
This includes the new foreach construct, which is the building block for
container classes and iterations. I'll work on the bug reports next!

http://www.digitalmars.com/d/changelog.html

I believe you mean write and append. Ant
Sep 03 2003
parent "Walter" <walter digitalmars.com> writes:
"And" <And_member pathlink.com> wrote in message
news:bj6ds0$212l$1 digitaldaemon.com...
 I believe you mean write and append.

Yes, thanks.
Sep 03 2003
prev sibling next sibling parent reply Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:
 This includes the new foreach construct, which is the building block for
 container classes and iterations. I'll work on the bug reports next!
 
 http://www.digitalmars.com/d/changelog.html
 

while I like the inclusion of foreach, would it not be a little nicer if it was foreach( <type> <name> (in|inout) <expr>) <statement> i.e. void func( char[] str ) { foreach( char c in str ) { // modifying c does not effect str[i] } foreach( char c inout str ) { modifying c modifies str[i]; } } the apply member: why is it not void apply( bit delegate( in T ) ) { .. } // foreach in void apply( bit delegate( inout T ) ) { .. } // foreach inout. the apply delegate returns true if the iteration should continue, I don't see the need for apply to return 0 or for the delegate to return an int implemented as class canForeach { char[] str; void apply( bit delegate( in char ) applyfunc ) { for( int i = 0; i < str.length; i++ ) { if ( !applyfunc( str[i] ) ) { return; } } } }
Sep 04 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bj7gn4$gtb$1 digitaldaemon.com...
 while I like the inclusion of foreach,
 would it not be a little nicer if it was

 foreach( <type> <name> (in|inout) <expr>) <statement>
 i.e.

 void func( char[] str ) {
 foreach( char c in str ) {
 // modifying c does not effect str[i]
 }
 foreach( char c inout str ) {
 modifying c modifies str[i];
 }
 }

I thought about making it compatible with function parameter syntax that uses inout.
 the apply member:
 why is it not
 void apply( bit delegate( in T ) ) { .. } // foreach in
 void apply( bit delegate( inout T ) ) { .. } // foreach inout.

 the apply delegate returns true if the iteration should continue, I
 don't see the need for apply to return 0 or for the delegate to return
 an int

 implemented as
 class canForeach {
 char[] str;
 void apply( bit delegate( in char ) applyfunc ) {
 for( int i = 0; i < str.length; i++ ) {
 if ( !applyfunc( str[i] ) ) { return; }
 }
 }
 }

The need for it to return an int rather than bit is because of the compiler 'magic' involved with making this work. The foreach body is transformed into a nested function (!). The return value is needed to handle the various ways of exiting the foreach body, such as goto, break label, return, etc. The foreach itself is replaced with a switch statement calling the apply function. It sounds a bit hairy, but that's all hidden from the programmer. The neato thing is there is no need to write state-preserving iterators, a rather tricky thing to get right for containers that are recursive or otherwise need state-preserving iterators.
Sep 04 2003
next sibling parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
"Walter" <walter digitalmars.com> wrote in message
news:bj84jb$1e41$1 digitaldaemon.com...

 The need for it to return an int rather than bit is because of the

 'magic' involved with making this work. The foreach body is transformed

 a nested function (!). The return value is needed to handle the various

 of exiting the foreach body, such as goto, break label, return, etc. The
 foreach itself is replaced with a switch statement calling the apply
 function. It sounds a bit hairy, but that's all hidden from the

 The neato thing is there is no need to write state-preserving iterators, a
 rather tricky thing to get right for containers that are recursive or
 otherwise need state-preserving iterators.

This is pretty cool, Walter. It sounds to me as if you're halfway to making general inplace anonymous functions! Will it inline these calls to the nested function if everything is final? And you can, say, break out of a foreach? This (behind the scenes) returns a certain value from the nested function? One nice thing about having actual iterators is that you can form a range over which to iterate. It doesn't sound like this setup allows that... it just allows sequentially iterating from the beginning until the Apply() function decides to return a "stop" value or it hits the end, correct? Granted, 90% of the cases would want to iterate the entire container, but in the other 10% it may prove a bit constrictive. Combined with not being able to overload operator [] means it's very hard to make a good usable array class that looks and feels like a builtin. Everybody will do it differently. Usually with array-like things you can use operator []. So by providing operator [] and a size() function you can emulate an array, and by providing the iterator functions you can mimic a list or generic forward-iterating container. foreach would try to use the iterators if available and if not, default to indexing 0 .. size-1 and using operator []? How is foreach going to work with user-defined containers? Sean
Sep 05 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bj9eod$aps$1 digitaldaemon.com...
 Will it inline these calls to the nested function if everything is final?

No, since it is done as a pointer to a function.
 And you can, say, break out of a foreach?  This (behind the scenes)

 a certain value from the nested function?

Yes.
 One nice thing about having actual iterators is that you can form a range
 over which to iterate.  It doesn't sound like this setup allows that... it
 just allows sequentially iterating from the beginning until the Apply()
 function decides to return a "stop" value or it hits the end, correct?
 Granted, 90% of the cases would want to iterate the entire container, but

 the other 10% it may prove a bit constrictive.

The secret to modifying this for specific cases is to override/modify the apply(). Here's a little template technique to run an array in reverse: template ReverseIterator(T : T[]) { class RI { T[] a; this(T[] a) { this.a = a; } int apply(int delegate(inout T v) dg) { int result; for (int i = a.length; i; i--) { result = dg(a[i - 1]); if (result) break; } return result; } } } void main() { int[3] a; a[0] = 1; a[1] = 2; a[2] = 3; foreach (int i; new instance ReverseIterator(int[]).RI(a)) { printf("i = %d\n", i); } }
 Combined with not being able to overload operator [] means it's very hard

 make a good usable array class that looks and feels like a builtin.

I need to provide [] overloads, no question about it.
 Everybody will do it differently.  Usually with array-like things you can
 use operator [].  So by providing operator [] and a size() function you

 emulate an array, and by providing the iterator functions you can mimic a
 list or generic forward-iterating container.  foreach would try to use the
 iterators if available and if not, default to indexing 0 .. size-1 and

 operator []?
 How is foreach going to work with user-defined containers?

All the container needs is an apply() function - no need for iterators. I have never particularly liked the iterator paradigm, as it just involves lots of cruft to write. Things get really ugly when trying to write an iterator for a recursive structure, articles on iterators tend to ignore these cases and show instead the trivial ones. The C# people seem to be trying to get around this by using coroutines, but the complexities of that are just too awful. There hasn't been a foreach in D before because I just could never figure out a solution that I liked, and then finally had a brainwave about using a magic nested function. The more I use nested functions, it turns out, the more useful they become <g>.
Sep 05 2003
parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
"Walter" <walter digitalmars.com> wrote in message
news:bj9ha5$eff$1 digitaldaemon.com...
 "Sean L. Palmer" <palmer.sean verizon.net> wrote in message
 news:bj9eod$aps$1 digitaldaemon.com...
 Will it inline these calls to the nested function if everything is


 No, since it is done as a pointer to a function.

Can you make it a referece to a function instead so future compilers can inline it if they want? This is the great thing about STL's for_each.
 One nice thing about having actual iterators is that you can form a


 over which to iterate.  It doesn't sound like this setup allows that...


 just allows sequentially iterating from the beginning until the Apply()
 function decides to return a "stop" value or it hits the end, correct?
 Granted, 90% of the cases would want to iterate the entire container,


 the other 10% it may prove a bit constrictive.

The secret to modifying this for specific cases is to override/modify the apply(). Here's a little template technique to run an array in reverse:

So to iterate over a range you can just make a slice and iterate that, I guess. ;)
 All the container needs is an apply() function - no need for iterators. I
 have never particularly liked the iterator paradigm, as it just involves
 lots of cruft to write. Things get really ugly when trying to write an
 iterator for a recursive structure, articles on iterators tend to ignore
 these cases and show instead the trivial ones. The C# people seem to be
 trying to get around this by using coroutines, but the complexities of

 are just too awful. There hasn't been a foreach in D before because I just
 could never figure out a solution that I liked, and then finally had a
 brainwave about using a magic nested function.

I'm glad you figured it out! It seems like a pretty good solution.
 The more I use nested functions, it turns out, the more useful they become
 <g>.

Awesome! One thing you might think about is the name apply(). Most functional languages would call this feature "map". With D having associative arrays built in, there's not much need for STL-style map container, so this may be an ok name. Sean
Sep 06 2003
parent "Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> writes:
"Sean L. Palmer" <palmer.sean verizon.net> escreveu na mensagem
news:bjcuo2$2c19$1 digitaldaemon.com...

[snip]

 One thing you might think about is the name apply().  Most functional
 languages would call this feature "map".  With D having associative arrays
 built in, there's not much need for STL-style map container, so this may

 an ok name.

 Sean

This isn't correct. In functional languages map is a function that transform one kind of collection in another. As a foreach can do several different things (e.g. sum the elements) it's just a matter of finding the correct name. For example in DTL I defined both a foreach function (a command to be executed for each item) and an apply function (a transformation to be applied for each array entry): public void foreach(T[] array, void function(T) operation); public void apply(inout T[] array, T function(T) operation); IMO we should call this operation "forEach", but I'm ok with apply (I'll change DTL "apply" to "transform"). Best regards, Daniel Yokomiso. "Boss: "D?" What happened to C? Or that other one, C++, that you're always harping about? Me: Well, D is a further refinement of C++, and ... Boss: Weren't you just telling me that "J" language was supposed to be the next big thing? Isn't J further along than D? Me: Yes and no ... Boss: And if only want to make small changes, is a "+" higher than a "#"? [suddenly, a shot rang out]" - sunwukong at /. --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.514 / Virus Database: 312 - Release Date: 29/8/2003
Sep 06 2003
prev sibling parent reply Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:
 
 The need for it to return an int rather than bit is because of the compiler
 'magic' involved with making this work. The foreach body is transformed into
 a nested function (!). The return value is needed to handle the various ways
 of exiting the foreach body, such as goto, break label, return, etc. The
 foreach itself is replaced with a switch statement calling the apply
 function. It sounds a bit hairy, but that's all hidden from the programmer.
 The neato thing is there is no need to write state-preserving iterators, a
 rather tricky thing to get right for containers that are recursive or
 otherwise need state-preserving iterators.
 

void func( SomeObj ar ) { outer: { ...... foreach( int a; ar ) { printf( "%d ", a ); if ( a == '\n' ) { break outer; } } } // outer loop } ... becomes ... void func( SomeObj ar ) { outer: { ..... { // foreach int __iter_000_state = 0; bit irf( int a ) { printf( "%d", a ); if ( a == '\n' ) { __iter_000_state = BREAK_000; // update parent stack frame return false; } return true; } // end iterator func ar.apply( irf ); switch( __iter_000_state ) { default: break; BREAK_000: break outer; } } } // outer } simple break is just return false (stop iterating) simple continue is just return true (stop current iteration, but continue) I agree this is a little slower, BUT removes the "magic" from the programmers view, I feel the int magic although a little faster does expose the inner workings a little too much and is a bug waiting to happen.
Sep 05 2003
next sibling parent reply "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bjam8u$21if$1 digitaldaemon.com...
 I agree this is a little slower, BUT removes the "magic" from the
 programmers view, I feel the int magic although a little faster does
 expose the inner workings a little too much and is a bug waiting to

That would work just as well, but I don't think there's any real difference.
Sep 05 2003
parent reply Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:
 "Mike Wynn" <mike l8night.co.uk> wrote in message
 news:bjam8u$21if$1 digitaldaemon.com...
 
I agree this is a little slower, BUT removes the "magic" from the
programmers view, I feel the int magic although a little faster does
expose the inner workings a little too much and is a bug waiting to

happen. That would work just as well, but I don't think there's any real difference.

the inner workings of the block->nested_function/delgate is hidden from the programmer. the apply method can not cause unexpected errors by returning a silly value. to experienced D/C developers maybe there is no difference, but to those new to D, I think the need to keep the return value from the delgate and pass it back may seem odd, (and who reads the manual?) to me the difference is that its more robust, no "you need to know this to use X", and the types used reflect their use explicitly.
Sep 05 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bjb0en$2gk9$1 digitaldaemon.com...
 Walter wrote:
 "Mike Wynn" <mike l8night.co.uk> wrote in message
 news:bjam8u$21if$1 digitaldaemon.com...
I agree this is a little slower, BUT removes the "magic" from the
programmers view, I feel the int magic although a little faster does
expose the inner workings a little too much and is a bug waiting to

That would work just as well, but I don't think there's any real


 the inner workings of the block->nested_function/delgate is hidden from
 the programmer.
 the apply method can not cause unexpected errors by returning a silly

 to experienced D/C developers maybe there is no difference, but to those
 new to D, I think the need to keep the return value from the delgate and
 pass it back may seem odd, (and who reads the manual?)
 to me the difference is that its more robust, no "you need to know this
 to use X", and the types used reflect their use explicitly.

Ok, I see your point, although am not convinced. If experience shows you to be correct, I'll change it. I'm also a bit reluctant to take the speed sacrifice, even though it is only a 3 or 4 instructions. This needs to be speed competitive with C++.
Sep 06 2003
parent Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:
 "Mike Wynn" <mike l8night.co.uk> wrote in message
 news:bjb0en$2gk9$1 digitaldaemon.com...
 
the inner workings of the block->nested_function/delgate is hidden from
the programmer.
the apply method can not cause unexpected errors by returning a silly

value.
to experienced D/C developers maybe there is no difference, but to those
new to D, I think the need to keep the return value from the delgate and
pass it back may seem odd, (and who reads the manual?)
to me the difference is that its more robust, no "you need to know this
to use X", and the types used reflect their use explicitly.

Ok, I see your point, although am not convinced. If experience shows you to be correct, I'll change it. I'm also a bit reluctant to take the speed sacrifice, even though it is only a 3 or 4 instructions. This needs to be speed competitive with C++.

I did think about that, I'm not convinced that it will be slower, EAX will be free as a temp until the return value is assign, instead of it having to retain the value or be spilled to the stack. on other platforms the same may be true too, ARM for instance uses R0..R3 as the first 4 params and also as the return/temp regs, (I think MIPS is the same) which would force you to save R0 on each loop. I know you only need to save it if its not 0, in which case you should be exiting the loop so want it as a return value, but the iterator might call another function (hence it need storing across the func call on almost all archs, sparc with reg windows is about the only one i can thnk of that might not be effected. this has to be done inside the iterating loop, where as saving the "enum" to the parent frame is only done once at the time that it needed (just before exiting prematurly) it seems to me it might actually be faster.
Sep 06 2003
prev sibling parent "Sean L. Palmer" <palmer.sean verizon.net> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bjam8u$21if$1 digitaldaemon.com...
 Walter wrote:
 The need for it to return an int rather than bit is because of the


 'magic' involved with making this work. The foreach body is transformed


 a nested function (!). The return value is needed to handle the various


 of exiting the foreach body, such as goto, break label, return, etc. The
 foreach itself is replaced with a switch statement calling the apply
 function. It sounds a bit hairy, but that's all hidden from the


 The neato thing is there is no need to write state-preserving iterators,


 rather tricky thing to get right for containers that are recursive or
 otherwise need state-preserving iterators.

simple break is just return false (stop iterating) simple continue is just return true (stop current iteration, but continue) I agree this is a little slower, BUT removes the "magic" from the programmers view, I feel the int magic although a little faster does expose the inner workings a little too much and is a bug waiting to

I agree... at very least consider making it an enum instead of a plain int. Sean
Sep 06 2003
prev sibling parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
Yup, that's pretty much what I had in mind.

Is there a way to obtain the 'type' of the elements to be iterated over
automatically?  It should be possible to infer the type.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:bj5pa5$156u$1 digitaldaemon.com...
 This includes the new foreach construct, which is the building block for
 container classes and iterations. I'll work on the bug reports next!

 http://www.digitalmars.com/d/changelog.html

Sep 04 2003
parent "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bj7pc7$tie$1 digitaldaemon.com...
 Yup, that's pretty much what I had in mind.

 Is there a way to obtain the 'type' of the elements to be iterated over
 automatically?  It should be possible to infer the type.

No. It sounds like a good idea, though.
Sep 04 2003