www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Problem with closures

reply Alain <alainpoint yahoo.fr> writes:
Hello,

I am experimenting with closures and i have problem with the following code:

import std.stdio;
alias int delegate(int x) AddFunc;

AddFunc addN(int n) {
    int add(int i) {
        return i + n;
    }
    return &add; // the add function captured will remember the value of n
}

void apply(int[] array, AddFunc func) {
    foreach (inout int i; array) {
        i = func(i);
    }
}

int main() {
    int[] numbers = [1,2,3];
    writefln("numbers before=",numbers);
    apply(numbers, addN(40)); // add 40 to each number
    writefln("numbers after=",numbers);
    return 0;
}

The result should be [41,42,43] but i get [4202694,4202695,4202696]

Am i missing something?

Alain
Jan 11 2007
next sibling parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Alain wrote:
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the value of n
 }
[snip]
 Am i missing something?
You shouldn't return delegates to nested functions, just like you shouldn't return pointers to local variables. Delegates to nested functions contain a pointer to the stack frame of the enclosing function. If that function has returned, the stack frame may be corrupted (especially if you have called another function, like apply(), since then).
Jan 11 2007
parent reply Alain <alainpoint yahoo.fr> writes:
Frits van Bommel Wrote:

 Alain wrote:
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the value of n
 }
[snip]
 Am i missing something?
You shouldn't return delegates to nested functions, just like you shouldn't return pointers to local variables. Delegates to nested functions contain a pointer to the stack frame of the enclosing function. If that function has returned, the stack frame may be corrupted (especially if you have called another function, like apply(), since then).
How come this compiles? Is it a bug ? Alain
Jan 11 2007
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Alain wrote:
 Frits van Bommel Wrote:
 
 Alain wrote:
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the value of n
 }
[snip]
 Am i missing something?
You shouldn't return delegates to nested functions, just like you shouldn't return pointers to local variables. Delegates to nested functions contain a pointer to the stack frame of the enclosing function. If that function has returned, the stack frame may be corrupted (especially if you have called another function, like apply(), since then).
How come this compiles? Is it a bug ? Alain
This is valid code. It is up to the programmer to make sure the delegate doesn't use vars that are out of scope. There was some talk about this, to copy the 'evironment' for later use, it might be solved in the future. std.bind might be of use here.
Jan 11 2007
parent reply Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Lutger wrote:
 Alain wrote:
 Frits van Bommel Wrote:

 Alain wrote:
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the 
 value of n
 }
[snip]
 Am i missing something?
You shouldn't return delegates to nested functions, just like you shouldn't return pointers to local variables. Delegates to nested functions contain a pointer to the stack frame of the enclosing function. If that function has returned, the stack frame may be corrupted (especially if you have called another function, like apply(), since then).
How come this compiles? Is it a bug ? Alain
This is valid code. It is up to the programmer to make sure the delegate doesn't use vars that are out of scope. There was some talk about this, to copy the 'evironment' for later use, it might be solved in the future. std.bind might be of use here.
As could a template. <code> import std .stdio ; alias int delegate(int n) AddFunc; template TAddN (int N) { const TAddN = (int i) { return i + N; } ; } void apply (int[] array, AddFunc func) { foreach (inout i; array) { i = func(i); } } void main () { int[] numbers = [1,2,3] ; writefln("numbers before = ", numbers); apply(numbers, TAddN!(40)); writefln("numbers after = ", numbers); } </code> -- Chris Nicholson-Sauls
Jan 11 2007
parent reply Bradley Smith <digitalmars-com baysmith.com> writes:
Using DMD 1.0, I get errors on the template.

closureWithTemplate.d(19): delegate 
closureWithTemplate.TAddN!(40).__dgliteral2 is a nested function and 
cannot be accessed from main
closureWithTemplate.d(6): Error: non-constant expression __dgliteral2


   Bradley

Chris Nicholson-Sauls wrote:
 Lutger wrote:
 Alain wrote:
 Frits van Bommel Wrote:

 Alain wrote:
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the 
 value of n
 }
[snip]
 Am i missing something?
You shouldn't return delegates to nested functions, just like you shouldn't return pointers to local variables. Delegates to nested functions contain a pointer to the stack frame of the enclosing function. If that function has returned, the stack frame may be corrupted (especially if you have called another function, like apply(), since then).
How come this compiles? Is it a bug ? Alain
This is valid code. It is up to the programmer to make sure the delegate doesn't use vars that are out of scope. There was some talk about this, to copy the 'evironment' for later use, it might be solved in the future. std.bind might be of use here.
As could a template. <code> import std .stdio ; alias int delegate(int n) AddFunc; template TAddN (int N) { const TAddN = (int i) { return i + N; } ; } void apply (int[] array, AddFunc func) { foreach (inout i; array) { i = func(i); } } void main () { int[] numbers = [1,2,3] ; writefln("numbers before = ", numbers); apply(numbers, TAddN!(40)); writefln("numbers after = ", numbers); } </code> -- Chris Nicholson-Sauls
Jan 11 2007
parent reply Bradley Smith <digitalmars-com baysmith.com> writes:
Nevermind. I figured it out. The following works.

import std.stdio;

alias int delegate(int n) AddFunc;

template addN(int N) {
   AddFunc addN() {
     return delegate int (int i) { return i + N; };
   }
}

void apply (int[] array, AddFunc func) {
   foreach (inout i; array) {
     i = func(i);
   }
}

void main () {
     int[] numbers = [1,2,3] ;
     writefln("numbers before = ", numbers);
     apply(numbers, addN!(40));
     writefln("numbers after = ", numbers);
}

Bradley Smith wrote:
 Using DMD 1.0, I get errors on the template.
 
 closureWithTemplate.d(19): delegate 
 closureWithTemplate.TAddN!(40).__dgliteral2 is a nested function and 
 cannot be accessed from main
 closureWithTemplate.d(6): Error: non-constant expression __dgliteral2
 
 
   Bradley
 
 Chris Nicholson-Sauls wrote:
 Lutger wrote:
 Alain wrote:
 Frits van Bommel Wrote:

 Alain wrote:
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the 
 value of n
 }
[snip]
 Am i missing something?
You shouldn't return delegates to nested functions, just like you shouldn't return pointers to local variables. Delegates to nested functions contain a pointer to the stack frame of the enclosing function. If that function has returned, the stack frame may be corrupted (especially if you have called another function, like apply(), since then).
How come this compiles? Is it a bug ? Alain
This is valid code. It is up to the programmer to make sure the delegate doesn't use vars that are out of scope. There was some talk about this, to copy the 'evironment' for later use, it might be solved in the future. std.bind might be of use here.
As could a template. <code> import std .stdio ; alias int delegate(int n) AddFunc; template TAddN (int N) { const TAddN = (int i) { return i + N; } ; } void apply (int[] array, AddFunc func) { foreach (inout i; array) { i = func(i); } } void main () { int[] numbers = [1,2,3] ; writefln("numbers before = ", numbers); apply(numbers, TAddN!(40)); writefln("numbers after = ", numbers); } </code> -- Chris Nicholson-Sauls
Jan 11 2007
next sibling parent Chris Nicholson-Sauls <ibisbasenji gmail.com> writes:
Bradley Smith wrote:
 Nevermind. I figured it out. The following works.
 
 import std.stdio;
 
 alias int delegate(int n) AddFunc;
 
 template addN(int N) {
   AddFunc addN() {
     return delegate int (int i) { return i + N; };
   }
 }
 
 void apply (int[] array, AddFunc func) {
   foreach (inout i; array) {
     i = func(i);
   }
 }
 
 void main () {
     int[] numbers = [1,2,3] ;
     writefln("numbers before = ", numbers);
     apply(numbers, addN!(40));
     writefln("numbers after = ", numbers);
 }
 
 Bradley Smith wrote:
 Using DMD 1.0, I get errors on the template.

 closureWithTemplate.d(19): delegate 
 closureWithTemplate.TAddN!(40).__dgliteral2 is a nested function and 
 cannot be accessed from main
 closureWithTemplate.d(6): Error: non-constant expression __dgliteral2


   Bradley

 Chris Nicholson-Sauls wrote:
 import std .stdio ;

 alias int delegate(int n) AddFunc;

 template TAddN (int N) {
   const TAddN = (int i) { return i + N; } ;
 }

 void apply (int[] array, AddFunc func) {
   foreach (inout i; array) {
     i = func(i);
   }
 }

 void main () {
     int[] numbers = [1,2,3] ;

     writefln("numbers before = ", numbers);
     apply(numbers, TAddN!(40));
     writefln("numbers after = ", numbers);
 }
Now that, actually, I consider possibly a bug. But a least the workaround was simple enough (and small enough to be inlined). -- Chris Nicholson-Sauls
Jan 12 2007
prev sibling parent Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
[edited quotes to fix top-posting]

Bradley Smith wrote:
 Bradley Smith wrote:
 Chris Nicholson-Sauls wrote:
 import std .stdio ;

 alias int delegate(int n) AddFunc;

 template TAddN (int N) {
   const TAddN = (int i) { return i + N; } ;
 }

 void apply (int[] array, AddFunc func) {
   foreach (inout i; array) {
     i = func(i);
   }
 }

 void main () {
     int[] numbers = [1,2,3] ;

     writefln("numbers before = ", numbers);
     apply(numbers, TAddN!(40));
     writefln("numbers after = ", numbers);
 }
Using DMD 1.0, I get errors on the template. closureWithTemplate.d(19): delegate closureWithTemplate.TAddN!(40).__dgliteral2 is a nested function and cannot be accessed from main closureWithTemplate.d(6): Error: non-constant expression __dgliteral2
Nevermind. I figured it out. The following works. import std.stdio; alias int delegate(int n) AddFunc; template addN(int N) { AddFunc addN() { return delegate int (int i) { return i + N; }; } } void apply (int[] array, AddFunc func) { foreach (inout i; array) { i = func(i); } } void main () { int[] numbers = [1,2,3] ; writefln("numbers before = ", numbers); apply(numbers, addN!(40)); writefln("numbers after = ", numbers); }
This also works: ----- // Delegates replaced by functions and TAddN changed to template // template function instead of template function pointer variable // (so no delegate or function literals are needed) import std.stdio; alias int function(int n) AddFunc; int TAddN (int N) (int i) { return i + N; } void apply (int[] array, AddFunc func) { foreach (inout i; array) { i = func(i); } } void main () { int[] numbers = [1,2,3] ; writefln("numbers before = ", numbers); apply(numbers, &TAddN!(40)); writefln("numbers after = ", numbers); } ----- But I agree that the fact the below doesn't work seems to be a bug: ----- // Delegates replaced by functions import std.stdio; alias int function(int n) AddFunc; template TAddN (int N) { const TAddN = function (int i) { return i + N; } ; } void apply (int[] array, AddFunc func) { foreach (inout i; array) { i = func(i); } } void main () { int[] numbers = [1,2,3] ; writefln("numbers before = ", numbers); apply(numbers, TAddN!(40)); writefln("numbers after = ", numbers); } ----- test.d(6): Error: non-constant expression __funcliteral2 I can understand why the delegate version wouldn't work though. I mean, what would the context pointer be? But the function version should compile IMHO.
Jan 12 2007
prev sibling parent reply David Medlock <noone nowhere.com> writes:
Alain wrote:

 Hello,
 
 I am experimenting with closures and i have problem with the following code:
 
 import std.stdio;
 alias int delegate(int x) AddFunc;
 
 AddFunc addN(int n) {
     int add(int i) {
         return i + n;
     }
     return &add; // the add function captured will remember the value of n
 }
 
 void apply(int[] array, AddFunc func) {
     foreach (inout int i; array) {
         i = func(i);
     }
 }
 
 int main() {
     int[] numbers = [1,2,3];
     writefln("numbers before=",numbers);
     apply(numbers, addN(40)); // add 40 to each number
     writefln("numbers after=",numbers);
     return 0;
 }
 
 The result should be [41,42,43] but i get [4202694,4202695,4202696]
 
 Am i missing something?
 
 Alain
D doesn't support true closures, just delegates. Local variables in functions are still allocated on the stack and therefore invalid after you return from them. You must use objects for this purpose. -DavidM
Jan 11 2007
parent reply Alain <alainpoint yahoo.fr> writes:
David Medlock Wrote:

 D doesn't support true closures, just delegates.  Local variables in 
 functions are still allocated on the stack and therefore invalid after 
 you return from them.
 
 You must use objects for this purpose.
 
 -DavidM
Thank you for the suggestion. I post hereunder my solution to the problem. import std.stdio; class ClassClosure { this(int n) { num = n;} private uint num; int opCall(int x) {return num+x;} } void capply(int[] array, ClassClosure func) { foreach (inout int i; array) { i = func(i); } } int main() { int[] numbers = [1,2,3]; writefln("numbers before=",numbers); ClassClosure myclosure= new ClassClosure(40); capply(numbers, myclosure); // add 40 to each number writefln("numbers after=",numbers); return 0; } Alain
Jan 11 2007
parent reply Bradley Smith <digitalmars-com baysmith.com> writes:
Here is an alternate solution. One which requires fewer changes to the 
original design. Only the contents of the addN function have been 
changed. It is not necessarily better than the other solution. Just 
different.

import std.stdio;
alias int delegate(int x) AddFunc;

AddFunc addN(int n) {
     class A {
         int n;
         this(int n) {
             this.n = n;
         }
         int add(int i) {
             return i + n;
         }
     }
     A a = new A(n);
     return &a.add; // the add function captured will remember the value 
of n
}

void apply(int[] array, AddFunc func) {
     foreach (inout int i; array) {
         i = func(i);
     }
}

int main() {
     int[] numbers = [1,2,3];
     writefln("numbers before=",numbers);
     apply(numbers, addN(40)); // add 40 to each number
     writefln("numbers after=",numbers);
     return 0;
}

Alain wrote:
 David Medlock Wrote:
 
 D doesn't support true closures, just delegates.  Local variables in 
 functions are still allocated on the stack and therefore invalid after 
 you return from them.

 You must use objects for this purpose.

 -DavidM
Thank you for the suggestion. I post hereunder my solution to the problem. import std.stdio; class ClassClosure { this(int n) { num = n;} private uint num; int opCall(int x) {return num+x;} } void capply(int[] array, ClassClosure func) { foreach (inout int i; array) { i = func(i); } } int main() { int[] numbers = [1,2,3]; writefln("numbers before=",numbers); ClassClosure myclosure= new ClassClosure(40); capply(numbers, myclosure); // add 40 to each number writefln("numbers after=",numbers); return 0; } Alain
Jan 11 2007
parent reply Daniel Giddings <dgiddings bigworldtech.com> writes:
There is also a templated Curry function example in the D template docs:

http://www.digitalmars.com/d/template.html

/* R is return type
  * A is first argument type
  * U is TypeTuple of rest of argument types
  */
R delegate(U) Curry(R, A, U...)(R delegate(A, U) dg, A arg)
{
     struct Foo
     {
	typeof(dg) dg_m;
	typeof(arg) arg_m;

	R bar(U u)
	{
	    return dg_m(arg_m, u);
	}
     }

     Foo* f = new Foo;
     f.dg_m = dg;
     f.arg_m = arg;
     return &f.bar;
}

void main()
{
     int plus(int x, int y, int z)
     {
	return x + y + z;
     }

     auto plus_two = Curry(&plus, 2);
     printf("%d\n", plus_two(6, 8));	// prints 16
}
Jan 11 2007
parent reply Bradley Smith <digitalmars-com baysmith.com> writes:
Applying the Curry method to the current example yields the following.

import std.stdio;

alias int delegate(int x) AddFunc;

AddFunc addN(int n) {
     int add(int a, int b) {
         return a + b;
     }
     return Curry(&add, n); // the add function captured will remember the
value of n
}

void apply(int[] array, AddFunc func) {
     foreach (inout int i; array) {
         i = func(i);
     }
}

int main() {
     int[] numbers = [1,2,3];
     writefln("numbers before=",numbers);
     apply(numbers, addN(40)); // add 40 to each number
     writefln("numbers after=",numbers);

     return 0;
}

/* R is return type
  * A is first argument type
  * U is TypeTuple of rest of argument types
  */
R delegate(U) Curry(R, A, U...)(R delegate(A, U) dg, A arg) {
     struct Closure {
         typeof(dg) originalDelegate;
         A value;

         R subDelegate(U u) {
             return originalDelegate(value, u);
         }
     }

     Closure* f = new Closure;
     f.originalDelegate = dg;
     f.value = arg;
     return &f.subDelegate;
}


Daniel Giddings wrote:
 There is also a templated Curry function example in the D template docs:
 
 http://www.digitalmars.com/d/template.html
 
 /* R is return type
  * A is first argument type
  * U is TypeTuple of rest of argument types
  */
 R delegate(U) Curry(R, A, U...)(R delegate(A, U) dg, A arg)
 {
     struct Foo
     {
     typeof(dg) dg_m;
     typeof(arg) arg_m;
 
     R bar(U u)
     {
         return dg_m(arg_m, u);
     }
     }
 
     Foo* f = new Foo;
     f.dg_m = dg;
     f.arg_m = arg;
     return &f.bar;
 }
 
 void main()
 {
     int plus(int x, int y, int z)
     {
     return x + y + z;
     }
 
     auto plus_two = Curry(&plus, 2);
     printf("%d\n", plus_two(6, 8));    // prints 16
 }
Jan 11 2007
parent Daniel Giddings <dgiddings bigworldtech.com> writes:
Or with a small modification to Curry to curry a function instead of a 
delegate:

import std.stdio;

alias int delegate(int x) AddFunc;

int add(int a, int b) {
	return a + b;
}

void apply(int[] array, AddFunc func) {
     foreach (inout int i; array) {
         i = func(i);
     }
}

int main() {
     int[] numbers = [1,2,3];
     writefln("numbers before=",numbers);
     apply(numbers, CurryFunc( &add, 40 ) ); // add 40 to each number
     writefln("numbers after=",numbers);

     return 0;
}

/* R is return type
  * A is first argument type
  * U is TypeTuple of rest of argument types
  */
R delegate(U) CurryFunc(R, A, U...)(R function(A, U) fn, A arg) {
     struct Closure {
         typeof(fn) originalFunction;
         typeof(A) value;

         R subDelegate(U u) {
             return originalFunction(value, u);
         }
     }

     Closure* f = new Closure;
     f.originalFunction = fn;
     f.value = arg;
     return &f.subDelegate;
}

Bradley Smith wrote:
 Applying the Curry method to the current example yields the following.
 
 import std.stdio;
 
 alias int delegate(int x) AddFunc;
 
 AddFunc addN(int n) {
     int add(int a, int b) {
         return a + b;
     }
     return Curry(&add, n); // the add function captured will remember 
 the value of n
 }
 
 void apply(int[] array, AddFunc func) {
     foreach (inout int i; array) {
         i = func(i);
     }
 }
 
 int main() {
     int[] numbers = [1,2,3];
     writefln("numbers before=",numbers);
     apply(numbers, addN(40)); // add 40 to each number
     writefln("numbers after=",numbers);
 
     return 0;
 }
 
 /* R is return type
  * A is first argument type
  * U is TypeTuple of rest of argument types
  */
 R delegate(U) Curry(R, A, U...)(R delegate(A, U) dg, A arg) {
     struct Closure {
         typeof(dg) originalDelegate;
         A value;
 
         R subDelegate(U u) {
             return originalDelegate(value, u);
         }
     }
 
     Closure* f = new Closure;
     f.originalDelegate = dg;
     f.value = arg;
     return &f.subDelegate;
 }
 
 
Jan 11 2007