www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 3560] New: foreach on closure corrupted

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3560

           Summary: foreach on closure corrupted
           Product: D
           Version: 2.036
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: kovrov+puremagic gmail.com


--- Comment #0 from Serg Kovrov <kovrov+puremagic gmail.com> 2009-11-30
09:44:23 PST ---
import std.stream: File;
import std.stdio: writefln;

void outer()
{
    auto auth_file = new File("path-to-existing-file");
    writefln("outer: 0x%X", &auth_file);

    int inner(int delegate(ref ubyte[]) dg)
    {
        writefln("inner: 0x%X", &auth_file);
        return 0;
    }

    foreach (entry; &inner)
    {
        //...
    }
}

void main()
{
    outer();
}

--------------------

expected output:
 outer: 0xB7D19E44
 inner: 0xB7D19E44

actual output:
outer: 0xB7D19E44
inner: 0x4

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 30 2009
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3560


Don <clugdbug yahoo.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |wrong-code
                 CC|                            |clugdbug yahoo.com.au
            Summary|foreach on closure          |foreach on closure
                   |corrupted                   |corrupted after varargs
                   |                            |call
           Severity|normal                      |major


--- Comment #1 from Don <clugdbug yahoo.com.au> 2010-04-29 02:50:11 PDT ---
Reduced test case (this example segfaults at runtime) shows it is related to
varargs.

------------------------

void bug3560(int ...){ }

void main()
{
    int localvar = 7;
    bug3560(2);   // this call must be made before the foreach

    int inner(int delegate(ref int) dg) {
        int k = localvar;
        return 0;
    }

    foreach (entry; &inner)
    {
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 29 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3560


Don <clugdbug yahoo.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|foreach on closure          |foreach on closure
                   |corrupted after varargs     |corrupted after function
                   |call                        |call
         OS/Version|Linux                       |All
           Severity|major                       |critical


--- Comment #2 from Don <clugdbug yahoo.com.au> 2010-07-13 15:12:13 PDT ---
Further reduction shows that it doesn't require varargs. Raising priority to
critical, since it means that virtually all closures are broken.
What's happening: on the call to inner, EAX should hold a pointer to
the closure variables (on the heap). But when there's another function call,
it's using EBX instead, and the EAX value is whatever garbage is left after the
function call.
So it gets the calling convention wrong.
====================

void bug3560(int x){ }

void main()
{
    int localvar = 7;

    int inner(int delegate(ref int) dg) {
        int k = localvar; // BUG: &localvar == 0x24 (!)
        return 0;
    }
    bug3560(0x20);
    foreach (entry; &inner) { }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 13 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3560


Don <clugdbug yahoo.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|2.036                       |D1 & D2
            Summary|foreach on closure          |foreach over nested
                   |corrupted after function    |function generates wrong
                   |call                        |code


--- Comment #3 from Don <clugdbug yahoo.com.au> 2010-07-14 12:41:39 PDT ---
It doesn't even need a function call, and doesn't need a closure.
Anything which modifies the EAX register before the foreach will do it.
This test case generates bad code (runtime segfault) even on D1 (even old
versions, eg D1.020).
The inner function assumes that EAX contains the context pointer, but the
foreach code doesn't set EAX. Instead, it's passing the context pointer in EBX.
=====================
void main()
{
    int localvar = 7;

    int inner(int delegate(ref int) dg) {
        int k = localvar;
        return 0;
    }

    int a = localvar * localvar; // This modifies the EAX register

    foreach (entry; &inner)
    {
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 14 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3560


Don <clugdbug yahoo.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |patch


--- Comment #4 from Don <clugdbug yahoo.com.au> 2010-07-15 14:42:34 PDT ---
This is a front-end problem.
ForeachStatement::semantic() immediately runs aggr->semantic(). 
In this case, aggr is an address of a nested function 'inner'.
AddrExp->semantic() turns it into a DelegateExp.

Later on in Foreach::semantic, if the aggregate is of delegate type, it wraps
it in a CallExp, then runs CallExp::semantic. The problem is that CallExp
assumes that semantic has NOT been run on its argument.
This works fine if the aggregate is a delegate variable. But if it's a delegate
expression, CallExp transforms the DelegateExp for 'inner' into a
DotVarExp(inner, inner). And then it's a mess.  CallExp needs to be passed
'inner', not the delegateExp.

I can see two possible fixes. 
(1) Change CallExp so that changes DelegateExp(f,f) into CallExp(f), if f is a
nested function, instead of changing it into a DotVarExp.

OR
(2) If it's a delegate expression for a nested function, call the function
directly.
This patch implements the second method.

statement.c, Foreach::semantic(), line 1891

            else if (tab->ty == Tdelegate)
            {
                /* Call:
                 *      aggr(flde)
                 */
                Expressions *exps = new Expressions();
                exps->push(flde);

+                if (aggr->op == TOKdelegate &&
+                    ((DelegateExp *)aggr)->func->isNested())
+                    e = new CallExp(loc, ((DelegateExp *)aggr)->e1, exps);
+                else
+                    e = new CallExp(loc, aggr, exps);
-                e = new CallExp(loc, aggr, exps);
                e = e->semantic(sc);

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 15 2010
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3560


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla digitalmars.com
         Resolution|                            |FIXED


--- Comment #5 from Walter Bright <bugzilla digitalmars.com> 2010-07-24
16:24:48 PDT ---
http://www.dsource.org/projects/dmd/changeset/586

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 24 2010