www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 7543] New: inout opApply should work properly

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543

           Summary: inout opApply should work properly
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: k.hara.pg gmail.com
        Depends on: 7542


--- Comment #0 from Kenji Hara <k.hara.pg gmail.com> 2012-02-19 03:50:14 PST ---
After fixing issue 7542, inout opApply should work on foreach.

class C
{
    int[] arr;
    this(int[] a){ arr = a; }

    int opApply(int delegate(ref inout(int)) dg) inout
    {
        foreach (ref e; arr)
            if (auto r = dg(e)) return r;
        return 0;
    }
}
void main()
{
    size_t i;

    i = 0;
    foreach (e; new C([1,2,3]))
    {
        static assert(is(typeof(e) == int));
        assert(e == ++i);
        e = 10;
    }
    assert(i == 3);

    i = 0;
    foreach (e; new const(C)([1,2,3]))
    {
        static assert(is(typeof(e) == const int));
        assert(e == ++i);
        static assert(!__traits(compiles, e = 10));
    }
    assert(i == 3);

    i = 0;
    foreach (e; new immutable(C)([1,2,3]))
    {
        static assert(is(typeof(e) == immutable int));
        assert(e == ++i);
        static assert(!__traits(compiles, e = 10));
    }
    assert(i == 3);
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 19 2012
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull


--- Comment #1 from Kenji Hara <k.hara.pg gmail.com> 2012-02-19 04:39:09 PST ---
https://github.com/D-Programming-Language/dmd/pull/735

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 19 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|pull                        |


--- Comment #2 from Kenji Hara <k.hara.pg gmail.com> 2012-02-19 06:15:26 PST ---
Sorry, my thought in bug 7543 is not correct, so my pull is also bad.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 19 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543


Stewart Gordon <smjg iname.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |smjg iname.com
         Depends on|                            |7105


--- Comment #3 from Stewart Gordon <smjg iname.com> 2012-02-19 06:57:39 PST ---
We'll need a clear spec of what it means to have inout at two nesting levels of
a function signature.

digitalmars.D
"inout and function/delegate parameters"

On web interface:
http://forum.dlang.org/thread/jhr0t6$24v6$1 digitalmars.com

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 19 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543


Boscop <kingboscop gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kingboscop gmail.com


--- Comment #4 from Boscop <kingboscop gmail.com> 2012-03-09 10:24:53 PST ---
(In reply to comment #3)

 We'll need a clear spec of what it means to have inout at two nesting levels of
 a function signature.


This follows naturally from variance rules:
In-arguments are contravariant, so you can pass anything whose type is a
subtype of the argument type. Then the usual rules of function subtyping apply
(e.g. see bug 7676)
So, if you have a function of type "R1 function(R2 delegate(inout(T1)) f)" you
can pass anything for f that is a subtype of "R2 delegate(inout(T1))". For
argument types to the delegate, the subtyping rules apply again.
Since in-arguments are contravariant (and return types (and types of out-args)
are covariant), any value of type "R delegate(T)" where (R <: R2 and inout(T1)
<: T) is a subtype of "R2 delegate(inout(T1))" (<: is the subtype relation).

Also the following subtyping rules for inout apply (R2 <: R1):
R2 delegate(inout(T)) <: R1 delegate(immutable(T))
R2 delegate(inout(T)) <: R1 delegate(const(T))
R2 delegate(inout(T)) <: R1 delegate(T)
And of course:
R2 delegate(inout(T)) <: R1 delegate(inout(T))

From that follows (given two types A,B and B <: A):
B delegate(immutable(A)) <: A delegate(inout(A))

Which is exactly what we want. (Analogous for functions. Btw, functions should
be a subtype of delegates).

Example:
class A {}
class B : A {}
void foo(A function(A function(inout(A))) dg);
B bar(A function(inout(A)) f);
B baz(inout(A) a);
foo(bar(baz(new immutable(A))));
foo(bar(baz(new const(A))));
foo(bar(baz(new A)));

so it all works out nicely...

To make this work in DMD, it has to support function subtyping according to
variance rules. It's important that functions with an inout arg are subtypes of
functions with an immutable/const/mutable arg (all other arg types / return
type being equal) because the former can be substituted for one of the latter.

(This issue is related to bug 7542.)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543


timon.gehr gmx.ch changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |timon.gehr gmx.ch


--- Comment #5 from timon.gehr gmx.ch 2012-03-09 12:30:41 PST ---
(In reply to comment #4)

 (In reply to comment #3)

 We'll need a clear spec of what it means to have inout at two nesting levels of
 a function signature.


 
 This follows naturally from variance rules:
 [snip.]


I think you may misunderstand this issue.


 Also the following subtyping rules for inout apply (R2 <: R1):
 R2 delegate(inout(T)) <: R1 delegate(immutable(T))
 R2 delegate(inout(T)) <: R1 delegate(const(T))
 R2 delegate(inout(T)) <: R1 delegate(T)
 And of course:
 R2 delegate(inout(T)) <: R1 delegate(inout(T))
 
 From that follows (given two types A,B and B <: A):
 B delegate(immutable(A)) <: A delegate(inout(A))
 


No, it does not. This is not sound.

class A{immutable(A) get(){return null;}}
class B:A{
    immutable(A) other;
    this(immutable(A) other){this.other = other;}
    override immutable(A) get(){return other;}
}

A delegate(inout(A)) dg = (immutable(A) a) => new B(a);
A a = new A;
A x=dg(a);
immutable(A) b = x.get();

// now a is mutable, b is immutable and (a is b).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #6 from Boscop <kingboscop gmail.com> 2012-03-09 14:27:43 PST ---
(In reply to comment #5)

 (In reply to comment #4)

 (In reply to comment #3)

 We'll need a clear spec of what it means to have inout at two nesting levels of
 a function signature.


 
 This follows naturally from variance rules:
 [snip.]


 
 I think you may misunderstand this issue.
 

 Also the following subtyping rules for inout apply (R2 <: R1):
 R2 delegate(inout(T)) <: R1 delegate(immutable(T))
 R2 delegate(inout(T)) <: R1 delegate(const(T))
 R2 delegate(inout(T)) <: R1 delegate(T)
 And of course:
 R2 delegate(inout(T)) <: R1 delegate(inout(T))
 
 From that follows (given two types A,B and B <: A):
 B delegate(immutable(A)) <: A delegate(inout(A))
 


 
 No, it does not. This is not sound.
 
 class A{immutable(A) get(){return null;}}
 class B:A{
     immutable(A) other;
     this(immutable(A) other){this.other = other;}
     override immutable(A) get(){return other;}
 }
 
 A delegate(inout(A)) dg = (immutable(A) a) => new B(a);
 A a = new A;
 A x=dg(a);
 immutable(A) b = x.get();
 
 // now a is mutable, b is immutable and (a is b).


Your example is unsound (no offense :)

You can't do:
A delegate(inout(A)) dg = (immutable(A) a) => new B(a);
and neither:
void function(inout(int)*) wfp = function(int*)(*p = 1;} // as you did in
comment 2 of bug 7542

1. as I wrote in bug 7542:
Given (R2 <: R1)
R2 delegate(inout(T)) <: R1 delegate(immutable(T))
(because of argument contravariance and immutable(T) <: inout(T)).
You can't assign an instance of a supertype to an instance of a subtype.

2. What do you expect from that func ptr? That you can assign a function that
takes mutable, const or immutable values as args?
You should not be allowed to assign a function that takes an immutable arg
because it assumes it won't be modified outside (and you could be passing in
mutable/const args)!
And you can't assign a function that takes a mutable because it could change
the const arg!
So you are left with being able to assign functions that take a const arg and
those that take an inout arg, because they can be substituted for the former.

Thats why you'd use a pointer to a function taking a const arg! (see bug 7542)
void function(const(int)*) wfp;
and then you can also assign a void function(inout(int)*)
You can call wfp with mutable, const and immutable args because as I wrote in
comment 4 of bug 7542:
T <: const(T)
immutable(T) <: const(T)

3. Aside from all the above, inout already has different semantics inside
function bodies, so it would be ambiguous to have another instance of inout in
a variable declaration because you can only refer to one constness (that of the
args of the current function).
Consider:
void main() {
    class A {}
    inout(A) foo(inout(A) a) {
        inout(A) b = a; // inout already depends on a's constness
        // can't introduce a different inout like you want here
        inout(A) delegate(inout(A)) dg = (immutable(A) a) => new immutable(A);
        return b;
    }
    foo(new A);
    foo(new const(A));
    foo(new immutable(A));
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #7 from timon.gehr gmx.ch 2012-03-09 14:35:05 PST ---
(In reply to comment #6)

 (In reply to comment #5)

 (In reply to comment #4)

 From that follows (given two types A,B and B <: A):
 B delegate(immutable(A)) <: A delegate(inout(A))
 


 
 No, it does not. This is not sound.
 
 class A{immutable(A) get(){return null;}}
 class B:A{
     immutable(A) other;
     this(immutable(A) other){this.other = other;}
     override immutable(A) get(){return other;}
 }
 
 A delegate(inout(A)) dg = (immutable(A) a) => new B(a);
 A a = new A;
 A x=dg(a);
 immutable(A) b = x.get();
 
 // now a is mutable, b is immutable and (a is b).


 
 Your example is unsound (no offense :)


Obviously it is unsound. That is the point.


 
 You can't do:
 A delegate(inout(A)) dg = (immutable(A) a) => new B(a);


You claimed I could.

Boskop wrote:

 B delegate(immutable(A)) <: A delegate(inout(A))



 and neither:
 void function(inout(int)*) wfp = function(int*)(*p = 1;} // as you did in
 comment 2 of bug 7542
 


I know that both of those should not compile. They were specifically
constructed to demonstrate why certain subtyping relationships do not hold. I
suggest you to carefully read the relevant posts again.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #8 from Stewart Gordon <smjg iname.com> 2012-03-09 15:42:40 PST ---
I think the claim that the reporter is actually making is that the inout
constancy of the delegate parameter should vary with that of this.  In other
words,

    int opApply(int delegate(ref inout(int)) dg) inout

should be callable as if it's any one of these:

    int opApply(int delegate(ref int) dg)
    int opApply(int delegate(ref const(int)) dg) const
    int opApply(int delegate(ref immutable(int)) dg) immutable

I was trying to do this myself before this issue was filed.

(In reply to comment #6)

 3.  Aside from all the above, inout already has different semantics 
 inside function bodies, so it would be ambiguous to have another 
 instance of inout in a variable declaration because you can only 
 refer to one constness (that of the args of the current function).


Indeed.  The point is that there are two possible interpretations of the
opApply signature, and the spec isn't clear on which applies:

(a) the constancy is passed through to the delegate
(b) the delegate has an inout parameter in its own right

See the newsgroup thread I've already linked to for a more detailed discussion
of this.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #9 from timon.gehr gmx.ch 2012-03-09 16:05:28 PST ---
(In reply to comment #8)

 
 Indeed.  The point is that there are two possible interpretations of the
 opApply signature, and the spec isn't clear on which applies:
 
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right
 
 See the newsgroup thread I've already linked to for a more detailed discussion
 of this.


The main issue is that both (a) and (b) are useful in different contexts.
Otherwise this would be a no-brainer.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #10 from Steven Schveighoffer <schveiguy yahoo.com> 2012-03-09
16:07:27 PST ---
(In reply to comment #8)


 Indeed.  The point is that there are two possible interpretations of the
 opApply signature, and the spec isn't clear on which applies:
 
 (a) the constancy is passed through to the delegate
 (b) the delegate has an inout parameter in its own right


I think it must be (b).  Consider you don't know when the delegate was
constructed:

void foo(inout(int)* x, inout(int)* delegate(inout(int)* x) dg)
{
   inout(int)* bar(inout(int)* m) { return m;}
   auto dg2 = &bar;
   assert(typeof(dg2) == typeof(dg)); // ???

   immutable int y = 5;
   dg2(&y); // ok
   dg(&y); // must fail!
}

If the assert doesn't pass, then what is the type of dg vs. dg2?  If it passes,
then dg and dg2 are interchangeable, and you will violate const (what if x is
mutable?).  Even if the assert fails, it's going to be way *way* too confusing
to have two types that are identical in syntax be actually different types
under the hood.

We *absolutely* need a new syntax if case (a) is to be included.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #11 from timon.gehr gmx.ch 2012-03-09 16:16:14 PST ---
(In reply to comment #10)

 We *absolutely* need a new syntax if case (a) is to be included.


I agree.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #12 from Boscop <kingboscop gmail.com> 2012-03-09 16:21:28 PST ---
(In reply to comment #7)

 (In reply to comment #6)

 (In reply to comment #5)

 (In reply to comment #4)


 
 You can't do:
 A delegate(inout(A)) dg = (immutable(A) a) => new B(a);


 
 You claimed I could.
 
 Boskop wrote:

 B delegate(immutable(A)) <: A delegate(inout(A))




This is wrong, I didn't realize that I accidentally wrote that, sorry.
What I wrote above this (and in comment 6) was right:

Given R2 <: R1,
R2 delegate(inout(T)) <: R1 delegate(immutable(T))
(because of argument contravariance and immutable(T) <: inout(T)).

I didn't know at first that you wanted to point this out (probably was too
distracted by your code example).

(In reply to comment #8)

 I think the claim that the reporter is actually making is that the inout
 constancy of the delegate parameter should vary with that of this.  In other
 words,
 
     int opApply(int delegate(ref inout(int)) dg) inout
 
 should be callable as if it's any one of these:
 
     int opApply(int delegate(ref int) dg)
     int opApply(int delegate(ref const(int)) dg) const
     int opApply(int delegate(ref immutable(int)) dg) immutable


The problem is more deep-rooted. You can only pass delegates to opApply that
take a supertype of "ref inout(int)" as argument, but immutable(T) <: inout(T)
and const(T) <: inout(T), and T <: inout(T).

If it were possible to use inout that way, you would have to enable implicit
casts from T to immutable(T) (T <: immutable(T)) but you can't do that because
functions taking an immutable arg assume that it's not changed outside!
There is no way to have inout delegates like you want without enabling implicit
casts from T to immutable(T)!

But we don't need to enable implicit casts from T to immutable(T).

We can have our cake and eat it, too!

Now this is the important thing to notice:
In the case with opApply you actually want to transfer the constness of the
this object to the argument to dg!

foreach (e; new immutable(C)([1,2,3])) opApply takes ONLY a "int opApply(int
delegate(ref immutable(int)) dg) immutable"
foreach (e; new const(C)([1,2,3])) opApply takes ONLY a "int opApply(int
delegate(ref const(int)) dg) const"
foreach (e; new C([1,2,3])) opApply takes ONLY a "int opApply(int delegate(ref
int) dg)"

So we need a way to transfer the constness of the this object to declarations
within the class. This is basically inout for types (inout_t). (It's related to
the inout that is written after method signatures, but can now be be
transferred to arbitrary declarations).

So, with inout_t you would write:
class C {
    int[] arr;
    this(int[] a){arr = a;}
    int opApply(int delegate(ref inout_t(int)) dg) inout {
        foreach(ref e; arr)
            if(auto r = dg(e)) return r;
        return 0;
    }
}

BTW: If D didn't have transitive const, you could implement it with inout_t
like this:
struct S{inout_t(S)* next;}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #13 from timon.gehr gmx.ch 2012-03-09 16:36:19 PST ---
(In reply to comment #12)

 
 I didn't know at first that you wanted to point this out (probably was too
 distracted by your code example).
 


It seems to me in general that it could be beneficial to the quality of your
contributions if you would spend more time reading and less time writing. ;)


 
 So, with inout_t you would write:
 class C {
     int[] arr;
     this(int[] a){arr = a;}
     int opApply(int delegate(ref inout_t(int)) dg) inout {
         foreach(ref e; arr)
             if(auto r = dg(e)) return r;
         return 0;
     }
 }
 


This is an interesting suggestion. There are some other ideas discussed here:
http://forum.dlang.org/post/jhr0t6$24v6$1 digitalmars.com


 BTW: If D didn't have transitive const, you could implement it with inout_t
 like this:
 struct S{inout_t(S)* next;}


This is moot.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 09 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #14 from Stewart Gordon <smjg iname.com> 2012-03-10 03:45:21 PST ---
(In reply to comment #10)

 void foo(inout(int)* x, inout(int)* delegate(inout(int)* x) dg)
 {
    inout(int)* bar(inout(int)* m) { return m;}
    auto dg2 = &bar;
    assert(typeof(dg2) == typeof(dg)); // ???


Whether we pick (a) or (b), it should apply equally to declarations in the body
of foo as to parameters of foo itself.  So this assert would pass either way.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 10 2012
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=7543



--- Comment #15 from Stewart Gordon <smjg iname.com> 2012-03-10 04:00:38 PST ---
(In reply to comment #10)

 Even if the assert fails, it's going to be way *way* too confusing
 to have two types that are identical in syntax be actually different types
 under the hood.


Following on from my last comment, I see now that it reinforces why (b) is
better as the default behaviour.  If we went with (a), removing the inouts from
the signature of foo would completely change the meaning of the inouts in bar.

What (b) is saying is that inout is treated as referencing the constancy level
with which the function is called _only_ if it would be illegal in the absence
of an enclosing inout context.


 We *absolutely* need a new syntax if case (a) is to be included.


Would we allow it in the body of bar as well, for when we want to reference the
constancy level with which foo was called?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 10 2012