www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - logical const without casts!

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
I just thought of an interesting way to make a logical const object  
without casts.  It requires a little extra storage, but works without  
changes to the current compiler (and requires no casts).

Here's the code, then I'll talk about the implications:

import std.stdio;

class D
{
     D getme() { return this;}
     void foo() {writeln("mutable!");}
}

class C
{
     D delegate() d;
     this()
     {
       auto dinst = new D;
       this.d = &dinst.getme;
     }
     void bar() const { d().foo();}
}

void main()
{
     auto c = new C;
     c.bar();
}

outputs:

mutable!

So how does it work?  It works because delegates and especially the  
delegate data is *not* affected by const.  So even when C is temporarily  
cast to const, the delegate is not affected (i.e. it's context pointer is  
not temporarily cast to const).

Doesn't this poke holes in const?  Of course it does, but no more holes  
than are present via another logical const scheme (i.e. using a globally  
stored AA to retrieve the data).

Why is this better than simply casting away const?

First, because simply casting away const and modifying the data is  
undefined, I don't think the method I created results in undefined  
behavior.  Second, because compiler is still fully enforcing  
const/immutable.  For example, you could not assign d to &dinst.getme if  
dinst is const or immutable, and once inside a const function, I cannot  
change the delegate, I can only call it.  Third, because the D instance  
*doesn't actually live* in the C object.  You can see, I *never* assigned  
a D instance to a member variable of C, I only assigned the delegate.  The  
D instance lives on the heap (and technically can be passed in via the  
constructor if need be).

The only drawbacks here are, we have to needlessly store the delegate  
function pointer (assuming we are always going to use &getme), and the  
delegate cannot be inlined.

I'm actually thinking that very controlled patterns of logical const like  
this could be implemented via mixin, and be sanctioned by the library.   
The way this pattern works, you can dictate as the author of a class  
whether that class can be a logically const part of another object or not,  
simply by choosing to implement getme or not.

What do people think about this?

-Steve
Sep 29 2011
next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).

 What do people think about this?

This is what I think about it: class A { int n; void delegate( ) dg; } pure A createAnA( int n ) { A result = new A; result.n = n; result.dg = (){ result.n++; }; return result; } void main( ) { immutable A tmp = createAnA( 3 ); assert( tmp.n == 3 ); tmp.dg(); assert( tmp.n == 3 ); } -- Simen
Sep 29 2011
parent reply travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145767), a
 écrit :
 On Thu, 29 Sep 2011 13:26:11 -0400, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:
 
 On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas  
 <simen.kjaras gmail.com> wrote:

 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).

 What do people think about this?

This is what I think about it:

I agree this breaks immutability, and needs to be addressed. I think probably implicit casting of delegates (or items that containe delegates) to immutable from strong-pure functions should be disallowed. But it's not the pattern I described, and uses a relatively new trick (implicit immutable casting). I'll file a bug for this case.

http://d.puremagic.com/issues/show_bug.cgi?id=6741

Well, if the langage wants to be consistent, you should prevent implicit casting of delegates to const, not just to immutable. What you revealed is very interesting, but it is clearly a bug. A delegate context pointer in a const object should be const, and the delegate function should be const with regard to its context pointer if you want the function to be called when the delegate is const. Constness has to apply to delegates just like it applies to structs. The explanation gets messy. An example: This is what the compiler should say:
 class A {
      int n;
      void delegate( ) dg;
 }

 pure
 A createAnA( int n ) {
      A result = new A;
      result.n = n;
      result.dg = (){ result.n++; };
      return result;
 }

 void main( ) {
      immutable A tmp = createAnA( 3 );
      assert( tmp.n == 3 );
      tmp.dg();



      assert( tmp.n == 3 );
 }



An this is the fix:
 class A {
      int n;



 }

 pure
 A createAnA( int n ) {
      A result = new A;
      result.n = n;



// error: non const (unamed) delegate cannot be assigned to // const delegate A.dg. result.dg = (){ result.n+1; }; // this one is fine.
      return result;
 }

 void main( ) {
      immutable A tmp = createAnA( 3 );
      assert( tmp.n == 3 );



      assert( tmp.n == 3 );
 }



I guess you could also hide a mutable pointer of an immutable object during its construction with the same trick. -- Christophe
Sep 30 2011
parent reply travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145821), a
 écrit :
 If const must be a part of the signature, then you have lost the typeless  
 aspect of the context pointer.  If we expose the const (or not const)  
 nature of the pointer, then you lose the interoperability that delegates  
 provide.  The beauty of delegates is you don't *have to* care what the  
 type of the context pointer is.  That's defined by the function the  
 delegate represents.

No, you don't lose the typeless aspect of the context pointer. You lose a litle bit of that aspect, but not that much. In D, const is transitive. That means: everything you touch via a const object remains const. non-const delegate context pointer breaks that rule. You have const member functions, and initially, delegates were member functions associated with an object, so why could not the delegate be const ? A const delegate would just be a delegate that promise it doesn't touch to its context. It's not a type-defined delegate. If I implement a delegate at the library level, when I call the function, I have to make a cast to access the context pointer. With cast removing implicitely the constness of the context pointer, I get the current D implementation of delegates. This is legal, but this is also undefined behavior. I believe that langage delegate work that way, that they are thus undefined behavior, and this should be corrected. I agree the langage could stipulate that delegate context pointer are exceptions, to allow the valuable trick like you revealed. But, to me, that is not the current policy about constness. Do you see the problem in the following code: class Foo { this(ref int _i) { i = () { return _i; } } ref int delegate() dg; ref int i() const { return dg(); } } int globalInt; immutable Foo globalFoo = Foo(globalInt); int bar(const Foo foo) { return foo.i++; } int globalBar() { return bar(globalFoo); } Answer: In multithreaded application, globalFoo, which is immutable, is automatically shared. However, globalInt is not. globalBar allows you to access to (and modify) globalInt. But globalInt is not protected for concurrent access. And globalInt from which thread is accessed ? Possible solutions: - do nothing, and let the programmer introducing multi-threading in the application deal with this, even if the programmers of Foo, bar and buggy did not cared to document the impact of their code for multithreaded applications. - forbid to put/call any delegate into an immutable object: that almost means forbiding to put/call a delegate into a const object. - what I propose: implement the separate kind of const delegates, that allows to protect their context pointers, and that you can safely call from const/immutable data.
 But in the case of the bug I filed, we are talking about a  
 compiler-sanctioned implicit transformation to immutable.  We *must*  
 guarantee that when we allow an implicit transformation, that there are no  
 existing mutable copies of the data.  An explicit transformation should be  
 allowed, because then the user has accepted responsibility.

With my proposal, you can very easily keep an immutable reference in a const delegate. The delegate will just not be callable if its function pointer is not const with regard to the context pointer. In any case, if the langage decides that delegate context pointer should escape const protection, great care should be taken to make sure they don't escape purity. In my example, there should be a compiler error if I tried to declare Foo.i and bar pure (maybe there is already an error, I didn't test). -- Christophe
Sep 30 2011
parent reply travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145850), a
 As long as you assume the context pointer is part of the state of the  
 aggregate, then yes.  But I don't see it that way.  The context pointer is  
 part of the state of the delegate, and the delegate reference itself is  
 the same as a function pointer -- it's not affected by immutable or const.

That is actually where our opinions differ. I wouldn't blame the langage for choosing you proposal rather than mine. I just personnaly think my proposal in more in the spirit of the langage, and offer more guarantees when dealing with immutables.
 This also doesn't work:

globalFoo to immutable.
 class Foo
 {
     this(ref int _i) { dg = () { return _i; } }
     ref const(int) delegate() const dg;
     ref const(int) i() const { return dg(); }
 }
 
 int globalInt;
 
 immutable Foo globalFoo = Foo(globalInt);

Foo.this()._delegate_1.
 void thread1()
 {
     writeln(globalFoo.i);
 }
 
 int globalBar()
 {
     spawn(&thread1);
     globalInt = 5; // does thread1 see this change or not?
 }

 It looks like any time you call a delegate member of an immutable object,  
 it could access a context pointer that is not implicitly shared.

Not if the compiler cares to check the delegate context is indeed immutable when the immutable delegate is created. It is not much more complicated than if Foo contained an explicit pointer to i instead of a delegate
 Not to mention, what the hell does a const function mean for a delegate  
 literal?  The context pointer is the context of the function, not an  
 object.  How does that even work?

Internally, it is a pointer to a structure containing the delegate context that is generated by the compiler. If the delegate is const, the pointer should be a const pointer. If the delegate is immutable, it should be an immutable pointer, and the compiler as to check there is no mutable version of that pointer before making an implicit cast.
 What is probably the "right" solution is to disallow implicit immutable  
 objects which have a delegate.  This means:
 
 a) you cannot initialize a shared or immutable such object without a  
 cast.  This means the line immutable Foo globalFoo = new Foo(globalInt) is  
 an error without a cast.

Agreed, but with my proposal, you can have an immutable Foo, if the delegate context pointer can be cast to immutable.
 b) Any time you have a const object that contains a delegate, it can be  
 assumed that the object is not shared.

If the object is const, you have no guarantee that the objet is not shared, making an exception for object containing a delegate is quite wierd.
 And then we avoid dealing with the const delegate issue altogether.

Dealing with this issue could be very useful for multithreaded applications.
 With my proposal, you can very easily keep an immutable reference in a
 const delegate. The delegate will just not be callable if its
 function pointer is not const with regard to the context pointer.

You can also very easily keep a mutable reference in a const delegate inside an immutable object. It's not mutable through the delegate, but it's also not immutable.

No, the compiler should check the delegate context is immutable when making the cast to immutable. I admit you have found a very interesting corner-case for my proposal that I had not thought of that case. Thank you for that. I think the proposal passed the test (until now).
Sep 30 2011
parent travert phare.normalesup.org (Christophe) writes:
"Steven Schveighoffer" , dans le message (digitalmars.D:145867), a
 écrit :
 But ref int _i is the parameter to the constructor.  Here is where your  
 proposal breaks down.
 
 It's entirely feasible for class Foo to be compiled *separately* from the  
 module that contains globalFoo.  And it's also possible that when  
 compiling the above line, the language *does not have* access to the  
 source code of the constructor.  How does it know that _i gets put into a  
 const delegate?
 
 Likewise, when compiling Foo, how does the compiler know that the  
 constructor is being used to cast to immutable?  I just don't think there  
 is enough information for the compiler to make the correct decision.

You say it is impossible for the compiler to check the constness of the arguments passed to the constructor ? Well, allow me to doubt your statement, because my compiler can (it is gdc 4.4.5 using dmd 2.052). Easy test, with a pointer instead of a delegate: struct Foo { int* _i; ref int i() { return *_i; } this(int* m_i) { _i = m_i; } this(const int* m_i) const { _i = m_i; } this(immutable(int)* m_i) immutable // change immutable attribute of // the constructor and main won't compile. { _i = m_i; } } void main() { int ma; Foo mfoo = Foo(&ma); const int ca; const Foo cfoo = Foo(&ca); immutable int ia; immutable Foo ifoo = Foo(&ia); } Now try to remove the const or the immutable attributes in the different this(int*) overload. You can play and try many changes, if you find a way to put an immutable int inside a mutable Foo, you get a banana for having found a great hole in the langage (or at least in the compiler). Protect can (and has to!) be assured for a simple pointer. Why would it not be the case with a delegate context pointer ? Now the full test with delegate: module a; struct B { int* _b; this(int* m_b) { _b = m_b; } this(const(int)* m_b) const { _b = m_b; } this(immutable(int)* m_b) immutable { _b = m_b; } int* b() { return _b; } const(int)* b() const { return _b; } immutable(int)* b() immutable { return _b; } } struct Foo { int* delegate() _i; ref int i() { return *_i(); } this(int* m_i) { B b = B(m_i); _i = &b.b; } this(const int* m_i) const { const B b = B(m_i); _i = &b.b; } this(immutable int* m_i) immutable { immutable B b = B(m_i); _i = &b.b; // try to uncomment the following line: // _i = () { return m_i; }; } } module test; import a; void main() { int ma; Foo mfoo = Foo(&ma); const int ca; const Foo cfoo = Foo(&ca); immutable int ia; immutable Foo ifoo = Foo(&ia); } To know why I created the struct B, try to uncomment the line I indicated in a.d and compile. What do you discover ? a.d:34: Error: cannot implicitly convert expression (__dgliteral1) of type immutable(int*) delegate() to immutable(int* delegate()) So the type I thought I was proposing, the const or immutable delegate that protects its context pointer already exists in my compiler. The only problem is that at the moment, you have to use an helper structure to create this type of delegate...
 What if it has no access to the delegate creation code?  How does it  
 disallow it?

The compiler cannot cast a newly constructed structure to immutable if it cannot access the creation code, unless the creation code is marked immutable (and then there is actually no cast to do). Me thinking I could improve the langage... Well, the conversation was interesting and I learned a lot about d today. -- Christophe
Sep 30 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas  
<simen.kjaras gmail.com> wrote:

 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).

 What do people think about this?

This is what I think about it: class A { int n; void delegate( ) dg; } pure A createAnA( int n ) { A result = new A; result.n = n; result.dg = (){ result.n++; }; return result; } void main( ) { immutable A tmp = createAnA( 3 ); assert( tmp.n == 3 ); tmp.dg(); assert( tmp.n == 3 ); }

I agree this breaks immutability, and needs to be addressed. I think probably implicit casting of delegates (or items that containe delegates) to immutable from strong-pure functions should be disallowed. But it's not the pattern I described, and uses a relatively new trick (implicit immutable casting). I'll file a bug for this case. -Steve
Sep 29 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 29 Sep 2011 13:26:11 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas  
 <simen.kjaras gmail.com> wrote:

 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).

 What do people think about this?

This is what I think about it: class A { int n; void delegate( ) dg; } pure A createAnA( int n ) { A result = new A; result.n = n; result.dg = (){ result.n++; }; return result; } void main( ) { immutable A tmp = createAnA( 3 ); assert( tmp.n == 3 ); tmp.dg(); assert( tmp.n == 3 ); }

I agree this breaks immutability, and needs to be addressed. I think probably implicit casting of delegates (or items that containe delegates) to immutable from strong-pure functions should be disallowed. But it's not the pattern I described, and uses a relatively new trick (implicit immutable casting). I'll file a bug for this case.

http://d.puremagic.com/issues/show_bug.cgi?id=6741 -Steve
Sep 29 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 09:27:31 -0400, Christophe  =

<travert phare.normalesup.org> wrote:

 "Steven Schveighoffer" , dans le message (digitalmars.D:145767), a
  =C3=A9crit :
 On Thu, 29 Sep 2011 13:26:11 -0400, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 On Thu, 29 Sep 2011 13:06:55 -0400, Simen Kjaeraas
 <simen.kjaras gmail.com> wrote:

 On Thu, 29 Sep 2011 16:54:24 +0200, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 I just thought of an interesting way to make a logical const objec=





 without casts.  It requires a little extra storage, but works with=





 changes to the current compiler (and requires no casts).

 What do people think about this?

This is what I think about it:

I agree this breaks immutability, and needs to be addressed. I thin=



 probably implicit casting of delegates (or items that containe
 delegates) to immutable from strong-pure functions should be  =



 disallowed.

 But it's not the pattern I described, and uses a relatively new tric=



 (implicit immutable casting).  I'll file a bug for this case.

http://d.puremagic.com/issues/show_bug.cgi?id=3D6741

Well, if the langage wants to be consistent, you should prevent implic=

 casting of delegates to const, not just to immutable.

 What you revealed is very interesting, but it is clearly a bug. A
 delegate context pointer in a const object should be const, and the
 delegate function should be const with regard to its context pointer i=

 you want the function to be called when the delegate is const.
 Constness has to apply to delegates just like it applies to structs. T=

 explanation gets messy.

If const must be a part of the signature, then you have lost the typeles= s = aspect of the context pointer. If we expose the const (or not const) = nature of the pointer, then you lose the interoperability that delegates= = provide. The beauty of delegates is you don't *have to* care what the = type of the context pointer is. That's defined by the function the = delegate represents. It's why, for example, a function accepting a delegate does not = distinguish between a delegate literal and a delegate to a member functi= on = of type Foo or a delegate to a member function of type const(Bar). You = = must think of the context pointer as a hidden parameter to the delegate,= = as defined when the delegate was created *not* when it is called. The = fact that it's actually stored with the delegate pointer is irrelevant. = = Conceptually, it's not stored anywhere, it's just a parameter. But in the case of the bug I filed, we are talking about a = compiler-sanctioned implicit transformation to immutable. We *must* = guarantee that when we allow an implicit transformation, that there are = no = existing mutable copies of the data. An explicit transformation should = be = allowed, because then the user has accepted responsibility. The same is not true for const. It's *perfectly legal* to have a const = = and mutable reference to an object at the same time. The only reason = transitive const exists and is necessary is to support transitive = immtuable. Const is never guaranteed to prevent changing data on an object: class C { int x; void multiply(C other) const {other.x *=3D x;} } Is multiply guaranteed not to change the object? No: c.multiply(c); However, if c is immutable, it *is* guaranteed, because there's no way t= o = pass c as the parameter to multiply. That is why I think the solution works -- if you allow the compiler to = enforce the rules of const and immutable, you can still do unexpected = things, but you do not break the guarantees of immutability. It's = definitely a loophople, but I think it's valid.
 I guess you could also hide a mutable pointer of an immutable object
 during its construction with the same trick.

But saving a mutable object as immutable requires a cast. It means = "compiler, I take responsibility for ensuring this no longer has any = mutable references". When the cast is *not* required, it's the compiler= 's = responsibility. -Steve
Sep 30 2011
prev sibling next sibling parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 29.09.2011, 16:54 Uhr, schrieb Steven Schveighoffer  
<schveiguy yahoo.com>:

 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).

 Here's the code, then I'll talk about the implications:

 import std.stdio;

 class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }

 class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }

 void main()
 {
      auto c = new C;
      c.bar();
 }

 [...]

 What do people think about this?

 -Steve

I think this is a creative use of delegates! -------- class Vendor { void sell() { ++_sold; } property int sold() { return _sold; } private: int _sold = 1; } class Car { this(Vendor vendor) { _vendorDlg = () { return vendor; }; } Vendor getVendor() const { return _vendorDlg(); } private: Vendor delegate() _vendorDlg; } void main(string[] args) { immutable Car car = new immutable(Car)(new Vendor()); Vendor vendor = car.getVendor(); vendor.sell(); }
Sep 30 2011
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 I just thought of an interesting way to make a logical const object  
 without casts.  It requires a little extra storage, but works without  
 changes to the current compiler (and requires no casts).
 
 Here's the code, then I'll talk about the implications:
 
 import std.stdio;
 
 class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
 
 class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
 
 void main()
 {
      auto c = new C;
      c.bar();
 }
 
 outputs:
 
 mutable!
 
 So how does it work?  It works because delegates and especially the  
 delegate data is *not* affected by const.  So even when C is 
 temporarily  cast to const, the delegate is not affected (i.e. it's 
 context pointer is  not temporarily cast to const).
 
 Doesn't this poke holes in const?  Of course it does, but no more holes 
  than are present via another logical const scheme (i.e. using a 
 globally  stored AA to retrieve the data).

This is a hole in the transitive const, because the delegate contains a pointer to mutable data. It also is a potential source of of low level races since returning that type from a pure function could make it immutable, which can then make this mutable data accessible to multiple threads with no synchronization or atomics to protect the data's integrity.
 I'm actually thinking that very controlled patterns of logical const 
 like  this could be implemented via mixin, and be sanctioned by the 
 library.   The way this pattern works, you can dictate as the author of 
 a class  whether that class can be a logically const part of another 
 object or not,  simply by choosing to implement getme or not.

Whatever the implementation I think this is deeply needed. It is needed because people are trying all sorts of things to work around const transitivity, many of which are subtly unsafe. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Sep 30 2011
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-09-30 17:46:13 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 On Fri, 30 Sep 2011 13:38:31 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 
 On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:
 
 I just thought of an interesting way to make a logical const object   
 without casts.  It requires a little extra storage, but works without   
 changes to the current compiler (and requires no casts).
  Here's the code, then I'll talk about the implications:
  import std.stdio;
  class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
  class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
  void main()
 {
      auto c = new C;
      c.bar();
 }
  outputs:
  mutable!
  So how does it work?  It works because delegates and especially the   
 delegate data is *not* affected by const.  So even when C is  
 temporarily  cast to const, the delegate is not affected (i.e. it's  
 context pointer is  not temporarily cast to const).
  Doesn't this poke holes in const?  Of course it does, but no more  
 holes  than are present via another logical const scheme (i.e. using a  
 globally  stored AA to retrieve the data).

This is a hole in the transitive const, because the delegate contains a pointer to mutable data. It also is a potential source of of low level races since returning that type from a pure function could make it immutable, which can then make this mutable data accessible to multiple threads with no synchronization or atomics to protect the data's integrity.

Simen Kjaeraas and Christophe brought up the same points. I filed a bug on it: http://d.puremagic.com/issues/show_bug.cgi?id=6741

Interesting proposal. You realize if we had a true 'mutable' storage class for class members it could obey the same rule as you propose in that bug report: it should be illegal to cast a mutable-containing object or struct to immutable implicitly. An explicit cast should be required. Also, if we had shared delegates -- delegates which are guarantied to only manipulate shared data -- we could allow the class that contains a shared delegate to be implicitly converted to immutable. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Sep 30 2011
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2011-09-30 18:07:54 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:

 On Fri, 30 Sep 2011 14:00:36 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 
 Interesting proposal. You realize if we had a true 'mutable' storage  
 class for class members it could obey the same rule as you propose in  
 that bug report: it should be illegal to cast a mutable-containing  
 object or struct to immutable implicitly. An explicit cast should be  
 required.

Yes, but the benefit of the proposal is -- it already works in the current compiler :) You have a monstrous hill to climb to convince Walter to *add* mutable storage class, but we don't even have to ask for this one, it works today.

It works today but is unsafe due to the implicit cast. But yeah, lets use delegates for now. After the implicit cast is fixed it might be the right time to point out to Walter that this exception for delegate is no worse than having a direct mutable keyword. Especially if everyone is using this delegate detour to achieve what would be simpler and more efficient with 'mutable'.
 Also, if we had shared delegates -- delegates which are guarantied to  
 only manipulate shared data -- we could allow the class that contains a 
  shared delegate to be implicitly converted to immutable.

I'd like to avoid the complication of having any attributes that apply to the context pointer. I like how delegates just fit in anywhere, no matter what the context pointer type is (or its constancy).

But that's no longer true. Delegates no longer fit anywhere with the fix you just proposed. Only 'shared' delegates would fit anywhere. I really means *anywhere* since implicitly casting a shared delegate to a non-shared delegates is a non-issue: it does not change the type of the variables the context is pointing to. So basically you'd need to require a shared delegate only where a normal delegate cannot do the job (like in a class you want to make immutable, or wanting to pass the delegate across threads). -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Sep 30 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 12:16:03 -0400, Christophe  =

<travert phare.normalesup.org> wrote:

 "Steven Schveighoffer" , dans le message (digitalmars.D:145821), a
  =C3=A9crit :
 If const must be a part of the signature, then you have lost the  =


 typeless
 aspect of the context pointer.  If we expose the const (or not const)=


 nature of the pointer, then you lose the interoperability that delega=


 provide.  The beauty of delegates is you don't *have to* care what th=


 type of the context pointer is.  That's defined by the function the
 delegate represents.

No, you don't lose the typeless aspect of the context pointer. You los=

 a litle bit of that aspect, but not that much.

 In D, const is transitive. That means: everything you touch via a cons=

 object remains const. non-const delegate context pointer breaks that
 rule.

But transitive const by itself doesn't get you any guarantees. It does = = not prevent you from modifying that (mutable) value via another pointer.= What it does is allow code that works with both transitive immutable and= = mutable without having to copy code.
 You have const member functions, and initially, delegates were member
 functions associated with an object, so why could not the delegate be
 const ? A const delegate would just be a delegate that promise it
 doesn't touch to its context. It's not a type-defined delegate.

A delegate for a const member function today already does not touch its = = context. There is no need to expose the type.
 If I implement a delegate at the library level, when I call the
 function, I have to make a cast to access the context pointer. With ca=

 removing implicitely the constness of the context pointer, I get the
 current D implementation of delegates. This is legal, but this is also=

 undefined behavior. I believe that langage delegate work that way, tha=

 they are thus undefined behavior, and this should be corrected.

As long as you assume the context pointer is part of the state of the = aggregate, then yes. But I don't see it that way. The context pointer = is = part of the state of the delegate, and the delegate reference itself is = = the same as a function pointer -- it's not affected by immutable or cons= t.
 Do you see the problem in the following code:

 class Foo
 {
   this(ref int _i) { i =3D () { return _i; } }
   ref int delegate() dg;
   ref int i() const { return dg(); }
 }

 int globalInt;

 immutable Foo globalFoo =3D Foo(globalInt);

 int bar(const Foo foo)
 {
     return foo.i++;
 }

 int globalBar()
 {
     return bar(globalFoo);
 }


 Answer:

 In multithreaded application, globalFoo, which is immutable, is
 automatically shared. However, globalInt is not. globalBar allows you =

 access to (and modify) globalInt. But globalInt is not protected for
 concurrent access. And globalInt from which thread is accessed ?

Yes, I see the problem. It comes from immutable being shared implicitly= . = To reiterate, the fact that the data is not immutable or const is *not* = an = issue, globalInt is not immutable. The issue is purely the sharing aspe= ct = of immutable.
 Possible solutions:

 - do nothing, and let the programmer introducing multi-threading in
 the application deal with this, even if the programmers of Foo, bar an=

 buggy did not cared to document the impact of their code for
 multithreaded applications.

 - forbid to put/call any delegate into an immutable object: that almos=

 means forbiding to put/call a delegate into a const object.

 - what I propose: implement the separate kind of const delegates, that=

 allows to protect their context pointers, and that you can safely call=

 from const/immutable data.

This also doesn't work: class Foo { this(ref int _i) { i =3D () { return _i; } } ref const(int) delegate() const dg; ref const(int) i() const { return dg(); } } int globalInt; immutable Foo globalFoo =3D Foo(globalInt); void thread1() { writeln(globalFoo.i); } int globalBar() { spawn(&thread1); globalInt =3D 5; // does thread1 see this change or not? } It looks like any time you call a delegate member of an immutable object= , = it could access a context pointer that is not implicitly shared. Not to mention, what the hell does a const function mean for a delegate = = literal? The context pointer is the context of the function, not an = object. How does that even work? What is probably the "right" solution is to disallow implicit immutable = = objects which have a delegate. This means: a) you cannot initialize a shared or immutable such object without a = cast. This means the line immutable Foo globalFoo =3D new Foo(globalInt= ) is = an error without a cast. b) Any time you have a const object that contains a delegate, it can be = = assumed that the object is not shared. And then we avoid dealing with the const delegate issue altogether.
 But in the case of the bug I filed, we are talking about a
 compiler-sanctioned implicit transformation to immutable.  We *must*
 guarantee that when we allow an implicit transformation, that there a=


 no
 existing mutable copies of the data.  An explicit transformation shou=


 be
 allowed, because then the user has accepted responsibility.

With my proposal, you can very easily keep an immutable reference in a=

 const delegate. The delegate will just not be callable if its
 function pointer is not const with regard to the context pointer.

You can also very easily keep a mutable reference in a const delegate = inside an immutable object. It's not mutable through the delegate, but = = it's also not immutable.
 In any case, if the langage decides that delegate context pointer shou=

 escape const protection, great care should be taken to make sure they
 don't escape purity. In my example, there should be a compiler error i=

 I tried to declare Foo.i and bar pure (maybe there is already an error=

 I didn't test).

Yes, delegates that are pure must be typed that way (there's actually a = = bug I think where you can't explicitly name a pure delegate type). But = a = pure attribute applies to the *function* not the context pointer. -Steve
Sep 30 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 13:38:31 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:

 On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:

 I just thought of an interesting way to make a logical const object   
 without casts.  It requires a little extra storage, but works without   
 changes to the current compiler (and requires no casts).
  Here's the code, then I'll talk about the implications:
  import std.stdio;
  class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
  class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
  void main()
 {
      auto c = new C;
      c.bar();
 }
  outputs:
  mutable!
  So how does it work?  It works because delegates and especially the   
 delegate data is *not* affected by const.  So even when C is  
 temporarily  cast to const, the delegate is not affected (i.e. it's  
 context pointer is  not temporarily cast to const).
  Doesn't this poke holes in const?  Of course it does, but no more  
 holes  than are present via another logical const scheme (i.e. using a  
 globally  stored AA to retrieve the data).

This is a hole in the transitive const, because the delegate contains a pointer to mutable data. It also is a potential source of of low level races since returning that type from a pure function could make it immutable, which can then make this mutable data accessible to multiple threads with no synchronization or atomics to protect the data's integrity.

Simen Kjaeraas and Christophe brought up the same points. I filed a bug on it: http://d.puremagic.com/issues/show_bug.cgi?id=6741 -Steve
Sep 30 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 13:46:13 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 Simen Kjaeraas and Christophe brought up the same points.  I filed a bug  
 on it:

 http://d.puremagic.com/issues/show_bug.cgi?id=6741

Also now: http://d.puremagic.com/issues/show_bug.cgi?id=6747 -Steve
Sep 30 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 14:00:36 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:

 On 2011-09-30 17:46:13 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:

 On Fri, 30 Sep 2011 13:38:31 -0400, Michel Fortin   
 <michel.fortin michelf.com> wrote:

 On 2011-09-29 14:54:24 +0000, "Steven Schveighoffer"   
 <schveiguy yahoo.com> said:

 I just thought of an interesting way to make a logical const object    
 without casts.  It requires a little extra storage, but works  
 without   changes to the current compiler (and requires no casts).
  Here's the code, then I'll talk about the implications:
  import std.stdio;
  class D
 {
      D getme() { return this;}
      void foo() {writeln("mutable!");}
 }
  class C
 {
      D delegate() d;
      this()
      {
        auto dinst = new D;
        this.d = &dinst.getme;
      }
      void bar() const { d().foo();}
 }
  void main()
 {
      auto c = new C;
      c.bar();
 }
  outputs:
  mutable!
  So how does it work?  It works because delegates and especially  
 the   delegate data is *not* affected by const.  So even when C is   
 temporarily  cast to const, the delegate is not affected (i.e. it's   
 context pointer is  not temporarily cast to const).
  Doesn't this poke holes in const?  Of course it does, but no more   
 holes  than are present via another logical const scheme (i.e. using  
 a  globally  stored AA to retrieve the data).

a pointer to mutable data. It also is a potential source of of low level races since returning that type from a pure function could make it immutable, which can then make this mutable data accessible to multiple threads with no synchronization or atomics to protect the data's integrity.

bug on it: http://d.puremagic.com/issues/show_bug.cgi?id=6741

Interesting proposal. You realize if we had a true 'mutable' storage class for class members it could obey the same rule as you propose in that bug report: it should be illegal to cast a mutable-containing object or struct to immutable implicitly. An explicit cast should be required.

Yes, but the benefit of the proposal is -- it already works in the current compiler :) You have a monstrous hill to climb to convince Walter to *add* mutable storage class, but we don't even have to ask for this one, it works today.
 Also, if we had shared delegates -- delegates which are guarantied to  
 only manipulate shared data -- we could allow the class that contains a  
 shared delegate to be implicitly converted to immutable.

I'd like to avoid the complication of having any attributes that apply to the context pointer. I like how delegates just fit in anywhere, no matter what the context pointer type is (or its constancy). -Steve
Sep 30 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Sep 2011 14:51:19 -0400, Christophe  
<travert phare.normalesup.org> wrote:

 "Steven Schveighoffer" , dans le message (digitalmars.D:145850), a

 This also doesn't work:

globalFoo to immutable.
 class Foo
 {
     this(ref int _i) { dg = () { return _i; } }
     ref const(int) delegate() const dg;
     ref const(int) i() const { return dg(); }
 }

 int globalInt;

 immutable Foo globalFoo = Foo(globalInt);

Foo.this()._delegate_1.

But ref int _i is the parameter to the constructor. Here is where your proposal breaks down. It's entirely feasible for class Foo to be compiled *separately* from the module that contains globalFoo. And it's also possible that when compiling the above line, the language *does not have* access to the source code of the constructor. How does it know that _i gets put into a const delegate? Likewise, when compiling Foo, how does the compiler know that the constructor is being used to cast to immutable? I just don't think there is enough information for the compiler to make the correct decision.
 void thread1()
 {
     writeln(globalFoo.i);
 }

 int globalBar()
 {
     spawn(&thread1);
     globalInt = 5; // does thread1 see this change or not?
 }

 It looks like any time you call a delegate member of an immutable  
 object,
 it could access a context pointer that is not implicitly shared.

Not if the compiler cares to check the delegate context is indeed immutable when the immutable delegate is created. It is not much more complicated than if Foo contained an explicit pointer to i instead of a delegate

What if it has no access to the delegate creation code? How does it disallow it?
 Not to mention, what the hell does a const function mean for a delegate
 literal?  The context pointer is the context of the function, not an
 object.  How does that even work?

Internally, it is a pointer to a structure containing the delegate context that is generated by the compiler. If the delegate is const, the pointer should be a const pointer. If the delegate is immutable, it should be an immutable pointer, and the compiler as to check there is no mutable version of that pointer before making an implicit cast.

Hm.. I don't know if there's a way to create such a delegate. I'd have to check.
 What is probably the "right" solution is to disallow implicit immutable
 objects which have a delegate.  This means:

 a) you cannot initialize a shared or immutable such object without a
 cast.  This means the line immutable Foo globalFoo = new Foo(globalInt)  
 is
 an error without a cast.

Agreed, but with my proposal, you can have an immutable Foo, if the delegate context pointer can be cast to immutable.

You can also have an immutable Foo if you cast it to immutable. For example, this line should compile: immutable foo = cast(immutable(Foo))new Foo(globalInt); But then the onus is on you to ensure you are doing the right thing. So in both solutions, it's possible. While I don't think your solution is feasible or solves the problem you set out to solve, if you did find a way to make the compiler make more guarantees, it would be better. I just don't know if it's worth the extra syntax and restrictions.
 b) Any time you have a const object that contains a delegate, it can be
 assumed that the object is not shared.

If the object is const, you have no guarantee that the objet is not shared, making an exception for object containing a delegate is quite wierd.

Well, since you cannot implicitly create such an immutable object, it could be an assumption you can make. However, I can see why it wouldn't be a good idea to make that assumption (especially if you created one via casting). We can probably drop this assumption, because to write code like the above, even if explicitly casting, you are asking for race conditions. Probably safe to assume that it works just like any other const object.
 And then we avoid dealing with the const delegate issue altogether.

Dealing with this issue could be very useful for multithreaded applications.
 With my proposal, you can very easily keep an immutable reference in a
 const delegate. The delegate will just not be callable if its
 function pointer is not const with regard to the context pointer.

You can also very easily keep a mutable reference in a const delegate inside an immutable object. It's not mutable through the delegate, but it's also not immutable.

No, the compiler should check the delegate context is immutable when making the cast to immutable.

As I said above, it's not always possible to check the context. -Steve
Sep 30 2011