www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - mutable keyword

reply ciechowoj <keepitsimplesirius gmail.com> writes:
Is there D equivalent of C++'s mutable keyword? Like the one that 
allows to modify a field of struct from constant method. Or some 
alternative solution?
May 19 2016
next sibling parent QAston <qaston gmail.com> writes:
On Thursday, 19 May 2016 at 20:44:54 UTC, ciechowoj wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one 
 that allows to modify a field of struct from constant method. 
 Or some alternative solution?


There isn't an equivalent of mutable keyword because D const 
differs from C++ const. D const has stronger guarantees (nothing 
reachable from const object will be mutable) so you'll use it 
less often compared to C++.

There's a talk on usage of const in D and I think its author also 
wrote on the subject:
https://www.youtube.com/watch?v=mPr2UspS0fE
May 19 2016
prev sibling next sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 19 May 2016 at 20:44:54 UTC, ciechowoj wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one 
 that allows to modify a field of struct from constant method. 
 Or some alternative solution?


  A little personal experience. I wrote a tool that had certain 
flags that did things turned on by default, and turning them off 
using other flags. This seemed the best choice at the time, but 
upon later reflection it's hard to remember details you'd not 
think about unless you wanted them active. Think about a simple 
action like 'get home', which you also have to 'turn sink off, 
don't change bedding, don't shave, don't walk dog' because 
otherwise you have to do all those when all you planned when 
getting home was taking a shower and going to bed. Remembering to 
add all the flags/changes/options for the utility quickly was 
becoming a pain, to which I reverted my options to only be on 
when called/intended them rather than thinking them on by default 
was the best option.

  As for mutable. It's mutable unless you specify otherwise, and 
the whole object as a whole can be immutable/const; Don't 
complicate the rules so "it's const (except when it's not)" 
logic, either make it intentionally const/immutable or don't.
May 19 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Thursday, May 19, 2016 20:44:54 ciechowoj via Digitalmars-d-learn wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one that
 allows to modify a field of struct from constant method. Or some
 alternative solution?


No. D's const and immutable provide no backdoors. Rather, they provide
strong guarantees. So, if a variable is const, then it cannot be mutated
(even internally) except via a mutable reference to the same data. This has
the upside that you can rely on a const object not mutating on you unless
you're also messing around with mutable referencs to the same data, but it
has the downside that you can't do things like caching or lazy
initialization, and you can't have portions of a const object be treated as
mutable. You can cast away const, but it's undefined behavior if you mutate
afterwards (and could result in subtle bugs depending on what optimizations
the compiler does). So, it's not going to provide a backdoor around const's
strong guarantees about mutation.

The result is that if you need something like C++'s mutable, you can't use
D's const. You're stuck using mutable values and are forced to avoid const.

Now, if your functions aren't pure, you can put state outside of the object
itself and have a const member function access and mutate that external
state, but that's not exactly great for encapsulation, and then you can't
use that function in pure code. But it's the closest thing to a backdoor
from const that exists in D, because const is set up so that it's actually
const and not just const until the implementation decides to mutate it
anyway. Whether that's better or worse than C++'s const depends on what
you're trying to do, but the reality of the matter is that D's const is
ultimately very different from C++'s const because of how restrictive it is.
You get better guarantees but can't use it anywhere near as much precisely
because of the restrictions that are required to provide those guarantees.

- Jonathan M Davis
May 19 2016
next sibling parent reply Namespace <rswhite4 gmail.com> writes:
On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:

 On Thursday, May 19, 2016 20:44:54 ciechowoj via 
 Digitalmars-d-learn wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one 
 that allows to modify a field of struct from constant method. 
 Or some alternative solution?


 No. D's const and immutable provide no backdoors.


But you can cheat:
----
int* _id;

struct A
{
	int id = 0;
	
	this(int id)
	{
		this.id = id;
		_id = &this.id;
	}
	
	void change() const
	{
		(*_id)++;
	}
}

void main() {
	import std.stdio;
	
	A a = A(42);
	a.change();
	
	writeln(a.id);
}
----
May 20 2016
next sibling parent reply Jack Applegame <japplegame gmail.com> writes:
On Friday, 20 May 2016 at 17:28:55 UTC, Namespace wrote:

 But you can cheat:


You can just cast const away:
struct A {
	int id = 0;
	
	this(int id) {
		this.id = id;
	}
	
	void change() const {
		(cast() id)++;
	}
}

void main() {
	import std.stdio;
	
	A a = A(42);
	a.change();
	
	writeln(a.id);
}
May 20 2016
next sibling parent ag0aep6g <anonymous example.com> writes:
On 05/20/2016 08:23 PM, Jack Applegame wrote:

 You can just cast const away:
 struct A {
      int id = 0;

      this(int id) {
          this.id = id;
      }

      void change() const {
          (cast() id)++;
      }
 }


That's not a valid D program, though.
May 20 2016
prev sibling next sibling parent ciechowoj <keepitsimplesirius gmail.com> writes:
On Friday, 20 May 2016 at 18:23:26 UTC, Jack Applegame wrote:

 On Friday, 20 May 2016 at 17:28:55 UTC, Namespace wrote:

 But you can cheat:


 You can just cast const away:
 struct A {
 	int id = 0;
 	
 	this(int id) {
 		this.id = id;
 	}
 	
 	void change() const {
 		(cast() id)++;
 	}
 }

 void main() {
 	import std.stdio;
 	
 	A a = A(42);
 	a.change();
 	
 	writeln(a.id);
 }



Is it valid code (I'm asking about undefined behavior)?
May 20 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, May 20, 2016 18:23:26 Jack Applegame via Digitalmars-d-learn wrote:

 On Friday, 20 May 2016 at 17:28:55 UTC, Namespace wrote:

 But you can cheat:


 You can just cast const away:
 struct A {
   int id = 0;

   this(int id) {
       this.id = id;
   }

   void change() const {
       (cast() id)++;
   }
 }

 void main() {
   import std.stdio;

   A a = A(42);
   a.change();

   writeln(a.id);
 }


Casting away const and mutating is undefined behavior in D. No D program
should ever do it.

- Jonathan M Davis
May 20 2016
parent reply Jack Applegame <japplegame gmail.com> writes:
On Friday, 20 May 2016 at 20:46:18 UTC, Jonathan M Davis wrote:

 Casting away const and mutating is undefined behavior in D. No 
 D program should ever do it.


Really? Mutating immutable is UB too, but look at 
std.typecons.Rebindable.
May 21 2016
next sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Saturday, May 21, 2016 17:15:16 Jack Applegame via Digitalmars-d-learn 
wrote:

 On Friday, 20 May 2016 at 20:46:18 UTC, Jonathan M Davis wrote:

 Casting away const and mutating is undefined behavior in D. No
 D program should ever do it.


 Really? Mutating immutable is UB too, but look at
 std.typecons.Rebindable.


Rebindable is in kind of a weird grey area. It involves a union, not
casting, and it's only ever mutating the class reference, not the object
itself. Certainly, if it mutated the object, it would be undefined behavior,
but it's just mutating the reference - but then again, the type system
itself doesn't really differentiate between the two. So, I suspect that what
it comes down to is that Rebindable is doing something that you really
shouldn't, but because it's using a union rather than a cast, and it's so
limited in what it's mutating, its behavior is effectively defined in the
sense that the compiler has no leeway, but's probably still technically
undefined. I don't know what Walter would say though. Certainly, if casting
were involved, it would be undefined behavior without question, and it's not
the sort of thing that you should be doing in your own code.

So, Rebindable may actually be breaking the rules, but it's so useful that
we really have no other choice as long as there isn't a solution in the
actual language, and it's so limited in how it goes about it, that there's
really no way that it won't work even if it's not technically valid.

Personally, I think that what Rebindable has to do internally to work is the
biggest strike against it, but I doubt that it's enough to convince Walter
that we need a proper in-language solution given how much of a horrid pain
it supposedly is to sort it out in the compiler.

- Jonathan M Davis
May 21 2016
parent reply Jack Applegame <japplegame gmail.com> writes:
On Saturday, 21 May 2016 at 21:49:23 UTC, Jonathan M Davis wrote:

 Rebindable is in kind of a weird grey area. It involves a 
 union, not casting, and it's only ever mutating the class 
 reference, not the object itself. Certainly, if it mutated the 
 object, it would be undefined behavior, but it's just mutating 
 the reference - but then again, the type system itself doesn't 
 really differentiate between the two. So, I suspect that what 
 it comes down to is that Rebindable is doing something that you 
 really shouldn't, but because it's using a union rather than a 
 cast, and it's so limited in what it's mutating, its behavior 
 is effectively defined in the sense that the compiler has no 
 leeway, but's probably still technically undefined. I don't 
 know what Walter would say though. Certainly, if casting were 
 involved, it would be undefined behavior without question, and 
 it's not the sort of thing that you should be doing in your own 
 code.

 [...]


I agree. But I think we need something that allows *logical* 
const and immutable.
Strict binding constness to physical memory constancy is not 
always necessary and sometimes even harmful.
May 22 2016
next sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Sunday, 22 May 2016 at 09:42:54 UTC, Jack Applegame wrote:

 I agree. But I think we need something that allows *logical* 
 const and immutable.
 Strict binding constness to physical memory constancy is not 
 always necessary and sometimes even harmful.


http://wiki.dlang.org/DIP89
May 22 2016
parent chmike <christophe meessen.net> writes:
There is a benefit in not allowing to get pointers to class 
members. It allows to have movable object instances and this give 
access to some faster GC algorithms like generational garbage 
collection which is in use in Java.

As an old C++ programmer and newbee in D programming, the D 
constness and immutability concepts are confusing. They are so 
strong that I hardly see any use for them with objects.

There would be a much wider use to be able to tell user that he 
may assume the object is constant from the interface point of 
view. This is a huge help in code readability and program 
validity checking. I had the impression that this is the 
principle used for the pure keyword.


I would vote against the introduction of the mutable keyword as 
it exist in C++ because it is a half backed solution. First it 
tells the compiler that this variable is modifiable at any time 
by any method of the class. This is way I always felt 
uncomfortable using mutable. It punches a big hole in the 
constness protection.

The other problem is with inheritance (I know it's old school, 
but ok). If a derived class needs to modify a member variable of 
a base class that wasn't declared mutable, you're stuck. That is 
why C++ introduced the const_cast. This is much better in that 
the hole in the constness protection is very limited, but the 
code is also less pleasant to read.

How could be the D way to solve this ? My feeling is that it 
could be by introducing a mutate{...} block. All instructions in 
that block would be allowed to modify the const object. The 
developer intent would be clear, the code readable and the hole 
limited.

The difference with the C++ model is that in C++ we switch off 
the constness flag of member variables for a persistent or short 
time, in D we would switch off constness control in a block of 
instructions.

I didn't thought of all the implications yet.

I only talked about const objects. I still need to find a use 
case for immutable objects /S
May 22 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Sunday, May 22, 2016 09:42:54 Jack Applegame via Digitalmars-d-learn wrote:

 On Saturday, 21 May 2016 at 21:49:23 UTC, Jonathan M Davis wrote:

 Rebindable is in kind of a weird grey area. It involves a
 union, not casting, and it's only ever mutating the class
 reference, not the object itself. Certainly, if it mutated the
 object, it would be undefined behavior, but it's just mutating
 the reference - but then again, the type system itself doesn't
 really differentiate between the two. So, I suspect that what
 it comes down to is that Rebindable is doing something that you
 really shouldn't, but because it's using a union rather than a
 cast, and it's so limited in what it's mutating, its behavior
 is effectively defined in the sense that the compiler has no
 leeway, but's probably still technically undefined. I don't
 know what Walter would say though. Certainly, if casting were
 involved, it would be undefined behavior without question, and
 it's not the sort of thing that you should be doing in your own
 code.

 [...]


 I agree. But I think we need something that allows *logical*
 const and immutable.
 Strict binding constness to physical memory constancy is not
 always necessary and sometimes even harmful.


Actually doing logical const/immutable isn't really possible. Not even C++
does that. Rather, it provides backdoors that you can then use to implement
a sort of logical const, but there is zero guarantee that that's actually
what's happening. It's completely a matter of convention at that point. At
best, it would be possible to indicate that a portion of a type is const or
immutable while allowing certain parts of it to be mutable, in which case,
there are zero constness guarantees about the part that's always mutable and
no guarantees that the stuff that's mutable isn't part of the logical state
of the object.

Given how const and immutable work in D, having any portion of them be
treated as mutable, becomes highly problematic. It could theoretically be
done by having to mark such variables with an attribute and mark such types
with a similar attribute so that the compiler knows that the type in
question is not really following the rules of const or immutable and thus
doesn't make any assumptions about it like it would normally, but it risks
getting rather complicated, and Walter is most definitely not in favor of
such an idea. He is absolutely adamant that const is useless unless it's
fully const with no backdoors whatsoever. So, I'd be very surprised if any
kind of  mutable were added to the language. Certainly, you'd have to have a
very strong technical reason to convince him otherwise, and odds are that
he's just going to tell you to not use const, if it's not going to work for
the type to be const.

- Jonathan M Davis
May 22 2016
parent reply Jack Applegame <japplegame gmail.com> writes:
On Sunday, 22 May 2016 at 13:08:19 UTC, Jonathan M Davis wrote:

 Given how const and immutable work in D, having any portion of 
 them be treated as mutable, becomes highly problematic. It 
 could theoretically be done by having to mark such variables 
 with an attribute and mark such types with a similar attribute 
 so that the compiler knows that the type in question is not 
 really following the rules of const or immutable and thus 
 doesn't make any assumptions about it like it would normally, 
 but it risks getting rather complicated, and Walter is most 
 definitely not in favor of such an idea. He is absolutely 
 adamant that const is useless unless it's fully const with no 
 backdoors whatsoever. So, I'd be very surprised if any kind of 
  mutable were added to the language. Certainly, you'd have to 
 have a very strong technical reason to convince him otherwise, 
 and odds are that he's just going to tell you to not use const, 
 if it's not going to work for the type to be const.


Ha-ha. All this does not prevent immutable class to have mutable 
monitor. Why D so often violates its own rules?
May 23 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Monday, May 23, 2016 08:19:52 Jack Applegame via Digitalmars-d-learn wrote:

 On Sunday, 22 May 2016 at 13:08:19 UTC, Jonathan M Davis wrote:

 Given how const and immutable work in D, having any portion of
 them be treated as mutable, becomes highly problematic. It
 could theoretically be done by having to mark such variables
 with an attribute and mark such types with a similar attribute
 so that the compiler knows that the type in question is not
 really following the rules of const or immutable and thus
 doesn't make any assumptions about it like it would normally,
 but it risks getting rather complicated, and Walter is most
 definitely not in favor of such an idea. He is absolutely
 adamant that const is useless unless it's fully const with no
 backdoors whatsoever. So, I'd be very surprised if any kind of
  mutable were added to the language. Certainly, you'd have to
 have a very strong technical reason to convince him otherwise,
 and odds are that he's just going to tell you to not use const,
 if it's not going to work for the type to be const.


 Ha-ha. All this does not prevent immutable class to have mutable
 monitor. Why D so often violates its own rules?


I believe that the monitor predates immutable by a significant margin, and
aside from compiler bugs, usually if there is any violation of rules going
on it has to do with stuff that comes from D1 that wasn't necessarily
adjusted appropriately as the language progressed, though sometimes folks do
get things wrong, and it makes it into the compiler or std lib. But when
that sort of thing happens, we try and fix it. I fully expect that the
monitor mess will be fixed at some point. It just hasn't been high enough on
the todo list for it to be fully sorted out yet. I don't know why you think
that D violates its own rules frequently though. It's not perfect, but it
usually does follow its own rules - and when it doesn't, we fix it (though
not always as quickly as would be ideal).

- Jonathan M Davis
May 23 2016
parent reply Jack Applegame <japplegame gmail.com> writes:
On Monday, 23 May 2016 at 11:05:34 UTC, Jonathan M Davis wrote:

 I don't know why you think that D violates its own rules 
 frequently though. It's not perfect, but it usually does follow 
 its own rules - and when it doesn't, we fix it (though not 
 always as quickly as would be ideal).


The most PITA for me is lacking mutable references for immutable 
classes. I like immutability and hate Rebindable.
Also, how to embed reference counter into immutable class without 
violating immutability?
May 23 2016
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Monday, May 23, 2016 14:08:43 Jack Applegame via Digitalmars-d-learn wrote:

 On Monday, 23 May 2016 at 11:05:34 UTC, Jonathan M Davis wrote:

 I don't know why you think that D violates its own rules
 frequently though. It's not perfect, but it usually does follow
 its own rules - and when it doesn't, we fix it (though not
 always as quickly as would be ideal).


 The most PITA for me is lacking mutable references for immutable
 classes. I like immutability and hate Rebindable.


Well, Rebindable is kind of ugly, but honestly, a built in modifier probably
wouldn't look much better. e.g.

tail_const(MyClass) obj;

vs

Rebindable!MyClass obj;

The fact that you don't have a * to separate what the const applies to seems
to kill any chance of making it short, and since it's fundamentally part of
the design of D classes that they can't be used separately from their
reference, I don't see that changing even if we get some kind of tail const
built-in to the language for classes.


 Also, how to embed reference counter into immutable class without
 violating immutability?


You can't. It's fundamentally impossible as long as immutable is transitive.
I believe that the proposed solution is to put the ref-count in the memory
next to the object, or at least somewhere else where the memory-management
stuff can find it. In the case of GC-allocated memory, that should be fairly
straightforword, though I don't know how feasible it is to deal with it with
non-GC-allocated memory without losing out on pure. Really, it's going to
come down to how the ref-counting scheme that Walter is working on works.

- Jonathan M Davis
May 23 2016
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 05/21/2016 07:15 PM, Jack Applegame wrote:

 Really? Mutating immutable is UB too, but look at std.typecons.Rebindable.


Rebindable uses a union of a mutable and an immutable variant of the 
type. No access to the mutable union member is provided, and it's never 
dereferenced by Rebindable.

Assignment is done through the mutable member. So the immutable member 
is never overwritten explicitly, but its value does change, of course.

It's supposedly not a problem that the immutable member changes, as long 
as nobody looks. That is, as long as nobody obtains a reference to it. 
Copies of the immutable member are truly immutable.

For this to work, the compiler must recognize that an immutable union 
member isn't actually immutable when the union has mutable members as 
well. I don't know if this is specified anywhere, or if it's perfectly 
sound then.

And Rebindable does give out a reference, actually. That's unacceptable, 
as far as I can see. Filed an issue:
https://issues.dlang.org/show_bug.cgi?id=16054
May 21 2016
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 05/20/2016 10:28 AM, Namespace wrote:

 On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:

 On Thursday, May 19, 2016 20:44:54 ciechowoj via Digitalmars-d-learn
 wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one that
 allows to modify a field of struct from constant method. Or some
 alternative solution?


 No. D's const and immutable provide no backdoors.


 But you can cheat:
 ----
 int* _id;

 struct A
 {
      int id = 0;

      this(int id)
      {
          this.id = id;
          _id = &this.id;


Point taken but considering that D structs are freely movable value 
types, I don't think that's a valid D program. The spec does ban 
self-referencing structs but I think it also bans the code above in 
spirit. :)


      }

      void change() const
      {
          (*_id)++;
      }
 }

 void main() {
      import std.stdio;

      A a = A(42);
      a.change();

      writeln(a.id);
 }
 ----


Ali
May 20 2016
parent Namespace <rswhite4 gmail.com> writes:
On Friday, 20 May 2016 at 18:42:44 UTC, Ali Çehreli wrote:

 On 05/20/2016 10:28 AM, Namespace wrote:

 On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis


 wrote:

 On Thursday, May 19, 2016 20:44:54 ciechowoj via




 Digitalmars-d-learn

 wrote:

 Is there D equivalent of C++'s mutable keyword? Like the






 one that

 allows to modify a field of struct from constant method. Or






 some

 alternative solution?


 No. D's const and immutable provide no backdoors.


 But you can cheat:
 ----
 int* _id;

 struct A
 {
      int id = 0;

      this(int id)
      {
          this.id = id;
          _id = &this.id;


 Point taken but considering that D structs are freely movable 
 value types, I don't think that's a valid D program. The spec 
 does ban self-referencing structs but I think it also bans the 
 code above in spirit. :)


      }

      void change() const
      {
          (*_id)++;
      }
 }

 void main() {
      import std.stdio;

      A a = A(42);
      a.change();

      writeln(a.id);
 }
 ----


 Ali


Also works for classes. ;) It's valid as far as I can tell.
May 20 2016
prev sibling next sibling parent reply ciechowoj <keepitsimplesirius gmail.com> writes:
On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:

 On Thursday, May 19, 2016 20:44:54 ciechowoj via 
 Digitalmars-d-learn wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one 
 that allows to modify a field of struct from constant method. 
 Or some alternative solution?


 Now, if your functions aren't pure, you can put state outside 
 of the object itself and have a const member function access 
 and mutate that external state, but that's not exactly great 
 for encapsulation, and then you can't use that function in pure 
 code. But it's the closest thing to a backdoor from const that 
 exists in D, because const is set up so that it's actually 
 const and not just const until the implementation decides to 
 mutate it anyway. Whether that's better or worse than C++'s 
 const depends on what you're trying to do, but the reality of 
 the matter is that D's const is ultimately very different from 
 C++'s const because of how restrictive it is. You get better 
 guarantees but can't use it anywhere near as much precisely 
 because of the restrictions that are required to provide those 
 guarantees.

 - Jonathan M Davis


Thanks for explanation. It makes implementing things like 
shared_ptr somewhat troublesome when they are supposed to work in 
const environment. Isn't there a way to escape a pure environment 
(like trusted for safe)? Or/and would modifying an external state 
from pure function be an undefined behavior?
May 20 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, May 20, 2016 18:41:04 ciechowoj via Digitalmars-d-learn wrote:

 On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:

 On Thursday, May 19, 2016 20:44:54 ciechowoj via

 Digitalmars-d-learn wrote:

 Is there D equivalent of C++'s mutable keyword? Like the one
 that allows to modify a field of struct from constant method.
 Or some alternative solution?


 Now, if your functions aren't pure, you can put state outside
 of the object itself and have a const member function access
 and mutate that external state, but that's not exactly great
 for encapsulation, and then you can't use that function in pure
 code. But it's the closest thing to a backdoor from const that
 exists in D, because const is set up so that it's actually
 const and not just const until the implementation decides to
 mutate it anyway. Whether that's better or worse than C++'s
 const depends on what you're trying to do, but the reality of
 the matter is that D's const is ultimately very different from
 C++'s const because of how restrictive it is. You get better
 guarantees but can't use it anywhere near as much precisely
 because of the restrictions that are required to provide those
 guarantees.

 - Jonathan M Davis


 Thanks for explanation. It makes implementing things like
 shared_ptr somewhat troublesome when they are supposed to work in
 const environment. Isn't there a way to escape a pure environment
 (like trusted for safe)? Or/and would modifying an external state
 from pure function be an undefined behavior?


There is no backdoor for pure. You can cast it way via function pointer just
like you can cast away const, but it's then undefined behavior if you then
do anything in that function which wouldn't be legal in a pure function.

If you want something that's ref-counted and works in pure code, const will
_not_ work, because you can't legally alter the ref-count.

Now, Walter is currently working on language enhancements to add
ref-counting capabilities to the language, and presumably, that will have a
way around the problem for this particular use case, but when dealing with
const, you need to write your types such that they can work without being
able to mutate anything as long as they're const. And if your design
requires something like the mutable keyword, then you pretty much have to
give up on const. If you abandon pure, you can get around it to some extent,
but you're almost certainly better off if you just abandon trying to use
const if you can't refactor your code such that it doesn't need the mutable
keyword while still playing nicely with pure.

The net result is that while const can be use quite heavily in some types of
D code, it's often the case that it can't be used at all. This is
particularly true if you use templates heavily, since if they're templated
on type, you can't guarantee that the types being used play nicely with
const (at least not without having the template constraints require it) and
therefore usually can't use it. And ranges pretty much don't work with const
at all right now, since they can't be iterated over if they're const, and
there's not currently a way to get a tail-const slice of a range (even
though it works for arrays), so range-heavy code will definitely abandon
const.

So, while const is useful, its use is ultimately fairly limited.

- Jonathan M Davis
May 20 2016
parent reply ciechowoj <keepitsimplesirius gmail.com> writes:
On Friday, 20 May 2016 at 20:45:05 UTC, Jonathan M Davis wrote:

 If you want something that's ref-counted and works in pure 
 code, const will _not_ work, because you can't legally alter 
 the ref-count.


What about something like this (ignoring multi-threading issues):

struct RefCountPool {
    size_t acquireIndex();
    void releaseIndex(size_t index);

    size_t* accessRefCounter(size_t index);
}

RefCountPool refCountPool;

struct SharedPtr
{
    size_t index;
    void* ptr;

    SharedPtr(void* ptr) {
        this.ptr = ptr;
        index = refCountPool.acquireIndex();
    }

    // more methods, counter manipulation through accessRefCounter
}

Is is still legal? Would it breach  pure requirements (I believe 
so...)? After all it doesn't differs much from having a pointer 
instead of index.
May 20 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, May 20, 2016 21:35:27 ciechowoj via Digitalmars-d-learn wrote:

 On Friday, 20 May 2016 at 20:45:05 UTC, Jonathan M Davis wrote:

 If you want something that's ref-counted and works in pure
 code, const will _not_ work, because you can't legally alter
 the ref-count.


 What about something like this (ignoring multi-threading issues):

 struct RefCountPool {
     size_t acquireIndex();
     void releaseIndex(size_t index);

     size_t* accessRefCounter(size_t index);
 }

 RefCountPool refCountPool;

 struct SharedPtr
 {
     size_t index;
     void* ptr;

     SharedPtr(void* ptr) {
         this.ptr = ptr;
         index = refCountPool.acquireIndex();
     }

     // more methods, counter manipulation through accessRefCounter
 }

 Is is still legal? Would it breach  pure requirements (I believe
 so...)? After all it doesn't differs much from having a pointer
 instead of index.


Well, if you actually tried marking functions with pure, you'd see pretty
fast that this won't work with pure. A function that's marked with pure
cannot access any global, mutable state. It can only access what's passed to
it (though in the case of a member function, that includes the this
pointer/reference). So, your refCountPool will not be accessible from any
pure functions.

You can think of pure as  noglobal, because it can't access global variables
(unless they're constants). That's it's only restriction, but it's enough to
make it so that the only way that you'd have a backdoor out of const in a
pure, const member function is if you passed a mutable reference to the
object as one of the function arguments.

At this point, if you want ref-counting, you give up on const. They simply
do not go together. The same goes for stuff like caching or lazy
initialization.

Sure, you can get around const to some extent by giving up on pure, but that
only works because you're putting the state of the object outside of the
object itself, which is usally a bad idea. It also makes it so that const
seems like a lie, since the state of the object isn't really const, since
it's not actually in the object.

The standard library already has std.typecons.RefCounted, if you want to
ref-count anything other than classes, but it really doesn't work with const
and fundamentally can't. In order to have const ref-counting, we're going to
need language support. D does not and likely will never have any form of
"logical" const. If it's const, it's const. Either that fits with what
you're doing, and you can use const, or it doesn't, and you can't.

- Jonathan M Davis
May 20 2016
parent reply ciechowoj <keepitsimplesirius gmail.com> writes:
On Saturday, 21 May 2016 at 00:39:21 UTC, Jonathan M Davis wrote:

 Well, if you actually tried marking functions with pure, you'd 
 see pretty fast that this won't work with pure. A function 
 that's marked with pure cannot access any global, mutable 
 state. It can only access what's passed to it (though in the 
 case of a member function, that includes the this 
 pointer/reference). So, your refCountPool will not be 
 accessible from any pure functions.


Well, I do not have much experience with  pure in D. But I 
believe that as far as I'm not using refCounter-modifying methods 
of sharedPtr (constructors, assignement, destructor) it should 
work. Maybe I'll try it in some future.


 You can think of pure as  noglobal, because it can't access 
 global variables (unless they're constants). That's it's only 
 restriction, but it's enough to make it so that the only way 
 that you'd have a backdoor out of const in a pure, const member 
 function is if you passed a mutable reference to the object as 
 one of the function arguments.

 At this point, if you want ref-counting, you give up on const. 
 They simply do not go together. The same goes for stuff like 
 caching or lazy initialization.

 Sure, you can get around const to some extent by giving up on 
 pure, but that only works because you're putting the state of 
 the object outside of the object itself, which is usally a bad 
 idea. It also makes it so that const seems like a lie, since 
 the state of the object isn't really const, since it's not 
 actually in the object.


I didn't tried the proposed solution, but if this is only 
ideological problem and not a technical one, I would be good with 
such a solution. On one side the memory reachable from object 
isn't modified on the other side the object feels like a const 
for the end-used. I mean I miss a logical const from C++ : ).


 The standard library already has std.typecons.RefCounted, if 
 you want to ref-count anything other than classes, but it 
 really doesn't work with const and fundamentally can't. In 
 order to have const ref-counting, we're going to need language 
 support. D does not and likely will never have any form of 
 "logical" const. If it's const, it's const. Either that fits 
 with what you're doing, and you can use const, or it doesn't, 
 and you can't.


I'm currently doing that, but std.typecons.RefCounted is 
uncomfortable to use. Probably for reasons mentioned above.
May 21 2016
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Saturday, May 21, 2016 07:00:43 ciechowoj via Digitalmars-d-learn wrote:

 On Saturday, 21 May 2016 at 00:39:21 UTC, Jonathan M Davis wrote:

 Well, if you actually tried marking functions with pure, you'd
 see pretty fast that this won't work with pure. A function
 that's marked with pure cannot access any global, mutable
 state. It can only access what's passed to it (though in the
 case of a member function, that includes the this
 pointer/reference). So, your refCountPool will not be
 accessible from any pure functions.


 Well, I do not have much experience with  pure in D. But I
 believe that as far as I'm not using refCounter-modifying methods
 of sharedPtr (constructors, assignement, destructor) it should
 work. Maybe I'll try it in some future.


The problem is the global variable you were using. If the ref-counting is
completely internal, then it can be pure, but it can't access global
variables. I'd suggest that you read this article:

http://klickverbot.at/blog/2012/05/purity-in-d/


 You can think of pure as  noglobal, because it can't access
 global variables (unless they're constants). That's it's only
 restriction, but it's enough to make it so that the only way
 that you'd have a backdoor out of const in a pure, const member
 function is if you passed a mutable reference to the object as
 one of the function arguments.

 At this point, if you want ref-counting, you give up on const.
 They simply do not go together. The same goes for stuff like
 caching or lazy initialization.

 Sure, you can get around const to some extent by giving up on
 pure, but that only works because you're putting the state of
 the object outside of the object itself, which is usally a bad
 idea. It also makes it so that const seems like a lie, since
 the state of the object isn't really const, since it's not
 actually in the object.


 I didn't tried the proposed solution, but if this is only
 ideological problem and not a technical one, I would be good with
 such a solution. On one side the memory reachable from object
 isn't modified on the other side the object feels like a const
 for the end-used. I mean I miss a logical const from C++ : ).


Well, any function that isn't marked with pure is completely unusable in
pure code, and it's generally best practice in D to use pure as much as
possible. It makes it clear that the functions in question can't access
anything that you don't give them, it allows for compiler optimizations in
some cases, and it also makes it easier to do stuff like construct immutable
objects, since if the compiler can guarantee that the return value of a pure
function is unique, it can be implicitly converted to immutable (it might
also be implicitly convertible to shared - I don't recall for sure). And of
course, if you're interacting with D code that requires pure, then your code
is going to need to work with pure.

And since const is already borderline useless for heavily templated code
(which anything range-based tends to be), contorting your code to favor
const over pure is ill-advised.


From what I've seen, pretty much everyone who wants to do stuff like


ref-counting or lazy initialization abandons trying to use const. So, if you
need "mutable," I'd strongly encourage you to do the same rather than trying
to put state in global variables just to have "logical" const, but it's up
to you.


 The standard library already has std.typecons.RefCounted, if
 you want to ref-count anything other than classes, but it
 really doesn't work with const and fundamentally can't. In
 order to have const ref-counting, we're going to need language
 support. D does not and likely will never have any form of
 "logical" const. If it's const, it's const. Either that fits
 with what you're doing, and you can use const, or it doesn't,
 and you can't.


 I'm currently doing that, but std.typecons.RefCounted is
 uncomfortable to use. Probably for reasons mentioned above.


Well, fundamentally, ref-counting and const don't mix. C++ allows it by
basically making it so that C++'s const guarantees nothing. All it really
does is prevent accidental mutation and function as documentation that a
member function isn't supposed to modify its object's state. But nothing is
really guaranteed. D's const provides actual guarantees, but the result is
that it's usuable in far fewer cases, because a lot of code simply doesn't
work if it's forced to actually be const.

So, RefCounted will work on some level, but it really isn't going to work
with const, and no library solution even _can_ play nicely with const - not
without throwing pure out the window anyway, and the really isn't worth it,
especially since we keep getting more stuff added to the language that's
able to take advantage of pure.

But Walter is working on a language solution for ref-counting, and with
that, it should become possible for ref-counting to play nicely with const.

Still, I get the impression that most D programmers use const pretty
limitedly, because trying to use it like you would in C++ simply does not
play nicely with D - especially idiomatic D, which is generally range-based.

- Jonathan M Davis
May 21 2016
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:

 No. D's const and immutable provide no backdoors. Rather, they 
 provide strong guarantees. So, if a variable is const, then it 
 cannot be mutated (even internally) except via a mutable 
 reference to the same data.


The "even internally" is incorrect, and I think needs fixing: 
currently you can synchronize on an immutable/const object, but 
the synchronize implementation will write to __monitor inside the 
object (the field after vptr) and gone is the guarantee. This 
means that you cannot put immutable objects in readonly data, for 
example. We had to make TypeInfo objects mutable in LDC exactly 
because of optimization errors due to this const guarantee 
breakage.
I still have to write up a detailed bug report about it...

- Johan
May 20 2016
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, May 20, 2016 19:33:49 Johan Engelen via Digitalmars-d-learn wrote:

 On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:

 No. D's const and immutable provide no backdoors. Rather, they
 provide strong guarantees. So, if a variable is const, then it
 cannot be mutated (even internally) except via a mutable
 reference to the same data.


 The "even internally" is incorrect, and I think needs fixing:
 currently you can synchronize on an immutable/const object, but
 the synchronize implementation will write to __monitor inside the
 object (the field after vptr) and gone is the guarantee. This
 means that you cannot put immutable objects in readonly data, for
 example. We had to make TypeInfo objects mutable in LDC exactly
 because of optimization errors due to this const guarantee
 breakage.
 I still have to write up a detailed bug report about it...


The whole monitor thing is was a mistake, and I believe that it's pretty
much been agreed upon that we should rip it out, but no one seems to have
actually finished the job yet. IIRC, the proposal was to replace it with an
attribute such that if you used the attribute, then you get more or less
what we have now, but if you don't, then the class in question has no
monitor.

But whenever you have a monitor, you're in pretty much the same boat as with
something like having ref-counted in the language like Andrei wants added,
and you use it with immutable. If you want it to work, then the ref-count
needs to be outside of the object rather than in it, since the memory of the
object has to be treated as immutable. But if that almost certainly means
putting the ref-count (or monitor) next to the object in memory, which would
likely still prevent putting the object in ROM even though the object itself
would then be properly treated as immutable. But at least that would be
opt-in, whereas the monitor mess as it is now is not.

- Jonathan M Davis
May 20 2016