www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Should the "front" range primitive be "const" ?

reply Drone1h <drone1h gmail.com> writes:
Hello all,



I am trying to implement a ("struct template" ? what is the 
correct word ?) range that just forwards its primitives ("empty", 
"front", "popFront") to another range, possibly with some very 
limited filtering/alteration, as std.range.Take (just to learn).

Initially, the "front" member function (property) used to be 
declared "const", but that was not accepted when the underlying 
range (denoted "R" in the code below) was std.stdio.File.ByChunk 
("Error: mutable method std.stdio.File.ByChunk.front is not 
callable using a const object").

Is there any value in having the "front" range primitive declared 
to be a "const"  member function ?

And if so, is the following implementation okay ? Could it be 
further simplified ?

     struct Taker (R)
     {
         private R _r;

         ...

         static if (functionAttributes ! (R.front) & 
FunctionAttribute.const_)
             public  property auto front () const { return 
_r.front; }
         else
             public  property auto front ()       { return 
_r.front; }

         ...
     }



Thank you respectfully !
Drone1h
Jan 29 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, January 30, 2018 01:05:54 Drone1h via Digitalmars-d-learn wrote:

 Hello all,



 I am trying to implement a ("struct template" ? what is the
 correct word ?) range that just forwards its primitives ("empty",
 "front", "popFront") to another range, possibly with some very
 limited filtering/alteration, as std.range.Take (just to learn).

 Initially, the "front" member function (property) used to be
 declared "const", but that was not accepted when the underlying
 range (denoted "R" in the code below) was std.stdio.File.ByChunk
 ("Error: mutable method std.stdio.File.ByChunk.front is not
 callable using a const object").

 Is there any value in having the "front" range primitive declared
 to be a "const"  member function ?

 And if so, is the following implementation okay ? Could it be
 further simplified ?

      struct Taker (R)
      {
          private R _r;

          ...

          static if (functionAttributes ! (R.front) &
 FunctionAttribute.const_)
              public  property auto front () const { return
 _r.front; }
          else
              public  property auto front ()       { return
 _r.front; }

          ...
      }



 Thank you respectfully !
 Drone1h


If you want to put an attribute on it, inout is better, because then it will
work with any constness, but in general, I'd suggest just avoiding the use
of const or immutable altogether when dealing with ranges. front can return
a const element, and that will happen if you use auto and whatever you're
wrapping is const, but const ranges are utterly useless, because they can't
be mutated and thus can't be iterated. As such, almost no code is ever going
to have a range that is anything but mutable, which means that having front
be anything but mutable is generally pointless.

The design of ranges makes them fundamentally incompatible with const. We'd
had to have gone with the head/tail model where you get an entirely new
object every time you pop elements off if we wanted const or immutable to
work. The fact that popFront pretty much outright kills const.

If tail-const slicing were a thing for ranges, then we'd get something
similar to tail/cdr out of the deal, and const ranges could be made to work,
but it's a royal pain to do tail-const with user-defined types (especially
templated types), and slicing an entire range like that isn't actually part
of the range API. hasSlicing requires slicing indices but not the entire
range. Slicing without indices operation is used on containers to get ranges
but isn't used for ranges themselves.

So, as it stands at least, I'd suggest that you simply not bother using
const with ranges.

- Jonathan M Davis
Jan 29 2018
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/29/18 8:20 PM, Jonathan M Davis wrote:

 On Tuesday, January 30, 2018 01:05:54 Drone1h via Digitalmars-d-learn wrote:

 Hello all,



 I am trying to implement a ("struct template" ? what is the
 correct word ?) range that just forwards its primitives ("empty",
 "front", "popFront") to another range, possibly with some very
 limited filtering/alteration, as std.range.Take (just to learn).

 Initially, the "front" member function (property) used to be
 declared "const", but that was not accepted when the underlying
 range (denoted "R" in the code below) was std.stdio.File.ByChunk
 ("Error: mutable method std.stdio.File.ByChunk.front is not
 callable using a const object").

 Is there any value in having the "front" range primitive declared
 to be a "const"  member function ?

 And if so, is the following implementation okay ? Could it be
 further simplified ?

       struct Taker (R)
       {
           private R _r;

           ...

           static if (functionAttributes ! (R.front) &
 FunctionAttribute.const_)
               public  property auto front () const { return
 _r.front; }
           else
               public  property auto front ()       { return
 _r.front; }

           ...
       }



 Thank you respectfully !
 Drone1h


 
 If you want to put an attribute on it, inout is better, because then it will
 work with any constness, but in general, I'd suggest just avoiding the use
 of const or immutable altogether when dealing with ranges. front can return
 a const element, and that will happen if you use auto and whatever you're
 wrapping is const, but const ranges are utterly useless, because they can't
 be mutated and thus can't be iterated. As such, almost no code is ever going
 to have a range that is anything but mutable, which means that having front
 be anything but mutable is generally pointless.


Not necessarily. A main reason for const is to advertise "I'm not going 
to change your mutable data" on a function. So reading that front is 
const (or inout more appropriately) can assure you front makes this 
guarantee.

Yes, it also allows you to call on an immutable or const range, both of 
which are for the most part useless.

So I would say const ranges are useless, but const members of ranges 
provide some value.

That being said, const is viral, as is inout. So unfortunately if you 
*don't* mark your functions const or inout, then wrappers need to take 
this into account.

-Steve
Jan 30 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 30, 2018 at 08:54:00AM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 On 1/29/18 8:20 PM, Jonathan M Davis wrote:


[...]

 If you want to put an attribute on it, inout is better, because then
 it will work with any constness, but in general, I'd suggest just
 avoiding the use of const or immutable altogether when dealing with
 ranges. front can return a const element, and that will happen if
 you use auto and whatever you're wrapping is const, but const ranges
 are utterly useless, because they can't be mutated and thus can't be
 iterated. As such, almost no code is ever going to have a range that
 is anything but mutable, which means that having front be anything
 but mutable is generally pointless.




I think you're conflating a const range, which *is* pretty useless since
you can't iterate it, and a const .front, which only means "calling
.front will not change the state of the range".  The latter is very
possible, and potentially useful.  Well, there's also a .front that
returns a const element, which means "you can't change the current
element of the range". That's also possible, and useful.



 Not necessarily. A main reason for const is to advertise "I'm not
 going to change your mutable data" on a function. So reading that
 front is const (or inout more appropriately) can assure you front
 makes this guarantee.
 
 Yes, it also allows you to call on an immutable or const range, both
 of which are for the most part useless.
 
 So I would say const ranges are useless, but const members of ranges
 provide some value.
 
 That being said, const is viral, as is inout. So unfortunately if you
 *don't* mark your functions const or inout, then wrappers need to take
 this into account.


[...]

Simen has had some ideas recently about "head mutable" aka tail-const,
which could be a first step towards making const ranges, or rather
tail-const ranges, actually usable:

	https://forum.dlang.org/post/cpxfgdmklgusodqouqdr forum.dlang.org

tl;dr summary:

(1) Ranges implement a standard method, tentatively called
    opHeadMutable, that returns a head-mutable version of themselves.
    For example, const(MyRange!T).opHeadMutable would return
    MyRange!(const(T)).

(2) Standard library functions would recognize opHeadMutable and use it
    where needed, e.g., when you hand them a const range.

(3) Profit. :-P


T

-- 
In a world without fences, who needs Windows and Gates? -- Christian Surchi
Jan 30 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, January 30, 2018 07:49:28 H. S. Teoh via Digitalmars-d-learn 
wrote:

 On Tue, Jan 30, 2018 at 08:54:00AM -0500, Steven Schveighoffer via 


Digitalmars-d-learn wrote:

 On 1/29/18 8:20 PM, Jonathan M Davis wrote:


 [...]


 If you want to put an attribute on it, inout is better, because then
 it will work with any constness, but in general, I'd suggest just
 avoiding the use of const or immutable altogether when dealing with
 ranges. front can return a const element, and that will happen if
 you use auto and whatever you're wrapping is const, but const ranges
 are utterly useless, because they can't be mutated and thus can't be
 iterated. As such, almost no code is ever going to have a range that
 is anything but mutable, which means that having front be anything
 but mutable is generally pointless.




 I think you're conflating a const range, which *is* pretty useless since
 you can't iterate it, and a const .front, which only means "calling
 .front will not change the state of the range".  The latter is very
 possible, and potentially useful.  Well, there's also a .front that
 returns a const element, which means "you can't change the current
 element of the range". That's also possible, and useful.


Except that unless front returns by ref, it really doesn't matter whether
front is const unless it's violating the range API, since front is supposed
to return the same value until popFront is called (or if it's assigned a new
value via a front that returns by ref). So, in practice, putting const on
front really doesn't help you any, and it actually hurts you for range
composability.


 Simen has had some ideas recently about "head mutable" aka tail-const,
 which could be a first step towards making const ranges, or rather
 tail-const ranges, actually usable:

   https://forum.dlang.org/post/cpxfgdmklgusodqouqdr forum.dlang.org

 tl;dr summary:

 (1) Ranges implement a standard method, tentatively called
     opHeadMutable, that returns a head-mutable version of themselves.
     For example, const(MyRange!T).opHeadMutable would return
     MyRange!(const(T)).

 (2) Standard library functions would recognize opHeadMutable and use it
     where needed, e.g., when you hand them a const range.

 (3) Profit. :-P


I still need to look over what he's proposing in more detail - it's been
proposed before (by Andrei IIRC) that one possible solution would be to add
an operator for returning a tail-const version of a type, but no one has
ever taken that idea anywhere.

Personally, I'm getting to the point that I'd rather just avoid const than
deal with any further complications for ranges. In principle, I like the
idea of const, but in practice, it just constantly gets in the way, and I've
rarely actually seen any benefit from it in either C++ or D. I can think of
one time in my entire life where const has prevented a bug for me - which
was when I got the arguments backwards to C++'s std::copy function.

At least with immutable, you get implicit sharing and some optimization
opportunities. In principle, const can get you some of the optimization
opportunities but only in really restricted circumstances or circumstances
where you could have used immutable and the code would have been the same
(e.g. with int).

- Jonathan M Davis
Jan 30 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/30/18 8:05 PM, Jonathan M Davis wrote:

 
 Except that unless front returns by ref, it really doesn't matter whether
 front is const unless it's violating the range API, since front is supposed
 to return the same value until popFront is called (or if it's assigned a new
 value via a front that returns by ref). So, in practice, putting const on
 front really doesn't help you any, and it actually hurts you for range
 composability.


Right, but that is the difference between a convention ("front is 
supposed to...") vs. a compiler-enforced guarantee (modifying data by 
calling a const-tagged front is a compiler error).

If you are OK with conventions, you don't need const at all.

-Steve
Jan 31 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, January 31, 2018 11:58:38 Steven Schveighoffer via 
Digitalmars-d-learn wrote:

 On 1/30/18 8:05 PM, Jonathan M Davis wrote:

 Except that unless front returns by ref, it really doesn't matter
 whether
 front is const unless it's violating the range API, since front is
 supposed to return the same value until popFront is called (or if it's
 assigned a new value via a front that returns by ref). So, in practice,
 putting const on front really doesn't help you any, and it actually
 hurts you for range composability.


 Right, but that is the difference between a convention ("front is
 supposed to...") vs. a compiler-enforced guarantee (modifying data by
 calling a const-tagged front is a compiler error).

 If you are OK with conventions, you don't need const at all.


Except that if you're the one writing the function and decided whether it's
const or not, you're also the one deciding whether it returns by ref or not.
Unless you're dealing with a reference type, and it doesn't return by ref,
then const doesn't protect front at all. It just affects whether it can be
called on a const range.

If you're dealing with generic code, then you have less control, and const
starts mattering more, since you don't necessarily know what type is being
returned, and if you're returning front from an underlying range, you the
choice of eixther returning it by value or returning it by auto ref in case
the underlying range returned by ref and passing that refness on is
desirable. But const also interacts far more badly with generic code,
because the odds are pretty high that it won't work in many cases. So, while
in principle, using const to actually have the guarantees is valuable, in
practice, it isn't very viable, because D's const is so restrictive.

Personally, I avoid const in generic code like the plague, because unless
you've restricted the types enough to know what you're dealing with and know
that it will work with const, the odds are quite high that you're writing
code that's going to fall flat on its face with many types.

- Jonathan M Davis
Jan 31 2018
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/31/18 7:49 PM, Jonathan M Davis wrote:

 On Wednesday, January 31, 2018 11:58:38 Steven Schveighoffer via
 Digitalmars-d-learn wrote:

 On 1/30/18 8:05 PM, Jonathan M Davis wrote:

 Except that unless front returns by ref, it really doesn't matter
 whether
 front is const unless it's violating the range API, since front is
 supposed to return the same value until popFront is called (or if it's
 assigned a new value via a front that returns by ref). So, in practice,
 putting const on front really doesn't help you any, and it actually
 hurts you for range composability.


 Right, but that is the difference between a convention ("front is
 supposed to...") vs. a compiler-enforced guarantee (modifying data by
 calling a const-tagged front is a compiler error).

 If you are OK with conventions, you don't need const at all.


 
 Except that if you're the one writing the function and decided whether it's
 const or not, you're also the one deciding whether it returns by ref or not.
 Unless you're dealing with a reference type, and it doesn't return by ref,
 then const doesn't protect front at all. It just affects whether it can be
 called on a const range.


You are misunderstanding here. You don't put const on front for the 
purpose of allowing const ranges (which are useless), what it does is 
say that the compiler guarantees, *even if the range is mutable* that 
front won't modify it.

That is, code like the following is rejected by the compiler:

int front() const { return ++val; }

In other words, it's a contract that you can read without having to 
examine the code saying "this won't mutate the range".

Sure, you can document "front shouldn't modify the range", and use that 
convention, but without const, the compiler doesn't care.


 If you're dealing with generic code, then you have less control, and const
 starts mattering more, since you don't necessarily know what type is being
 returned, and if you're returning front from an underlying range, you the
 choice of eixther returning it by value or returning it by auto ref in case
 the underlying range returned by ref and passing that refness on is
 desirable. But const also interacts far more badly with generic code,
 because the odds are pretty high that it won't work in many cases. So, while
 in principle, using const to actually have the guarantees is valuable, in
 practice, it isn't very viable, because D's const is so restrictive.


Technically, wrapping requires introspection. If you don't care about 
forwarding the "guarantee" of constness, then you can just tag all your 
functions mutable, but if you do care, then you have to introspect.


 Personally, I avoid const in generic code like the plague, because unless
 you've restricted the types enough to know what you're dealing with and know
 that it will work with const, the odds are quite high that you're writing
 code that's going to fall flat on its face with many types.


Indeed, it's not straightforward, if you have to deal with types that 
aren't tagged the way they should be. In addition, const is not inferred 
for templates like other attributes, so you can't rely on that either.

-Steve
Jan 31 2018
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 30, 2018 at 06:05:47PM -0700, Jonathan M Davis via
Digitalmars-d-learn wrote:

 On Tuesday, January 30, 2018 07:49:28 H. S. Teoh via Digitalmars-d-learn 
 wrote:


[...]

 Simen has had some ideas recently about "head mutable" aka
 tail-const, which could be a first step towards making const ranges,
 or rather tail-const ranges, actually usable:

   https://forum.dlang.org/post/cpxfgdmklgusodqouqdr forum.dlang.org

 tl;dr summary:

 (1) Ranges implement a standard method, tentatively called
     opHeadMutable, that returns a head-mutable version of
     themselves.  For example, const(MyRange!T).opHeadMutable would
     return MyRange!(const(T)).

 (2) Standard library functions would recognize opHeadMutable and use
     it where needed, e.g., when you hand them a const range.

 (3) Profit. :-P


 
 I still need to look over what he's proposing in more detail - it's
 been proposed before (by Andrei IIRC) that one possible solution would
 be to add an operator for returning a tail-const version of a type,
 but no one has ever taken that idea anywhere.


Well, that's essentially what Simen has done, and he has code to show
for it.



 Personally, I'm getting to the point that I'd rather just avoid const
 than deal with any further complications for ranges. In principle, I
 like the idea of const, but in practice, it just constantly gets in
 the way, and I've rarely actually seen any benefit from it in either
 C++ or D. I can think of one time in my entire life where const has
 prevented a bug for me - which was when I got the arguments backwards
 to C++'s std::copy function.


I have been saved by const (in D) a few times when I by mistake tried
mutating something that shouldn't be mutated.  But yeah, more often than
not it just gets in the way.  However, my hope is that if Simen's
proposal gets somewhere, it will reduce the annoyance of const and
(hopefully) increase its benefits.

Note that while Simen's code example uses ranges, since that's a common
blocker for using const, it extends beyond that.  For example, consider
a const(RefCounted!Object).  Right now, this is unusable because you
cannot update the reference count of a const object without casting
const away and treading into UB territory.  But if
const(RefCounted!Object).opHeadConst returned a
RefCounted!(const(Object)) instead, then this could be made usable: the
payload can now become const while keeping the reference count mutable.

Of course, as currently designed, the API is kinda awkward. But that's
just a syntactic issue.  We could call it instead .headConst, and you'd
have:

	// mutable refcount, mutable payload
	RefCounted!Object

	// mutable refcount, const payload (useful)
	RefCounted!Object.headConst --> RefCounted!(const(Object))

	// const refcount, const payload (useless)
	const(RefCounted!Object)

Standardizing .headConst means that we now have a reliable way to
construct a RefCounted!(const(Object)) from a RefCounted!Object, whereas
currently we can only construct const(RefCounted!Object), which is
unusable.  In general, this lets us construct a Template!(const(T)) from
a Template!T without needing to know what Template is.

For example, Template could take multiple parameters, like
Template!(x,T), such that the correct head-const is actually
Template!(x, const(T)). Generic code can't know this, but if .headConst
is standardized, then it provides a way for generic code to create a
Template!(x, const(T)) from a Template(x,T) without needing special
knowledge of Template's implementation.

We could even put a generic .headConst in druntime that implements the
conversion for built-in types like int* -> const(int)*.  Then .headConst
becomes the standard idiom to go from any type T to a head-const version
of T.  Generic code that relies on .headConst would work for both
built-in types and custom user types without any change.

Best of all, this doesn't even require a language change, which is a big
plus.



 At least with immutable, you get implicit sharing and some
 optimization opportunities. In principle, const can get you some of
 the optimization opportunities but only in really restricted
 circumstances or circumstances where you could have used immutable and
 the code would have been the same (e.g. with int).


[...]

I haven't thought through it carefully, but if .headConst is a viable
solution to the head-const problem, then conceivably we could also
extend it to deal with immutable payloads too.  Then we could go from,
say, RefCounted!(immutable(T)) to RefCounted!(const(T)) generically,
without any casting or breaking the type system.  We could potentially
expand the scope of usefulness of immutable this way, if this approach
turns out to be workable.


T

-- 
If Java had true garbage collection, most programs would delete themselves upon
execution. -- Robert Sewell
Jan 30 2018
next sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Wednesday, 31 January 2018 at 01:45:57 UTC, H. S. Teoh wrote:

 .headConst


.headMutable. :p Head-const is something we generally want to 
avoid.

--
   Simen
Jan 30 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 31, 2018 at 07:08:58AM +0000, Simen Kjærås via Digitalmars-d-learn
wrote:

 On Wednesday, 31 January 2018 at 01:45:57 UTC, H. S. Teoh wrote:

 .headConst


 
 .headMutable. :p Head-const is something we generally want to avoid.


[...]

*facepalm* Yes, .headMutable, not .headConst. Argh...


T

-- 
VI = Visual Irritation
Jan 31 2018
prev sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Wednesday, 31 January 2018 at 01:45:57 UTC, H. S. Teoh wrote:

 I haven't thought through it carefully, but if .headConst is a 
 viable solution to the head-const problem, then conceivably we 
 could also extend it to deal with immutable payloads too.  Then 
 we could go from, say, RefCounted!(immutable(T)) to 
 RefCounted!(const(T)) generically, without any casting or 
 breaking the type system.  We could potentially expand the 
 scope of usefulness of immutable this way, if this approach 
 turns out to be workable.


Going from RefCounted!(immutable(T)) to RefCounted!(const(T)) is 
nice, but I'd like to see a conversion from, const(RefCounted!T) 
to RefCounted!(const(T)). While this cannot be done without 
casts, the logic can be put inside .headMutable(), and include 
relevant checks. This will make it much safer than having the 
programmer cast manually.

--
   Simen
Jan 31 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 01, 2018 at 07:52:32AM +0000, Simen Kjærås via Digitalmars-d-learn
wrote:

 On Wednesday, 31 January 2018 at 01:45:57 UTC, H. S. Teoh wrote:

 I haven't thought through it carefully, but if .headConst is a
 viable solution to the head-const problem, then conceivably we could
 also extend it to deal with immutable payloads too.  Then we could
 go from, say, RefCounted!(immutable(T)) to RefCounted!(const(T))
 generically, without any casting or breaking the type system.  We
 could potentially expand the scope of usefulness of immutable this
 way, if this approach turns out to be workable.


 
 Going from RefCounted!(immutable(T)) to RefCounted!(const(T)) is nice,
 but I'd like to see a conversion from, const(RefCounted!T) to
 RefCounted!(const(T)). While this cannot be done without casts, the
 logic can be put inside .headMutable(), and include relevant checks.
 This will make it much safer than having the programmer cast manually.


[...]

Hmm. I experimented with this for a bit, and found that if we limit
ourselves to head-mutable, then these casts are inescapable.

The problem is that if we're handed a const object like
const(RefCounted!T), then there's no way we can back out to
RefCounted!(const(T)) without casting, because the latter contains a
mutable refcount whereas the former, due to const transitivity, must be
entirely unmodifiable.  If the refcount were stored in the RefCount
struct itself, then we could get away with a by-value copy into
RefCounted!(const(T)), but unfortunately we can't do that without
breaking the refcounting semantics; the refcount must be on the payload
itself, and RefCounted is basically just a smart pointer. So
const(RefCounted!T), by const transitivity, can only contain a const
pointer to the payload, so we're forced to use a cast to get a
RefCounted!(const(T)) out of it.

And on that note, this casting is NOT safe; for example, if you start
with an immutable(RefCounted!T) and implicitly convert it to
const(RefCounted!T), then if you cast the latter to
RefCounted!(const(T)), you're now violating immutable.  And there's no
way you can tell from inside .headMutable whether it's safe to cast,
because by the time it gets to .headMutable, the original immutable type
is already "forgotten".

However, if we go back to the idea of tail-const, we could potentially
eliminate the need for casts and also avoid breaking immutable.
Basically, the problem with writing const(RefCounted!T) is that it's a
one-way street: on the scale of increasing restrictiveness, we have the
series:

	RefCounted!T < RefCounted!(const(T)) < const(RefCounted!T)

Once you've gone all the way down to const(RefCounted!T), you can no
longer safely back out to RefCounted!(const(T)).

So that means we want to avoid const(RefCounted!T) completely. Instead,
if we standardize on a way to produce a RefCounted!(const(T)) from a
RefCounted!T, then we can stop halfway down the one-way street and never
need to back up.  Let's say we standardize on an operation .tailConst
that does these conversions:

	Container!T.tailConst		--> Container!(const(T))
	Container!(const(T)).tailConst	--> Container!(const(T))
	const(Container!T).tailConst	--> const(Container!T)

	// This is allowed because it's possible to implicitly convert
	// immutable(T) to const(T) internally:
	Container!(immutable(T)).tailConst --> Container!(const(T))

	immutable(Container!T).tailConst --> const(Container!T)

where Container can be any template that might want to support
tail-const semantics.

Essentially, .tailConst becomes the mid-way stand-in for the language's
built-in implicit conversions from mutable/immutable to const. So
instead of passing around const(Container!T), we'd construct a
Container!(const(T)) by calling .tailConst on the original container,
and pass that around instead.

We can also generalize this via UFCS to built-in reference types:

	tailConst(T*)		--> const(T)*
	tailConst((const(T))*)	--> const(T)*
	tailConst(const(T*))	--> const(T*)

	tailConst(immutable(T)*) --> const(T)*
	tailConst(immutable(T*)) --> const(T*)

Then .tailConst becomes a standard way of constructing a tail-const type
in the language.  No explicit language support is needed.


T

-- 
Claiming that your operating system is the best in the world because more
people use it is like saying McDonalds makes the best food in the world. --
Carl B. Constantine
Feb 01 2018
parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Thursday, 1 February 2018 at 18:58:15 UTC, H. S. Teoh wrote:

 However, if we go back to the idea of tail-const, we could 
 potentially eliminate the need for casts and also avoid 
 breaking immutable. Basically, the problem with writing 
 const(RefCounted!T) is that it's a one-way street: on the scale 
 of increasing restrictiveness, we have the series:

 	RefCounted!T < RefCounted!(const(T)) < const(RefCounted!T)

 Once you've gone all the way down to const(RefCounted!T), you 
 can no
 longer safely back out to RefCounted!(const(T)).

 So that means we want to avoid const(RefCounted!T) completely.


I'm not really saying I disagree with that, but it's just not 
realistic. Which code would you rather write?

void foo(T)(const T t) {}
foo(myValue);

or:

void foo(T)(T t) if (isTailConst!T) {}
foo(myValue.tailConst);

The beauty of .headMutable is it generally doesn't affect user 
code. If we have to tell people not to use const(T) because its 
semantics are broken, we've failed.

Now, if at any point in your program you have an 
immutable(RefCounted!T), something's gone horribly wrong - 
basically all of the RefCounted's semantics break down when it's 
immutable, and any attempt at fixing it is undefined behavior. I 
think we can safely disregard the problems of 
immutable(RefCounted!T).

Once we've defined immutable(RefCounted!T) to be undefined 
behavior, suddenly casting from const(RefCounted!T) to 
RefCounted!(const(T)) is OK again.

--
   Simen
Feb 01 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Feb 02, 2018 at 07:06:56AM +0000, Simen Kjærås via Digitalmars-d-learn
wrote:

 On Thursday, 1 February 2018 at 18:58:15 UTC, H. S. Teoh wrote:

 However, if we go back to the idea of tail-const, we could
 potentially eliminate the need for casts and also avoid breaking
 immutable.  Basically, the problem with writing const(RefCounted!T)
 is that it's a one-way street: on the scale of increasing
 restrictiveness, we have the series:
 
 	RefCounted!T < RefCounted!(const(T)) < const(RefCounted!T)
 
 Once you've gone all the way down to const(RefCounted!T), you can no
 longer safely back out to RefCounted!(const(T)).
 
 So that means we want to avoid const(RefCounted!T) completely.


 
 I'm not really saying I disagree with that, but it's just not
 realistic.  Which code would you rather write?
 
 void foo(T)(const T t) {}
 foo(myValue);
 
 or:
 
 void foo(T)(T t) if (isTailConst!T) {}
 foo(myValue.tailConst);
 
 The beauty of .headMutable is it generally doesn't affect user code.
 If we have to tell people not to use const(T) because its semantics
 are broken, we've failed.


Its semantics are not broken; it's just harder to use. Due to const
transitivity, it's an all-or-nothing deal.  .tailConst gives us the
middle ground.



 Now, if at any point in your program you have an
 immutable(RefCounted!T), something's gone horribly wrong - basically
 all of the RefCounted's semantics break down when it's immutable, and
 any attempt at fixing it is undefined behavior. I think we can safely
 disregard the problems of immutable(RefCounted!T).


Immutable may be useless for RefCounted, but I'm thinking of containers
and wrapper types in general. Today, due to ranges being basically
useless when you can't mutate them, you're forced to choose between a
completely mutable range, or no range at all. Having .tailConst as a
standard construction lets you create a usable range that provides a
compiler-verified guarantee that nobody will be able to modify range
elements. Today we don't have a standard way of doing this, and so
people have given up on using const with ranges. We're missing out on
the guarantees that const provides. .tailConst lets us get some of those
guarantees back without requiring us to tie our hands behind our backs.



 Once we've defined immutable(RefCounted!T) to be undefined behavior,
 suddenly casting from const(RefCounted!T) to RefCounted!(const(T)) is
 OK again.


[...]

The problem with this is that we're now relying on convention rather
than something that can be statically verified by the compiler. Once you
allow casting away const, there's no longer a guarantee that somebody
didn't pass in an immutable, whether by mistake or otherwise. We know
from C/C++ where programming by convention leads us. :-P


T

-- 
Having a smoking section in a restaurant is like having a peeing section in a
swimming pool. -- Edward Burr
Feb 02 2018
next sibling parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Friday, 2 February 2018 at 14:29:34 UTC, H. S. Teoh wrote:

 Its semantics are not broken; it's just harder to use. Due to 
 const transitivity, it's an all-or-nothing deal.  .tailConst 
 gives us the middle ground.


If the semantics of const means that users will have to write 
.tailConst all over the place, it's broken. If it means that 
users can't use const(T) because they actually want TailConst!T, 
it's broken.

TailConst seems like the logical solution to the problem, but it 
isn't. It directly impacts user code and it leads to lots of 
boilerplate. In addition to .tailConst, we also need 
.tailImmutable. And to top it off, it just doesn't mix with const 
at all - if you pass it as a const parameter, it's broken. If 
it's part of a struct or class with const methods, it's broken. 
It infects every part of your codebase that touches it, it forces 
you to basically implement your own const system in templates, 
and it makes const even harder to use than it currently is.

Tail-const is the more intuitive way to think of it, so if I'm 
wrong, please show me.



 Once we've defined immutable(RefCounted!T) to be undefined 
 behavior,
 suddenly casting from const(RefCounted!T) to 
 RefCounted!(const(T)) is
 OK again.


 [...]

 The problem with this is that we're now relying on convention 
 rather than something that can be statically verified by the 
 compiler. Once you allow casting away const, there's no longer 
 a guarantee that somebody didn't pass in an immutable, whether 
 by mistake or otherwise. We know from C/C++ where programming 
 by convention leads us. :-P


True. Sadly, there's no way to tell the type system 'this type 
should never be immutable'. Maybe such a thing should be in the 
language. Meanwhile, if RefCounted!T implements .headMutable, it 
can check at runtime that the refcount is in writable memory.



 Though the above currently doesn't compile, it seems because 
 the compiler doesn't know how to resolve Wrapper!(const(T)) 
 given a Wrapper!T despite the alias this.


Yeah, I found the same bug when playing with .headMutable:
https://issues.dlang.org/show_bug.cgi?id=18260

--
   Simen
Feb 04 2018
prev sibling parent Seb <seb wilzba.ch> writes:
On Friday, 2 February 2018 at 14:29:34 UTC, H. S. Teoh wrote:

 On Fri, Feb 02, 2018 at 07:06:56AM +0000, Simen Kjærås via 
 Digitalmars-d-learn wrote:

 [...]


 Its semantics are not broken; it's just harder to use. Due to 
 const transitivity, it's an all-or-nothing deal.  .tailConst 
 gives us the middle ground.

 [...]


FWIW there's also head const in phobos since more than one year - 
known as Final:

https://dlang.org/phobos/std_experimental_typecons.html
Mar 19 2018
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Feb 02, 2018 at 07:06:56AM +0000, Simen Kjærås via Digitalmars-d-learn
wrote:
[...]

 Which code would you rather write?
 
 void foo(T)(const T t) {}
 foo(myValue);
 
 or:
 
 void foo(T)(T t) if (isTailConst!T) {}
 foo(myValue.tailConst);


[...]

More thoughts on this: what if we made it so that Wrapper!T is
implicitly convertible to Wrapper!(const T)? Something like this:

	auto foo(Wrapper,T)(Wrapper!(const(T)) t) ...

	struct Wrapper(T) {
		auto tailConst() {
			return Wrapper!(const(T))(...);
		}
		alias tailConst this;
	}

	void main() {
		Wrapper!int w;
		foo(w);
	}

It will interact badly if Wrapper is already using alias this for
something else, but this lets us keep the convenience of implicitly
decaying to tail-const without requiring an explicit call to .tailConst.

Though the above currently doesn't compile, it seems because the
compiler doesn't know how to resolve Wrapper!(const(T)) given a
Wrapper!T despite the alias this. However, removing `Wrapper` from the
template arguments of foo() does work; and seems to have the right
semantics.  Seems to be just a limitation of IFTI.  So calling a
function that expects, say, RefCounted!(const T), with an argument of
type RefCounted!T can already be made to work today, if we tie
.tailConst to alias this.

The fully general solution will have to wait until we can improve IFTI
to support the generic case where the wrapper type is also a template
parameter.  Might be worth filing an enhancement against the compiler to
support this?

If this can be made to work, we may not even need a standard name for
.tailConst, as long as the wrapper type can somehow make itself decay
into its tail-const version implicitly.


T

-- 
In theory, there is no difference between theory and practice.
Feb 02 2018
prev sibling parent reply Drone1h <drone1h gmail.com> writes:
First of all, thank you all for the replies. It has taken me some 
time to learn a bit more to be able to understand at least some 
parts of them. I have a further question below the quotes.

On Tuesday, 30 January 2018 at 01:20:09 UTC, Jonathan M Davis 
wrote:

 On Tuesday, January 30, 2018 01:05:54 Drone1h via 
 Digitalmars-d-learn wrote:



 [...]
 I am trying to implement a ("struct template" ? what is the
 correct word ?) range that just forwards its primitives 
 ("empty",
 "front", "popFront") to another range, possibly with some very
 limited filtering/alteration, as std.range.Take (just to 
 learn).

 Initially, the "front" member function (property) used to be
 declared "const", but that was not accepted when the underlying
 range (denoted "R" in the code below) was 
 std.stdio.File.ByChunk
 ("Error: mutable method std.stdio.File.ByChunk.front is not
 callable using a const object").
 [...]





 If you want to put an attribute on it, inout is better, because 
 then it will work with any constness


I am not sure whether I can make it work with "inout" instead of 
"const". Perhaps I am missing something.

Here is my code:

     auto Take2 (R) (R r, size_t n)
     {
         import std.range.primitives;

         struct Result
         {
         private:
             R      _r;
             size_t _n;
             size_t _i;

         public:
              property bool empty () const
             {
                 return _i >= _n || _r.empty;
             }

              property auto ref front () const // Replace "const" 
with "inout" here ?
             {
                 return _r.front;
             }

             void popFront ()
             {
                 ++_i;
                 if (_i < _n)
                     _r.popFront ();
             }

             this (R r, size_t n)
             {
                 _r = r;
                 _n = n;
                 _i = 0;
             }
         }

         return Result (r, n);
     }

     unittest
     {
         import std.stdio;

         auto file = File ("Example", "rb");
         foreach (c; file.byChunk (0x100).Take2 (5))
             stdout.rawWrite (c);
     }

I get compile error:

     Error: mutable method `std.stdio.File.ByChunkImpl.front` is 
not callable using a `const` object

If I replace "const" with "inout" for the "front" function, i.e. 
" property auto ref front () inout", I get similar error:

     Error: mutable method `std.stdio.File.ByChunkImpl.front` is 
not callable using a `inout` object

May I ask that you confirm that this is what you suggested ? 
Thank you.




 [...] const ranges are utterly useless, because they can't be 
 mutated and thus can't be iterated. [...]


I am considering a function that takes as parameter a (reference 
to) const range. It should be able to check whether the range is 
"empty" and, if not empty, it should be able to access the 
"front" element.

Now the caller of that function can rest assured that the 
function is not going to modify the range by "popFront".

The function might be more useful than a function that just takes 
as parameter a value (the result of "front") because it may be 
called with an empty range and it can detect that. This is 
similar to getting as parameter a (possibly null) pointer to a 
value instead of getting as parameter a value.

Therefore, it seems to me that a const range might be useful.

If you have already considered this and have actually seen 
one-step ahead of me, may I ask that you confirm, please ?



Thank you respectfully.
Mar 18 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, March 19, 2018 00:14:11 Drone1h via Digitalmars-d-learn wrote:

 I am not sure whether I can make it work with "inout" instead of
 "const". Perhaps I am missing something.


...

 May I ask that you confirm that this is what you suggested ?
 Thank you.


Marking a empty or front with inout has most of the problems that const has.
The difference is that if the range is mutable, then the return value will
be treated as mutable. The internals of the function, however, must still
work with const or inout, and that's not true for most ranges. It can work
just fine to mark empty or front as inout or const if you're in full control
of the element types and aren't wrapping other ranges, but as soon as you
start wrapping other ranges, you pretty much must give up on inout and
const, because most ranges won't work with them - even to just call empty or
front. And in many cases, they can't be made to work with const or inout,
because that often causes serious problems with the return type. The range
API simply does not include const or inout, so you can't assume that any
range will compile with them, meaning that ranges that wrap other ranges can
only use const or inout when they're intended to wrap a very specific set of
ranges that do work with const or inout. Ranges that generically wrap other
ranges cannot use const or inout, or they will not compile with many (most?)
ranges.


 [...] const ranges are utterly useless, because they can't be
 mutated and thus can't be iterated. [...]


 I am considering a function that takes as parameter a (reference
 to) const range. It should be able to check whether the range is
 "empty" and, if not empty, it should be able to access the
 "front" element.

 Now the caller of that function can rest assured that the
 function is not going to modify the range by "popFront".

 The function might be more useful than a function that just takes
 as parameter a value (the result of "front") because it may be
 called with an empty range and it can detect that. This is
 similar to getting as parameter a (possibly null) pointer to a
 value instead of getting as parameter a value.

 Therefore, it seems to me that a const range might be useful.

 If you have already considered this and have actually seen
 one-step ahead of me, may I ask that you confirm, please ?


Most ranges do not work with const in any way shape or form. _Some_ will
work if all you're doing is looking at is empty or front. But if all you're
doing is looking at the front, in general, why pass a range? Just call front
and pass it, and then the function will work with more than just ranges.
Yes, if you want a function that does something like

auto frontOrInit(R)(R range)
{
    return range.empty ? ElementType!R.init : range.front;
}

then you can make the range const, but in general, a function is either
going to be iterating over the range (so the range can't be const), or what
the function is doing really has nothing to do with ranges and would be far
more flexible if it just took the range's element type. Functions like
frontOrInit where it would make sense to only call front or empty on a range
are rare. And honestly, making a function like frontOrInit accept the range
by const doesn't buy you much if a range's front and empty are const, and it
makes the function useless with most ranges, because most ranges don't - and
many can't - mark front or empty as const.

Honestly, I think that you will be far better off if you just don't try and
use const or inout with ranges. You can make it work in very restricted
circumstances, but you will constantly be fighting problems where code does
not compile. I'd suggest that you read this:

http://jmdavisprog.com/articles/why-const-sucks.html

- Jonathan M Davis
Mar 18 2018
next sibling parent Drone1h <drone1h gmail.com> writes:
On Monday, 19 March 2018 at 00:50:06 UTC, Jonathan M Davis wrote:

 On Monday, March 19, 2018 00:14:11 Drone1h via 
 Digitalmars-d-learn wrote:

 I am not sure whether I can make it work with "inout" instead 
 of "const". Perhaps I am missing something.


 ...

 May I ask that you confirm that this is what you suggested ? 
 Thank you.


 Marking a empty or front with inout has most of the problems 
 that const has. The difference is that [...]


Thank you again for your kind replies. I will carefully read 
during the following days.
Mar 23 2018
prev sibling parent reply Drone1h <drone1h gmail.com> writes:
On Monday, 19 March 2018 at 00:50:06 UTC, Jonathan M Davis wrote:


 [...]
 http://jmdavisprog.com/articles/why-const-sucks.html


I have just read the reply and the article.

I cannot believe you have written this article in response to 
this thread (perhaps I am mis-interpreting the date of the 
article). But even if not so, thank you for clarifications and 
for describing root causes.

I will have to read it again, and then read the other replies to 
my question, because I am just beginning to understand what 
"head-const" and "tail-const" mean and to get an idea of the 
problems.

By this reply, I just wish to let you all know that your answers 
are really appreciated and that I believe they are very helpful 
for many programmers, even if they need time to understand them. 
Thank you Jonathan. Thank you all.
Apr 09 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, April 09, 2018 23:58:02 Drone1h via Digitalmars-d-learn wrote:

 On Monday, 19 March 2018 at 00:50:06 UTC, Jonathan M Davis wrote:

 [...]
 http://jmdavisprog.com/articles/why-const-sucks.html


 I have just read the reply and the article.

 I cannot believe you have written this article in response to
 this thread (perhaps I am mis-interpreting the date of the
 article). But even if not so, thank you for clarifications and
 for describing root causes.


I did not write the article in response to any thread. I'd been meaning to
write it for a while. It's just that it's relevant to this thread, and
pointing to the article rather than trying to explain everything that's in
there again saves me time.

- Jonathan M Davis
Apr 09 2018