www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - =?UTF-8?B?Q29tbWFuZOKAk3F1ZXJ5?= separation principle [re: mustuse as

reply mw <mingwu gmail.com> writes:
Following the discussion from:
--------------------------------------------
https://forum.dlang.org/post/kyonizwffgvpuvrwhoog forum.dlang.org

On Wednesday, 19 October 2022 at 01:49:26 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 01:38:27 UTC, Adam D Ruppe 
 wrote:

 On Wednesday, 19 October 2022 at 01:34:54 UTC, mw wrote:

 Is there any (design) doc about this?


 scroll up, click the link from this very thread.

 https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1038.md#design-goals-and-possible-alternatives


 """
 Rather than make either sacrifice, this DIP proposes a design 
 that allows both rigor and simplicity to be maintained, and 
 reserves the possibility for a future DIP to allow  mustUse as 
 a function attribute.
 """

 Another future DIP? I think this DIP is flawed for not using 
 " mustUse as a function attribute"


 mustuse as a function attribute was in the original version of 
the DIP. It was vetoed by Walter. Thus, only the type attribute 
remains in the accepted version.


added originally as a type attribute only, and later had its 
usage extended to functions.
--------------------------------------------


In case anyone does not know, I'd like to refer to the following: 
command query separation principle

https://en.wikipedia.org/wiki/Command%E2%80%93query_separation#:~:text=Command-query%20separation%20(CQS),the%20caller%2C%20but%20not%20

"""
It states that every method should either be a command that 
performs an action, or a query that returns data to the caller, 
but not both.
"""

For a query, its return value should never be disregarded, this 
is forced by the Eiffel compiler.

The kind of problem (wasting hours of development time, 
frustration and pain) the OP's author has experienced would have 
never happened if s/he is using Eiffel.


 Walter, you haven't study Eiffel well enough? although you said 
you have Meyer's book?
Oct 18 2022
next sibling parent reply mw <mingwu gmail.com> writes:
D does not strictly follow DbC, and has many legacy features from 
Java/C++ world, and many libraries does not follow command query 
separation principle at all. Also there is no compiler 
enforcement. That's the current status, we have to accept that.

In theory:

paths.remove(i);  // is just a command, should not return 
anything, and the `paths` object itself should have been 
modified, which is the OP author's expectation.

However, for whatever reason (implementation performance concern 
maybe?) the `remove()` method right now returns(!) a new object, 
which actually is the new `paths` object with the updated state, 
so the current correct way to use it is:

paths = paths.remove(i); // works - what I erroneously thought 
the previous line was doing


So the DIP to add  mustuse as a function attribute is a remedy to 
D's not being strictly follow DbC and the command query 
separation principle. At least it forces the programmers do not 
ignore the important return value as in the above example.

I really cannot see why  mustuse as a function attribute got 
rejected.

DbC is a seemingly simple concept, but actually it's deeper than 
you often thought, it affects many details of language design.
Oct 18 2022
next sibling parent zjh <fqbqrr 163.com> writes:
On Wednesday, 19 October 2022 at 05:41:19 UTC, mw wrote:


 I really cannot see why  mustuse as a function attribute got 
 rejected.

 DbC is a seemingly simple concept, but actually it's deeper 
 than you often thought, it affects many details of language 
 design.



There is also `class level private`.
Oct 18 2022
prev sibling parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 05:41:19 UTC, mw wrote:

 DbC is a seemingly simple concept, but actually it's deeper 
 than you often thought, it affects many details of language 
 design.


OK, an informal exercise: derive command query separation 
principle from DbC.

The contract in DbC mostly exhibits as assertions in the code.

The programmer can insert assertions at any place of the code, 
without changing the code's original semantics (i.e the behavior 
when the assertions are turned-off, e.g. in release mode). In an 
OOP language, most of the time the assertions are checking some 
properties of an object, hence any method that can be called in 
an assertion must be a query (i.e a query can be called on an 
object for any number times without changing that object's 
internal state). So now we have query method.

But we do need to change an object's state in imperative 
programming, then those methods are classified as commands. After 
changing an object state, the command must NOT return any value. 
Why? because otherwise, the programmer may accidentally want to 
call that command and check the returned value in some assertions 
... then you know what happens in the end: the program behaves 
differently when assertions are turn on in debug mode and off in 
release mode.

Therefore, we have this:
"""
every method should either be a command that performs an action, 
or a query that returns data to the caller, but not both.
"""
Oct 18 2022
prev sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Wednesday, 19 October 2022 at 04:52:31 UTC, mw wrote:


  mustuse as a function attribute was in the original version of 
 the DIP. It was vetoed by Walter. Thus, only the type attribute 
 remains in the accepted version.


Please include the entire context that I posted earliar from the 
summary of the formal assessment here:

https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1038.md#formal-assessment

Walter didn't just willy-nilly "veto" the attribute on functions. 
He accepted the proposal with three requests for enhancement, one 
of which I summarized as:


 develop rules for handling covariance and contravariance when 
 applied to functions.


Paul opted instead to punt on this and restrict the attribute to 
types, which was a perfectly reasonable thing to do. Note that 
the DIP was still accepted.

Walter doesn't *not* want the attribute to apply to functions. 
It's just that he wants it to be more fully specified. He and 
Paul had a series of emails on this and other aspects of the DIP.

If anyone is willing to take this and flesh it out further so 
that it meets Walter's criteria for applying to functions, please 
let me know. It's pretty much guaranteed to be approved.
Oct 19 2022
parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 07:28:21 UTC, Mike Parker wrote:

 On Wednesday, 19 October 2022 at 04:52:31 UTC, mw wrote:


  mustuse as a function attribute was in the original version 
 of the DIP. It was vetoed by Walter. Thus, only the type 
 attribute remains in the accepted version.


 Please include the entire context that I posted earliar from 
 the summary of the formal assessment here:

 https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1038.md#formal-assessment

 Walter didn't just willy-nilly "veto" the attribute on 
 functions. He accepted the proposal with three requests for 
 enhancement, one of which I summarized as:


 develop rules for handling covariance and contravariance when 
 applied to functions.




Covariance and contravariance is a big topic by itself.
But ..., what does it have anything to do here with  mustuse as a 
function attribute?

 mustuse here only enforces there must be a receiver of the 
function call result, and do not throw away the returned result.

I'd really like to see some examples what's this concern is about.



 Paul opted instead to punt on this and restrict the attribute 
 to types, which was a perfectly reasonable thing to do. Note 
 that the DIP was still accepted.

 Walter doesn't *not* want the attribute to apply to functions. 
 It's just that he wants it to be more fully specified. He and 
 Paul had a series of emails on this and other aspects of the 
 DIP.

 If anyone is willing to take this and flesh it out further so 
 that it meets Walter's criteria for applying to functions, 
 please let me know. It's pretty much guaranteed to be approved.


I wish I have more time, but I just have so many things to do 
these days.
Oct 19 2022
next sibling parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 07:41:33 UTC, mw wrote:

 If anyone is willing to take this and flesh it out further so 
 that it meets Walter's criteria for applying to functions, 
 please let me know. It's pretty much guaranteed to be approved.


 I wish I have more time, but I just have so many things to do 
 these days.


But if anyone want to work on it, I'd happy to help.
Oct 19 2022
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Wednesday, 19 October 2022 at 07:41:33 UTC, mw wrote:

 Covariance and contravariance is a big topic by itself.
 But ..., what does it have anything to do here with  mustuse as 
 a function attribute?

  mustuse here only enforces there must be a receiver of the 
 function call result, and do not throw away the returned result.

 I'd really like to see some examples what's this concern is 
 about.


Following are the examples Walter provided in his email 
discussions with Paul:

Covariance Ex. 1:

```d
class C {
   T h();
}
class D : C {
   override  mustuse T h();
}
```

Ex. 2
```d
class C {
    mustuse T h();
}
class D : C {
   override T h();
}
```

Contravariance Ex. 1:
```d
class C {
    void t(T function()  mustuse *);
}
class D {
    override void t(T function() *);
}
```

Ex 2:
```d
class C {
    void t(T function() *);
}
class D {
    override void t(T function()  mustuse *);
}
```

The rules for what happens in each case need to be clearly 
defined. Walter proposed the following:

====
To establish the rule, just resolve this simple case:

```d
T f();
 mustuse T g();
```

Can g be implicitly coerced into the type of f? No.
Can f be implicitly coerced into the type of g? Yes.

I.e.  mustuse can be added in an implicit type conversion, but it 
cannot be
subtracted. That's all we need to know, everything else follows 
inevitably.
====

In which case, both Examples 1 would fail, and both Examples 2 
would pass.

There was more to their discussion beyond this, though, and I'm 
not the person to summarize it.

So again, if anyone wants to take a stab at fleshing it out, 
please let me know. I can ask Paul and Walter to provide their 
thoughts to help get started.
Oct 19 2022
parent reply mw <mingwu gmail.com> writes:
OK, I saw what they are talking about.


Basically, in Eiffel, all query  mustuse.

Here in D  mustuse is an remedy add-on, which can be specified by 
the programmers in the middle of the inheritance tree, the 
question is what if the pointers got casted either upwards or 
downwards?


(It's too late for me here today.)


On Wednesday, 19 October 2022 at 08:15:32 UTC, Mike Parker wrote:

 On Wednesday, 19 October 2022 at 07:41:33 UTC, mw wrote:

 Covariance and contravariance is a big topic by itself.
 But ..., what does it have anything to do here with  mustuse 
 as a function attribute?

  mustuse here only enforces there must be a receiver of the 
 function call result, and do not throw away the returned 
 result.

 I'd really like to see some examples what's this concern is 
 about.


 Following are the examples Walter provided in his email 
 discussions with Paul:

 Covariance Ex. 1:

 ```d
 class C {
   T h();
 }
 class D : C {
   override  mustuse T h();
 }
 ```

 Ex. 2
 ```d
 class C {
    mustuse T h();
 }
 class D : C {
   override T h();
 }
 ```

 Contravariance Ex. 1:
 ```d
 class C {
    void t(T function()  mustuse *);
 }
 class D {
    override void t(T function() *);
 }
 ```

 Ex 2:
 ```d
 class C {
    void t(T function() *);
 }
 class D {
    override void t(T function()  mustuse *);
 }
 ```

 The rules for what happens in each case need to be clearly 
 defined. Walter proposed the following:

 ====
 To establish the rule, just resolve this simple case:

 ```d
 T f();
  mustuse T g();
 ```

 Can g be implicitly coerced into the type of f? No.
 Can f be implicitly coerced into the type of g? Yes.

 I.e.  mustuse can be added in an implicit type conversion, but 
 it cannot be
 subtracted. That's all we need to know, everything else follows 
 inevitably.
 ====

 In which case, both Examples 1 would fail, and both Examples 2 
 would pass.

 There was more to their discussion beyond this, though, and I'm 
 not the person to summarize it.

 So again, if anyone wants to take a stab at fleshing it out, 
 please let me know. I can ask Paul and Walter to provide their 
 thoughts to help get started.
Oct 19 2022
parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 08:29:20 UTC, mw wrote:

 OK, I saw what they are talking about.


 Basically, in Eiffel, all query  mustuse.

 Here in D  mustuse is an remedy add-on, which can be specified 
 by the programmers in the middle of the inheritance tree, the 
 question is what if the pointers got casted either upwards or 
 downwards?

 (It's too late for me here today.)


My gut feeling is: make it simple, if a method is marked as 
 mustuse anywhere in the inheritance tree, that method in all the 
class levels of the inheritance tree became  mustuse!
Oct 19 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 08:35:21 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 08:29:20 UTC, mw wrote:

 OK, I saw what they are talking about.


 Basically, in Eiffel, all query  mustuse.

 Here in D  mustuse is an remedy add-on, which can be specified 
 by the programmers in the middle of the inheritance tree, the 
 question is what if the pointers got casted either upwards or 
 downwards?

 (It's too late for me here today.)


 My gut feeling is: make it simple, if a method is marked as 
  mustuse anywhere in the inheritance tree, that method in all 
 the class levels of the inheritance tree became  mustuse!


The correct rule is exactly what Mike quoted from my email 
discussion with Walter: implicit conversions may add ` mustuse`, 
but can never remove it.

 From this, it follows that

* a ` mustuse` method cannot override a non-` mustuse` method.
* a ` mustuse` class/interface cannot inherit from a 
non-` mustuse` class/interface.

In other words, you cannot introduce ` mustuse` in a derived 
class; it must be present in the base class.

This is somewhat problematic for D, because we have a universal 
base class, `Object`, and neither it nor any of its methods are 
` mustuse`.
Oct 19 2022
next sibling parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 13:17:00 UTC, Paul Backus wrote:

 On Wednesday, 19 October 2022 at 08:35:21 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 08:29:20 UTC, mw wrote:

 OK, I saw what they are talking about.


 Basically, in Eiffel, all query  mustuse.

 Here in D  mustuse is an remedy add-on, which can be 
 specified by the programmers in the middle of the inheritance 
 tree, the question is what if the pointers got casted either 
 upwards or downwards?

 (It's too late for me here today.)


 My gut feeling is: make it simple, if a method is marked as 
  mustuse anywhere in the inheritance tree, that method in all 
 the class levels of the inheritance tree became  mustuse!


 The correct rule is exactly what Mike quoted from my email 
 discussion with Walter: implicit conversions may add 
 ` mustuse`, but can never remove it.

 From this, it follows that

 * a ` mustuse` method cannot override a non-` mustuse` method.
 * a ` mustuse` class/interface cannot inherit from a 
 non-` mustuse` class/interface.

 In other words, you cannot introduce ` mustuse` in a derived 
 class; it must be present in the base class.


Let's concentrate on only  mustuse as a function attribute for 
now, since I have not checked the exact semantics of   mustuse as 
a type annotation.


There are two directions to propagate the attribute in the 
inheritance tree, upwards and downwards. We all agree on the 
direction of downwards propagation to the derived class methods, 
I.e in your expression: "may add ` mustuse`, but can never remove 
it".

Now I only need to convince you the other direction to upwards is 
also needed:

It's actually quite simple, let us consider the original example 
which started this discussion, and do this exercise:



class AbstractPaths {
   AbstractPaths remove (int I) {...}
}


class Paths : AbstractPaths {
    mustuse
   AbstractPaths remove (int I) {...}
}


AbstractPaths absPaths = new Paths();

absPaths.remove(i);  // shall  mustuse be enforced here on 
absPaths?


Let's step back, and ask why we want to introduce  mustuse in the 
beginning?
The purpose of the annotation is to help programmers do not 
discard important return values accidentally as what the OP's 
author has experienced disaster.

Then the answer is very clear: we do want  mustuse be enforced on 
the absPaths.remove(i) call in the above example!

Otherwise, it will be a loophole in the  mustuse as a function 
attribute logic, which defeat its purpose.

Convinced?


And as a consequence, all the AbstractPaths' derived classes' 
remove(i) method now must have this annotation ( injected by the 
D compiler, the programmer does not have to manually add it in 
all the places). That is why I say:

if a method is marked as  mustuse anywhere in the inheritance 
tree, that method in all the class levels in the whole 
inheritance tree became  mustuse!
Oct 19 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 20/10/2022 5:31 AM, mw wrote:

 if a method is marked as  mustuse anywhere in the inheritance tree, that 
 method in all the class levels in the whole inheritance tree became 
  mustuse!


Unfortunately people don't compile their program all at once.

So therefore this won't work.

I considered something along these lines, but its ultimately error prone 
to try to force an attribute on for a class hierarchy (including not 
allowing casts to parents).
Oct 19 2022
parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 16:37:51 UTC, rikki cattermole 
wrote:

 On 20/10/2022 5:31 AM, mw wrote:

 if a method is marked as  mustuse anywhere in the inheritance 
 tree, that method in all the class levels in the whole 
 inheritance tree became  mustuse!


 Unfortunately people don't compile their program all at once.


No, that's a limitation of current D compiler, or the build 
system dub.

Yes, for this logic to work, every compilation need to do global 
system analysis (that is what Eiffel compiler does).

Now the compiler writer have to work harder to implement this 
logic. Do not compromise!




 So therefore this won't work.

 I considered something along these lines, but its ultimately 
 error prone to try to force an attribute on for a class 
 hierarchy (including not allowing casts to parents).
Oct 19 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 20/10/2022 5:42 AM, mw wrote:

 On Wednesday, 19 October 2022 at 16:37:51 UTC, rikki cattermole wrote:

 On 20/10/2022 5:31 AM, mw wrote:

 if a method is marked as  mustuse anywhere in the inheritance tree, 
 that method in all the class levels in the whole inheritance tree 
 became  mustuse!


 Unfortunately people don't compile their program all at once.


 
 No, that's a limitation of current D compiler, or the build system dub.


Or Meson
Or CMake
Or make...
Or cli script


 Yes, for this logic to work, every compilation need to do global system 
 analysis (that is what Eiffel compiler does).


But not for native compilers. They work in partial builds, this is core 
to how they work.


 Now the compiler writer have to work harder to implement this logic. Do 
 not compromise!


No amount of work is going to change the fact that the compiler may not 
be aware that code exists.

rt package one example.

Shared libraries another.
Oct 19 2022
parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 16:47:45 UTC, rikki cattermole 
wrote:

 Yes, for this logic to work, every compilation need to do 
 global system analysis (that is what Eiffel compiler does).


 But not for native compilers. They work in partial builds, this 
 is core to how they work.


Eiffel is a native compiler, and they do support incremental 
build. They have a compilation technology called melting ice:

https://www.eiffel.org/doc/eiffelstudio/Melting_Ice_Technology

(In the past, I'm more a language theorist than a language 
implementator, so I didn't study the details of it.)


 Now the compiler writer have to work harder to implement this 
 logic. Do not compromise!


 No amount of work is going to change the fact that the compiler 
 may not be aware that code exists.

 rt package one example.

 Shared libraries another.


Yes pre-built binaries is a concern, but since  mustuse now is a 
half-way introduced remedy (rather than D doing DbC right from 
the very beginning), I think we only need to enforce  mustuse on 
the source code that the compiler actually visit during each 
compilation, that will eliminate the problems the programmer has 
control with. (for pre-built binaries that the programmer have no 
control, there is no way to apply the enforcement either).
Oct 19 2022
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Wed, Oct 19, 2022 at 04:31:59PM +0000, mw via Digitalmars-d wrote:
[...]

 class AbstractPaths {
   AbstractPaths remove (int I) {...}
 }
 
 
 class Paths : AbstractPaths {
    mustuse
   AbstractPaths remove (int I) {...}
 }
 
 
 AbstractPaths absPaths = new Paths();
 
 absPaths.remove(i);  // shall  mustuse be enforced here on absPaths?


No, because it is not enforceable.  Consider:

	class MyPaths : AbstractPaths {
		// N.B.: no  mustuse
		override AbstractPaths remove (int I) {...}
	}

	AbstractPaths absPaths = new MyPaths;
	absPaths.remove(i); // shall  mustuse be enforced?

That is, given an instance of AbstractPaths, *you cannot tell* whether
 mustuse should be enforced or not. If it's an instance of Paths, then
it must be enforced, but if it's an instance of MyPaths, then it must
not be.  Conclusion: we cannot enforce  mustuse in the base class
AbstractPaths.



 Let's step back, and ask why we want to introduce  mustuse in the
 beginning?  The purpose of the annotation is to help programmers do
 not discard important return values accidentally as what the OP's
 author has experienced disaster.
 
 Then the answer is very clear: we do want  mustuse be enforced on the
 absPaths.remove(i) call in the above example!


No, the correct answer is that the base class method must also be
attributed with  mustuse.  It should be illegal to have a non- mustuse
base class method overridden by a  mustuse derived class method.

Why?  Consider the following scenarios:

1) Base class has  mustuse, derived class does not.  Then the base
class's  mustuse can be circumvented by a derived class that omits the
attribute, defeating the purpose of  mustuse in the base class.

2) Base class does not have  mustuse, but derived class does.  Then
instances of the derived class, cast to the base class references,
allow bypassing  mustuse in the derived class.

If indeed  mustuse's purpose is to prevent accidentally discarding
important return values, i.e., something marked  mustuse must *never* be
silently dropped, then neither (1) nor (2) is acceptable.

Conclusion: if the base class method has  mustuse, then so must the
derived class method. If the base class method does not have  mustuse,
then the derived class method cannot have it either.


[...]

 And as a consequence, all the AbstractPaths' derived classes'
 remove(i) method now must have this annotation ( injected by the D
 compiler, the programmer does not have to manually add it in all the
 places).


This is not enforceable, since the base class and derived class could be
in two far-flung files, and due to incremental compilation the compiler
cannot enforce  mustuse in an unmarked method in a class that might have
some distant relative in the inheritance tree that has  mustuse.

The only way this can be done is to make it a compile error for a base
class method and a derived class method to have a mismatch in the
 mustuse attribute.



 That is why I say:
 
 if a method is marked as  mustuse anywhere in the inheritance tree,
 that method in all the class levels in the whole inheritance tree
 became  mustuse!


This is correct.  But it cannot be implicit; if you attribute a method
 mustuse somewhere in the inheritance tree, then you must also write
 mustuse in every other class in the hierarchy. Otherwise it's not
enforceable.


T

-- 
Winners never quit, quitters never win. But those who never quit AND never win
are idiots.
Oct 19 2022
parent reply mw <mingwu gmail.com> writes:
Right now in this discussion thread there are two aspects:
1) the language logic, and
2) the implementation logic (and current status, including 
pre-built binaries).

I'm not interested in 2). We need to first make the language 
logic correct.


 If indeed  mustuse's purpose is to prevent accidentally 
 discarding
 important return values, i.e., something marked  mustuse must 
 *never* be
 silently dropped, then neither (1) nor (2) is acceptable.


That's because you still consider the  mustuse attribute on each 
class level individually, if you think about the global system 
analysis as a whole, it will work, and consistently. Yes, what I 
proposed is transitive closure:

     /--------Base------\
     |        |         |
Derived1   Derive2  Derive3

If the programmer only manually marked Derive3.remove() as 
 mustuse, everything else (Base, Derived1   Derive2  Derive3)'s 
.remove() method will become  mustuse (as seen by the compiler 
internally).



 This is not enforceable, since the base class and derived class 
 could be in two far-flung files, and due to incremental 
 compilation the compiler cannot enforce  mustuse in an unmarked 
 method in a class that might have some distant relative in the 
 inheritance tree that has  mustuse.


(all these comments are compiler implementation issues (2), which 
I don't want to comment)



 That is why I say:
 
 if a method is marked as  mustuse anywhere in the inheritance 
 tree, that method in all the class levels in the whole 
 inheritance tree became  mustuse!


 This is correct.  But it cannot be implicit; if you attribute a 
 method  mustuse somewhere in the inheritance tree, then you 
 must also write  mustuse in every other class in the hierarchy. 
 Otherwise it's not enforceable.


I'm glad you agree with my global analysis as a whole view here.

(I want the compiler to propagate and inject such attribute is 
only to save the programmers manual work, by no means it means 
it's implicit.

As to the compiler implementation, I'll let Walter figure it out).
Oct 19 2022
next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Wed, Oct 19, 2022 at 05:40:00PM +0000, mw via Digitalmars-d wrote:

 Right now in this discussion thread there are two aspects:
 1) the language logic, and
 2) the implementation logic (and current status, including pre-built
 binaries).
 
 I'm not interested in 2). We need to first make the language logic correct.
 

 If indeed  mustuse's purpose is to prevent accidentally discarding
 important return values, i.e., something marked  mustuse must
 *never* be silently dropped, then neither (1) nor (2) is acceptable.


 
 That's because you still consider the  mustuse attribute on each class
 level individually, if you think about the global system analysis as a
 whole, it will work, and consistently. Yes, what I proposed is
 transitive closure:
 
     /--------Base------\
     |        |         |
 Derived1   Derive2  Derive3
 
 If the programmer only manually marked Derive3.remove() as  mustuse,
 everything else (Base, Derived1   Derive2  Derive3)'s .remove() method
 will become  mustuse (as seen by the compiler internally).


This means Derived1.remove and Derive2.remove are implicitly  mustuse.
This is bad design; a programmer looking at the code for Derived1 cannot
be expected to know that it is  mustuse.  It introduces a discrepancy
between the surface source code vs. its semantics. Or, put another way,
Derived1.remove has an "invisible"  mustuse that isn't represented in
the source code, and that the programmer cannot possibly know about
unless he looks at *all* instances of .remove in the entire class
hierarchy.

The compiler must enforce that Derived1.remove and Derive2.remove are
explicitly marked  mustuse.  IOW, it must be a compile error for some
instances of .remove to be  mustuse and others not.


[...]

 That is why I say:
 
 if a method is marked as  mustuse anywhere in the inheritance
 tree, that method in all the class levels in the whole inheritance
 tree became  mustuse!


 
 This is correct.  But it cannot be implicit; if you attribute a
 method  mustuse somewhere in the inheritance tree, then you must
 also write  mustuse in every other class in the hierarchy. Otherwise
 it's not enforceable.


 
 I'm glad you agree with my global analysis as a whole view here.
 
 (I want the compiler to propagate and inject such attribute is only to
 save the programmers manual work, by no means it means it's implicit.


Of course it does.  How is it not implicit when Derived1.remove is *not*
marked  mustuse in the source code, but is treated by the compiler as
 mustuse?  The fact that what's in the source code doesn't correspond to
the compiler's internal representation, is the definition of "implicit".

It saves a few keystrokes but introduces long-distance implicit
relationships between distant pieces of code (i.e., Derive3.remove being
marked  mustuse implicitly causes Derived1.remove to be  mustuse, but
the latter is not represented in the source code). This makes the code
hard to understand and reduces maintainability.

Rather, the correct approach should be that all instances of .remove
should be *explicitly* marked  mustuse in the source code. It should be
a compile error for some instances of .remove to be marked  mustuse and
others not marked.


T

-- 
Computerese Irregular Verb Conjugation: I have preferences.  You have biases. 
He/She has prejudices. -- Gene Wirchenko
Oct 19 2022
parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 18:01:42 UTC, H. S. Teoh wrote:

 On Wed, Oct 19, 2022 at 05:40:00PM +0000, mw via Digitalmars-d 
 wrote:
 It saves a few keystrokes but introduces long-distance implicit 
 relationships between distant pieces of code (i.e., 
 Derive3.remove being marked  mustuse implicitly causes 
 Derived1.remove to be  mustuse, but the latter is not 
 represented in the source code). This makes the code hard to 
 understand and reduces maintainability.

 Rather, the correct approach should be that all instances of 
 .remove should be *explicitly* marked  mustuse in the source 
 code. It should be a compile error for some instances of 
 .remove to be marked  mustuse and others not marked.


Conceptually I agree  about the *explicitly* part. My only 
concern is that as in Paul Backus' example the classes are in 
separate dub packages that the programmer has *no* control with, 
e.g. the programmer decided to derived from one of the unmarked 
std.lib class, and added  mustuse mark to a method in his derived 
class, s/he will not be able to manually change the std.lib class 
source code to add  mustuse explicitly.

But for the  mustuse logic to be correct, the compiler need to 
internally propagate to the base std.lib classes.

And yes, I know this global analysis view will have conflict with 
the current compilation & build process.
Oct 19 2022
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 17:40:00 UTC, mw wrote:

 That's because you still consider the  mustuse attribute on 
 each class level individually, if you think about the global 
 system analysis as a whole, it will work, and consistently. 
 Yes, what I proposed is transitive closure:

     /--------Base------\
     |        |         |
 Derived1   Derive2  Derive3

 If the programmer only manually marked Derive3.remove() as 
  mustuse, everything else (Base, Derived1   Derive2  Derive3)'s 
 .remove() method will become  mustuse (as seen by the compiler 
 internally).


The fundamental problem with this on a conceptual level (leaving 
aside implementation issues) is that it completely breaks 
modularity.

Suppose we have the following module layout:

--- base.d
module base;

class Base {
     int fun() {...}
}

void doStuff(Base b)
{
     import std.stdio;

     writeln("Calling b.fun");
     b.fun(); // ok - Base.fun is not  mustuse
}

--- derived.d
module derived;

import base;

class Derived : Base {
     int fun() {...}
}
---

If I now decide to add  mustuse to Derived.fun, in the "derived" 
module, and we apply your proposed global analysis, this will 
cause a compilation error in the "doStuff" function in the "base" 
module!

Note that the "base" module does not have any explicit dependency 
on the "derived" module. It does not import it, or otherwise 
refer to it in any way. In the real world, these two modules 
might even be in separate dub packages.
Oct 19 2022
parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 18:04:54 UTC, Paul Backus wrote:

 On Wednesday, 19 October 2022 at 17:40:00 UTC, mw wrote:

 That's because you still consider the  mustuse attribute on 
 each class level individually, if you think about the global 
 system analysis as a whole, it will work, and consistently. 
 Yes, what I proposed is transitive closure:

     /--------Base------\
     |        |         |
 Derived1   Derive2  Derive3

 If the programmer only manually marked Derive3.remove() as 
  mustuse, everything else (Base, Derived1   Derive2  
 Derive3)'s .remove() method will become  mustuse (as seen by 
 the compiler internally).


 The fundamental problem with this on a conceptual level 
 (leaving aside implementation issues) is that it completely 
 breaks modularity.

 Suppose we have the following module layout:

 --- base.d
 module base;

 class Base {
     int fun() {...}
 }

 void doStuff(Base b)
 {
     import std.stdio;

     writeln("Calling b.fun");
     b.fun(); // ok - Base.fun is not  mustuse
 }

 --- derived.d
 module derived;

 import base;

 class Derived : Base {
     int fun() {...}
 }
 ---

 If I now decide to add  mustuse to Derived.fun, in the 
 "derived" module, and we apply your proposed global analysis, 
 this will cause a compilation error in the "doStuff" function 
 in the "base" module!

 Note that the "base" module does not have any explicit 
 dependency on the "derived" module. It does not import it, or 
 otherwise refer to it in any way. In the real world, these two 
 modules might even be in separate dub packages.


Yes, I know that. But this in my view is still a compiler 
implementation issue: even in separate dub packages, as long as 
the compiler visit that package, it need to propagate the 
attribute to that package's relevant class.
Oct 19 2022
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 18:16:39 UTC, mw wrote:

 Yes, I know that. But this in my view is still a compiler 
 implementation issue: even in separate dub packages, as long as 
 the compiler visit that package, it need to propagate the 
 attribute to that package's relevant class.


My point is that, even if we assume it is possible to implement, 
it is still a bad idea, because it would introduce implicit 
coupling between modules where no coupling (explicit or implicit) 
previously existed.

I hope I do not have to explain to you why implicit coupling is a 
bad thing in software development.
Oct 19 2022
next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Wed, Oct 19, 2022 at 06:25:54PM +0000, Paul Backus via Digitalmars-d wrote:

 On Wednesday, 19 October 2022 at 18:16:39 UTC, mw wrote:

 Yes, I know that. But this in my view is still a compiler
 implementation issue: even in separate dub packages, as long as the
 compiler visit that package, it need to propagate the attribute to
 that package's relevant class.


 
 My point is that, even if we assume it is possible to implement, it is
 still a bad idea, because it would introduce implicit coupling between
 modules where no coupling (explicit or implicit) previously existed.
 
 I hope I do not have to explain to you why implicit coupling is a bad
 thing in software development.


If we really wanted to enforce  mustuse across dub packages, the
solution is to include it in the mangled type. That will force a link
error when methods don't match up.

Of course, changing mangling will also cause breakage of existing code.
:-D


T

-- 
Век живи - век учись. А дураком помрёшь.
Oct 19 2022
prev sibling next sibling parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 18:25:54 UTC, Paul Backus wrote:

 On Wednesday, 19 October 2022 at 18:16:39 UTC, mw wrote:

 Yes, I know that. But this in my view is still a compiler 
 implementation issue: even in separate dub packages, as long 
 as the compiler visit that package, it need to propagate the 
 attribute to that package's relevant class.


 My point is that, even if we assume it is possible to 
 implement, it is still a bad idea, because it would introduce 
 implicit coupling between modules where no coupling (explicit 
 or implicit) previously existed.


That's because now we want to add  mustuse half-way, and it 
requires global system analysis. If D started from scratch as 
Eiffel did, and enforces all query methods are  mustuse from the 
very beginning, such problem and remedy headache would never 
existed.

So now we need to balance either we want prevent accidentally 
(but fatal in most cases) discarding function returns, or we want 
prevent more coupling between modules.

For me, in this particular situation, I prefer the former which I 
think will make the code more robust.
Oct 19 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 18:38:55 UTC, mw wrote:

 That's because now we want to add  mustuse half-way, and it 
 requires global system analysis.


It does not require global system analysis.


 If D started from scratch as Eiffel did, and enforces all query 
 methods are  mustuse from the very beginning, such problem and 
 remedy headache would never existed.


Sure. It's the same with other attributes, like  safe and nothrow.


 So now we need to balance either we want prevent accidentally 
 (but fatal in most cases) discarding function returns, or we 
 want prevent more coupling between modules.


It is possible to achieve both. If the compiler enforces the 
rules I laid out in my initial post [1] (i.e., inheritance can 
only remove  mustuse, not add it), then accidentally discarding 
the return value of a  mustuse function will be impossible, *and* 
there will be no additional coupling between modules.

Of course, this means that there will be some functions that can 
never be marked as  mustuse without a breaking change--just like 
with  safe and nothrow. But that's true even with your proposal. 
Global analysis does not prevent the addition of  mustuse from 
breaking code, it just increases the number of places where that 
breaking change can be made.

[1] 
https://forum.dlang.org/post/cqlwlnpcbtbkzqnhicwc forum.dlang.org
Oct 19 2022
parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 18:56:35 UTC, Paul Backus wrote:

 On Wednesday, 19 October 2022 at 18:38:55 UTC, mw wrote:

 It is possible to achieve both. If the compiler enforces the 
 rules I laid out in my initial post [1] (i.e., inheritance can 
 only remove  mustuse, not add it),


This won't work as Teoh has showed the loophole:

1) Base class has  mustuse, derived class does not.  Then the 
base class's  mustuse can be circumvented by a derived class that 
omits the attribute, defeating the purpose of  mustuse in the 
base class.



 Of course, this means that there will be some functions that 
 can never be marked as  mustuse without a breaking change--just 
 like with  safe and nothrow. But that's true even with your 
 proposal.


In any case it will be a breaking change, I never said otherwise; 
and it's not of my concern. My only concern is how to make the 
ideal D code more robust.
Oct 19 2022
prev sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 19 October 2022 at 18:25:54 UTC, Paul Backus wrote:

 because it would introduce implicit coupling between modules 
 where no coupling (explicit or implicit) previously existed.


How is this any different than inheritance of attributes like 
 safe, which exists now?

(i would argue the difference is that applies to the parameter 
and this applies to the return value, and you can normally 
strengthen params and weaken return contracts so it is kinda 
backward. but that's separate from the coupling thing... kinda, i 
guess the other difference is safe and friends are of most 
restriction to the method implementor, so if it is inherited the 
author of the class just deals with it where mustUse would be of 
most restriction to the method caller. but i gotta run to a 
meeting and haven't thought it all through)
Oct 19 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 18:45:40 UTC, Adam D Ruppe wrote:

 On Wednesday, 19 October 2022 at 18:25:54 UTC, Paul Backus 
 wrote:

 because it would introduce implicit coupling between modules 
 where no coupling (explicit or implicit) previously existed.


 How is this any different than inheritance of attributes like 
  safe, which exists now?


The difference between  mustuse and  safe is that adding  safe 
imposes additional restrictions on the *function*, but adding 
 mustuse imposes additional restrictions on the *calling code*.

Another way to think of it is:  safe is like an "out" contract, 
and  mustuse is like an "in" contract.

Derived classes are allowed to weaken in contracts and strengthen 
out contracts, but not the reverse. By the same logic, derived 
classes are allowed to remove  mustuse and add  safe, but not the 
reverse.
Oct 19 2022
next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Wed, Oct 19, 2022 at 06:50:41PM +0000, Paul Backus via Digitalmars-d wrote:
[...]

 The difference between  mustuse and  safe is that adding  safe imposes
 additional restrictions on the *function*, but adding  mustuse imposes
 additional restrictions on the *calling code*.
 
 Another way to think of it is:  safe is like an "out" contract, and
  mustuse is like an "in" contract.
 
 Derived classes are allowed to weaken in contracts and strengthen out
 contracts, but not the reverse. By the same logic, derived classes are
 allowed to remove  mustuse and add  safe, but not the reverse.


Hmm, this actually makes a lot of sense.

If a base class method Base.method has  mustuse but the derived class
method Derived.method doesn't, that's not a problem: callers who hold a
Base reference to the derived instance will respect  mustuse, but
Derived.method doesn't care.  Conversely, you can only cast a Base to
Derived if it's actually an instance of Derived, so calling .method
afterwards without respecting  mustuse doesn't break anything (this does
not allow you to circumvent  mustuse on AnotherDerived.method).

So yes,  mustuse propagates up the class hierarchy, but not necessarily
down.


T

-- 
He who does not appreciate the beauty of language is not worthy to bemoan its
flaws.
Oct 19 2022
prev sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 19 October 2022 at 18:50:41 UTC, Paul Backus wrote:

 Another way to think of it is:  safe is like an "out" contract, 
 and  mustuse is like an "in" contract.


ok yeah, that's exactly what i had in mind, i was just in a rush 
(made it to my meeting with 20s to spare! lol). I think this 
makes sense and is a useful framework for answering all the other 
questions.
Oct 19 2022
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 10/19/22 11:16, mw wrote:


 If I now decide to add  mustuse to Derived.fun, in the "derived"
 module, and we apply your proposed global analysis, this will cause a
 compilation error in the "doStuff" function in the "base" module!

 Note that the "base" module does not have any explicit dependency on
 the "derived" module. It does not import it, or otherwise refer to it
 in any way. In the real world, these two modules might even be in
 separate dub packages.


 Yes, I know that. But this in my view is still a compiler implementation
 issue:


It is a human issue: Let's assume twenty developers have been using an 
Animal hierarchy. One day, Bertrand decides to add  mustuse to the new 
class Giraffe, a decendent of Animal. Now the code is broken all over 
the place.

Twenty developers chase Bertrand in and around the building until he/she 
removes  mustuse. (Commenting-out is acceptable as well.) Very human 
indeed... :o)


 even in separate dub packages, as long as the compiler visit that
 package, it need to propagate the attribute to that package's relevant
 class.


The compiler does not and should not do anything like that. I can 
imagine the specification for compilers finding source code (of 
potentially pre-compiled libraries) and visiting all their source code 
would be very complicated and very different from  mustuse.

We started  mustuse as a simple concept of "this value must be used" and 
ended up with coming up with a whole new compilation system. Not 
practical... :)

All aside, I agree with the fact that  mustuse should somehow be 
per-function. If it were left to me, I would make it the default... 
which would annoy even myself because it would make quick-and-dirty 
prototype test code unnecessarily noisy. Perhaps a compiler switch to 
set the default behavior would work.

Ali
Oct 19 2022
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 16:31:59 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 13:17:00 UTC, Paul Backus 
 wrote:

 From this, it follows that

 * a ` mustuse` method cannot override a non-` mustuse` method.
 * a ` mustuse` class/interface cannot inherit from a 
 non-` mustuse` class/interface.

 In other words, you cannot introduce ` mustuse` in a derived 
 class; it must be present in the base class.


 Let's concentrate on only  mustuse as a function attribute for 
 now, since I have not checked the exact semantics of   mustuse 
 as a type annotation.


 There are two directions to propagate the attribute in the 
 inheritance tree, upwards and downwards. We all agree on the 
 direction of downwards propagation to the derived class 
 methods, I.e in your expression: "may add ` mustuse`, but can 
 never remove it".


You have this completely backwards. A derived class may 
**remove** ` mustuse`, but can never **add** it.

In your example, attempting to add ` mustuse` to `Paths.remove` 
would result in a compile-time error:

```d
class AbstractPaths {
   AbstractPaths remove (int I) {...}
}


class Paths : AbstractPaths {
    mustuse
   AbstractPaths remove (int I) {...}
   // Error:  mustuse method cannot override method without 
 mustuse
}
```
Oct 19 2022
parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 17:46:15 UTC, Paul Backus wrote:

 You have this completely backwards. A derived class may 
 **remove** ` mustuse`, but can never **add** it.


look like I didn't get a good sleep last night :-)

As to the two propagation directions, if we analyzed it 
individually, H. S. Teoh's post has covered both the scenarios.


But I believe you got my point: we need global system propagation 
here as I showed in the previous post:

if you think about the global system analysis as a whole, it will 
work, and consistently. Yes, what I proposed is transitive 
closure:

     /--------Base------\
     |        |         |
Derived1   Derive2  Derive3
     |        |         |
GrandDr1   DrandRr2 GradDr3
     |
    ....

If the programmer *only* manually marked Derive3.remove() as 
 mustuse, everything else (Base, Derived1   Derive2  Derive3, 
GrandDr1   DrandRr2 GradDr3, ...
)'s .remove() method will become  mustuse (as seen by the 
compiler internally).
Oct 19 2022
prev sibling parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 13:17:00 UTC, Paul Backus wrote:

 In other words, you cannot introduce ` mustuse` in a derived 
 class; it must be present in the base class.


I just realized that we two have different design motivations / 
goals: my goal is allow programmers to add  mustuse annotation to 
*any* methods that return values, and then make the whole thing 
work (to simulate Eiffel behavior to the max extent). So I need 
global analysis.

While your goal is to add  mustuse to method in such a way, which 
won't break current system (or with minimal impact), if the 
impact is too big, e.g conflict with prebuilt binary, then 
constraint the annotation in such a way that some of the 
annotations are not allowed to avoid the conflicts.



 This is somewhat problematic for D, because we have a universal 
 base class, `Object`, and neither it nor any of its methods are 
 ` mustuse`.


This is not a concern/problem at all, esp in your design goal, 
because your rule will prohibit adding such annotation to 
existing methods, e.g. Object.opEquals(); and for the methods 
that programmers do care, e.g. Paths.remove(i), there is no such 
method in Object.
Oct 19 2022
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 19 October 2022 at 21:13:00 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 13:17:00 UTC, Paul Backus 
 wrote:

 In other words, you cannot introduce ` mustuse` in a derived 
 class; it must be present in the base class.


 I just realized that we two have different design motivations / 
 goals: my goal is allow programmers to add  mustuse annotation 
 to *any* methods that return values, and then make the whole 
 thing work (to simulate Eiffel behavior to the max extent). So 
 I need global analysis.

 While your goal is to add  mustuse to method in such a way, 
 which won't break current system (or with minimal impact), if 
 the impact is too big, e.g conflict with prebuilt binary, then 
 constraint the annotation in such a way that some of the 
 annotations are not allowed to avoid the conflicts.


The only problem with your goal is that Walter and Atila would 
never have accepted a proposal that actually achieves it. :)

If you are designing your own language, you can do whatever you 
want, but if you are contributing to an existing language, you 
have to work within the limits of what the project leaders will 
allow.

My goal was to make  mustuse as useful as possible while (a) 
remaining inside those limits, and (b) keeping it simple enough 
that I could implement it on my own.
Oct 19 2022
prev sibling parent reply mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 21:13:00 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 13:17:00 UTC, Paul Backus 
 wrote:

 In other words, you cannot introduce ` mustuse` in a derived 
 class; it must be present in the base class.




I spent some more time thinking about this, and I do not agree 
with this rule even though I know what your design goals (and 
constraints) are.

I will make an improved proposal based on my initial transitive 
closure design, and I will write a longer post about it, which 
may take some time, please bear with me.

As a summary, this is what I am going to propose: introduce two 
variants of  mustuse

1)  mustUse_remedyLegacy: this annotation is to allow programmer 
flag existing legacy library code that s/he has no right to 
change, but want to set a flag and let the compiler to help to 
find violations, but the compiler only output warnings instead of 
errors.

2)  mustuse: proper, the default. For programmer to use in new 
code, or library code s/he can change (from the root of the 
inheritance tree). Violations of this annotation will cause 
compiler error.

And:

a) in both cases, this function property will be transitive 
closure, I.e. be propagated upwards and downwards, in all 
directions.

b) in both cases, removing the annotation is *not* allowed in the 
derived class if the supper class method carry such annotation.


I will write more about my rationale and considerations, and give 
examples when I get more time.
Oct 22 2022
parent mw <mingwu gmail.com> writes:
Finally I got some time to write this draft:

https://github.com/mw66/mustuse/blob/main/mustuse.md

which is copy & paste-ed below. Feel free to comment here, or log 
an issue on github.

I will update, or address concerns by revising the doc there.


--------------------------------------------------------------------------


There are two scenarios we need to handle:

- existing legacy library code, which the programmer has no right 
to modify
- new code, which the programmer has full control from the root 
of the inheritance tree


invariant!

 mustuse as a function attribute only enforces there must be a 
receiver of the function call result, and do not throw away
the returned result. It has nothing to do with the (sub-)types of 
the returned value.

Let's consider the following example:
```d
abstract class AbsPaths {
   // no annotation
   abstract AbsPaths remove(int i);  // return an AbsPaths object 
with i-th element removed
}

class ImperativePaths : AbsPaths {  // will modify `this` object 
itself
   // no  mustuse annotation, with the implicit assumption that 
the caller will continue to use `this` object
   override AbsPaths remove(int i) {
     ... // remove the i-th element of `this` object
     return this;
   }
}

class FunctionalPaths : AbsPaths {  // will NOT modify `this` 
object, but return new (separate) object on modification
    mustuse  // should have this annotation
   override AbsPaths remove(int i) {
     AbsPaths other = new ImmutablePaths();
     ...  // `other` is the object with the i-th element of `this` 
object being removed; `this` object did not change
     return other;
   }
}

void main() {
   AbsPaths paths = new ImperativePaths();  // or 
FunctionalPaths() interchangeably
   AbsPaths shortPaths = paths.remove(i);   // and this line 
should always work, as long as the return value is not discarded
}
```
Here both `ImperativePaths` and `FunctionalPaths` inherit from 
`AbsPaths`, if one branch has  mustuse and the other one
does not, and the programmer is not forced to explicitly take the 
return value and use it, these two derived classes cannot be used
interchangeably. And even worse: in the case of FunctionalPaths, 
the result will be totally wrong
([the OP author's 
problem](https://forum.dlang.org/post/ssagbvubpgwvewimsocj forum.dlang.org)).


In the following class inheritance tree:
```d
----------Base-------
|         |         |
Derived1  Derived2  Derived3 <-user only manually maked 
Derived3.remove() as  mustuse here
|         |         |
GrandDr1  GrandDr2  GrandDr3
|
...
```
If the programmer only manually marked Derived3.remove() as 
 mustuse, then everything else (Base, Derived1, Derived2,
Derived3, GrandDr1, GrandDr2 GrandDr3, ...)'s .remove() method 
will all become  mustuse (as seen by the compiler internally).

With transitive closure  mustuse rule,

Pros:

1. the method interface is consistent, e.g. at any call-site of 
`AbsPaths.remove(i)`, its return value must be received.
the programmer only need to remember one interface, instead of 
looking through docs/code for all the branches in the
inheritance tree, and check for a particular class, whether its 
`remove(i)` method is  mustuse or not.
2. the programmer can easily switch between different derived 
implementation classes of AbsPaths to maximize efficiency /
performance, without worrying about potential breakage.

Cons:

1. a few more key-strokes on every call-site of  mustuse marked 
method.



[Paul Backus 
proposal:](https://forum.dlang.org/post/cqlwlnpcbtbkzqnhicwc forum.dlang.org)


In other words, you cannot introduce  mustuse in a derived 
class; it must be present in the base class.

This is somewhat problematic for D, because we have a universal 
base class, Object, and neither it nor any of its methods are 
 mustuse.


His reasoning is logically correct by itself, with the constraint 
that legacy code is un-modifiable.
However, his proposal is not very useful because of the 
constraint:

1. it won't help the existing library code where there is no 
 mustuse presence today. But, these library code are where the 
programmer
**want the compiler help most**,  as demonstrated by the OP user 
who brought up this issue on the forum. (This is also why Paul 
talked about
Object, although it's not a very good example; instead we can 
think about std.lib.AbsPath example above).
2. if the new rule have to fully honor the legacy (with 
deficiency) code, how we can improve for future D?
Also honoring legacy code, does not mean we should not even check 
for potential problems.
Even typically we cannot modify the the legacy code, at least we 
want the compiler help to check where are the potential problems
are; the compiler can give warning messages, and if they are 
manually verified, these findings should be formally
logged as bugs, and be fixed in the next release.
3. and if we follow this line of reasoning, it also beg the 
question: whether one can remove  mustuse in a derived class.
E.g. let ImperativePaths (mutable implementation) inherit
from FunctionalPaths (immutable implementation), and the caller 
can just use the `this` object as the result of the computation. 
Again,
this logic is correct by itself, but it make the whole code base 
brittle: what if the library author decided later one day that 
s/he want to
modify the class ImperativePaths again to implement another 
immutable implementation?




1.  mustuse_remedy_legacy: this annotation is to allow programmer 
flag existing legacy library code that s/he has no
right to change, but want to set a flag and let the compiler to 
help to find violations; the compiler only output warnings
instead of errors. This annotation will be implicitly propagated 
by the compiler throughout the whole inheritance tree.

2.  mustuse: proper, the default. For programmer to use in new 
code, or library code s/he can change (from the root of the
inheritance tree). Violations of this annotation will cause 
compiler error. This annotation must be explicit (just as the 
keyword
`override`).

and:

a) in both cases, this function property will be transitive 
closure, i.e. be propagated upwards and downwards, in all 
directions.

b) in both cases, removing the annotation is not allowed in the 
derived class if any supper class method carry such annotation.

Actually this rule is very simple: there is only *one* consistent 
interface for any method,  mustuse is part of that method 
interface;
for legacy code,  mustuse_remedy_legacy will cause compiler to 
generate warning message, and for new code  mustuse will cause
the compiler to generate error message. That's all.



binaries
Let's revisit the motivating example:
```d
paths.remove(i); // compiles fine but does nothing
paths = paths.remove(i); // works - what I erroneously thought 
the previous line was doing
```

Suppose the `paths`' type is `AbsPaths`, and the programmer have 
no right to modify it (e.g. in std.lib, or even pre-built 
binaries),
the compiler can only issue warning (instead of error) messages. 
But the programmer must be made aware of where
these potential problems are located.

And, once the programmer discovered one such misuse problem, s/he 
can try to find and fix all such potential problems by defining
a helper `class RemedyPaths` as follows:
```d
// class Paths is located in the source file that the programmer 
has no right to modify
class RemedyPaths : std.lib.AbsPaths {  // helper class to trace 
all the occurrences of the  mustuse violation
    mustuse_remedy_legacy
   override Paths remove(int i) {return null;}
}
```
and the compiler will find out all the occurrences of the same 
issues in the code visited by the compilation, and issue
warnings (not errors), so the programmer have a chance to visit 
all the code locations where `remove(i)`'s function
return value are discarded.



Since this kind of warning message is transitive closure by 
design, so for the *implicit* markings made by the
compiler: as a debugging aid the warning message should indicate 
the **originating source of the annotation**
to make it clear to the programmers, e.g.:
```d
warning: std.lib.foo.d:123, AbsPaths.remove(int i)'s return value 
is discarded, violates the originating
annotation from user.codebase.RemedyPaths.d:456 
 mustuse_remedy_legacy.
```


With universal (i.e. transitive closure)  mustuse,

1. when the library author has decided to return a value from a 
function, typically it's represent the computation result or 
status report,
which the function caller should either use or check instead of 
discard. That is good engineering practice. It forces the 
programmer
to pay attention to the returned value, instead of assuming the 
semantics of the function e.g. based purely on the function name.
```d
   ResultType result = someFunction();
   ... // use or check `result`
```
it will save lots of debugging time, at the expense of just a few 
more key-strokes.

2. the library author can implement both ImperativePaths and 
FunctionalPaths, and the library users can choose
them interchangably for the maximal efficiency / performance 
without worrying about code breakage.




Not discarding function return value has its root from the 
command query separation principle.
As an informal exercise: let's derive command query separation 
principle from DbC.

The contract in DbC mostly exhibits as assertions in the code.

The programmer can insert assertions at any place of the code, 
without changing the code's original semantics (i.e the behavior 
when the assertions are turned-off, e.g. in release mode). In an 
OOP language, most of the time the assertions are checking some 
properties of an object, hence any method that can be called in 
an assertion must be a query (i.e a query can be called on an 
object for any number times without changing that object's 
internal state). So now we have query method.

But we do need to change an object's state in imperative 
programming, then those methods are classified as commands. After 
changing an object state, the command must NOT return any value. 
Why? because otherwise, the programmer may accidentally want to 
call that command and check the returned value in some assertions 
... then you know what happens in the end: the program behaves 
differently when assertions are turn on in debug mode and off in 
release mode.

Therefore, we have this:


 every method should either be a command that performs an 
 action, or a query that returns data to the caller, but not 
 both.




- [Liskov substitution 
principle](https://en.wikipedia.org/wiki/Liskov_substitution_principle)
- [Command query separation 
principle](https://en.wikipedia.org/wiki/Command%E2%80%93query_separation#:~:text=Command-query%20separation%20(CQS),the%20caller%2C%20but%20not%20)
- [ mustuse 
DIP1038.md](https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1038.md#mustuse-as-a-function-attribute)
--------------------------------------------------------------------------
Nov 02 2022
prev sibling parent mw <mingwu gmail.com> writes:
On Wednesday, 19 October 2022 at 07:41:33 UTC, mw wrote:

 On Wednesday, 19 October 2022 at 07:28:21 UTC, Mike Parker 
 wrote:

 On Wednesday, 19 October 2022 at 04:52:31 UTC, mw wrote:


  mustuse as a function attribute was in the original version 
 of the DIP. It was vetoed by Walter. Thus, only the type 
 attribute remains in the accepted version.


 Please include the entire context that I posted earliar from 
 the summary of the formal assessment here:

 https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1038.md#formal-assessment

 Walter didn't just willy-nilly "veto" the attribute on 
 functions. He accepted the proposal with three requests for 
 enhancement, one of which I summarized as:


 develop rules for handling covariance and contravariance when 
 applied to functions.




 Covariance and contravariance is a big topic by itself.
 But ..., what does it have anything to do here with  mustuse as 
 a function attribute?

  mustuse here only enforces there must be a receiver of the 
 function call result, and do not throw away the returned result.

 I'd really like to see some examples what's this concern is 
 about.


I read the above link again, esp these:

2) address issues that arise from the feature's interaction with 
inheritance when applied to classes.

3) develop rules for handling covariance and contravariance when 
applied to functions.


And I think, 3) is talking about 2) about types!

In short: it's about the  mustuse annotation on types! -- the 
accepted half of the DIP!

Actually, it's the other half I.e  mustuse annotation on function 
should be accepted! since it has nothing to do with covariance 
and contravariance.

And the accepted half on types should be addressed with 3).
Oct 19 2022