www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Thoughts about in contract inheritance

reply Stewart Gordon <smjg_1998 yahoo.com> writes:
There's been a lot of debate about the inheritance of in contracts, in
particular what's 
supposed to happen if a method in a base class has an in contract and its
override doesn't.
http://d.puremagic.com/issues/show_bug.cgi?id=6856

The spec isn't explicit on whether the overridden method retains the base's in
contract 
unchanged or loses its in contract altogether.

Some claim that the absence of an in contract is just syntactic sugar for an
empty in 
contract, in which case the override loses the in contract (can be called with
any 
arguments of the correct types).  This is the view taken by DMD.

But others claim that you shouldn't have to specify an in contract in order to
inherit the 
one you already have, and that if you want to remove it altogether then you
should have to 
do it explicitly.

What's more, there's a hole in the current equivalence between no in contract
and an empty 
in contract: in only the former case does the compiler reject an in contract on
an 
override further down the hierarchy.  This will want to be fixed if an explicit
empty in 
contract becomes the only way to lift all argument restrictions when overriding
a method.


Moreover, there are hijacking vulnerabilities.

http://dlang.org/hijack.html
addresses the danger that a derived class may define a method, and then a later
version of 
the base class coincidentally defines a method with the same name but different
semantics. 
  Further problems in this area would ensue if the default behaviour were to
pass through 
the base class's in contract unchanged.  Though doing and dusting
http://d.puremagic.com/issues/show_bug.cgi?id=3836
would alleviate this.

Another scenario I've thought of is:
- library class defines a method with an in contract
- application class overrides this method, and has the same argument
restrictions as the 
library class
- a later version of the library class widens the range of acceptable inputs
- however, the app's override is not prepared to deal with inputs that are
newly allowed 
by the API

I don't know if there's a way to deal with this short of doing away with in
contract 
inheritance altogether.  Of course, the app programmer could protect against
this by 
duplicating the contract checking in the body of the method, and doing so would
even show 
whether the input to the function was actually outside the range allowed by the
API or 
merely outside the range that the app class's version of the method can deal
with.  But 
it's better not to have to duplicate code like this.

But if we avoided this by getting rid of in contract inheritance, it might just
lead more 
programmers to break the inheritance model by forbidding operations on objects
of the 
derived class that are allowed on objects of the base class.  I suppose this is
a variant of
http://en.wikipedia.org/wiki/Circle-ellipse_problem
It would also get in the way of another proposal I'm inclined to agree with,
that anything 
that calls a method on an object reference of the base class type should be
required to 
adhere to the base class's API, of which the in contract is a part.
http://d.puremagic.com/issues/show_bug.cgi?id=6857
(see also the thread "define in contract according to the caller, not the
callee." on this 
'group)

Still, is there a good way of dealing with the scenario I've brought up here?


Another thing that's needed for a robust DbC system in D:
http://d.puremagic.com/issues/show_bug.cgi?id=6549

Stewart.
Feb 29 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/29/2012 03:06 PM, Stewart Gordon wrote:
 There's been a lot of debate about the inheritance of in contracts, in
 particular what's supposed to happen if a method in a base class has an
 in contract and its override doesn't.
 http://d.puremagic.com/issues/show_bug.cgi?id=6856

 The spec isn't explicit on whether the overridden method retains the
 base's in contract unchanged or loses its in contract altogether.

The front page of the web site is quite explicit about this: // Interfaces and classes interface Printable { void print(uint level) in { assert(level > 0); } // contract is part of the interface } // Interface implementation class Widget : Printable { void print(uint level) { ... } // <-- assumed to inherit contract } // Single inheritance of state class ExtendedWidget : Widget { override void print(uint level) in { /* weakening precondition is okay */ } body { // <-- see here! ... level may be 0 here ... } } Anyway, probably it is not stated explicitly in the relevant parts of the spec because it is assumed that the reader is familiar with similar features in other languages.
 Some claim that the absence of an in contract is just syntactic sugar
 for an empty in contract, in which case the override loses the in
 contract (can be called with any arguments of the correct types). This
 is the view taken by DMD.

This is the bug in DMD.
 But others claim that you shouldn't have to specify an in contract in
 order to inherit the one you already have, and that if you want to
 remove it altogether then you should have to do it explicitly.

 What's more, there's a hole in the current equivalence between no in
 contract and an empty in contract: in only the former case does the
 compiler reject an in contract on an override further down the
 hierarchy.
 This will want to be fixed if an explicit empty in contract
 becomes the only way to lift all argument restrictions when overriding a
 method.

Probably.
 Moreover, there are hijacking vulnerabilities.

 http://dlang.org/hijack.html
 addresses the danger that a derived class may define a method, and then
 a later version of the base class coincidentally defines a method with
 the same name but different semantics. Further problems in this area
 would ensue if the default behaviour were to pass through the base
 class's in contract unchanged.

Not at all. If anything, it would alleviate the issue. Likely, assertion failures would be introduced that show that there is a consistency problem in the application.
 Though doing and dusting
 http://d.puremagic.com/issues/show_bug.cgi?id=3836
 would alleviate this.

Indeed.
 Another scenario I've thought of is:
 - library class defines a method with an in contract
 - application class overrides this method, and has the same argument
 restrictions as the library class
 - a later version of the library class widens the range of acceptable
 inputs
 - however, the app's override is not prepared to deal with inputs that
 are newly allowed by the API

 ...

This is not a contract-related problem. It is a breaking API change, whether or not the library class defines language level contracts.
 But if we avoided this by getting rid of in contract inheritance,

If we want to avoid this we'd have to get rid of inheritance altogether, not just contract inheritance. A contract always has a corresponding implementation.
 it might just lead more programmers to break the inheritance model by
 forbidding operations on objects of the derived class that are allowed
 on objects of the base class. I suppose this is a variant of
 http://en.wikipedia.org/wiki/Circle-ellipse_problem
 It would also get in the way of another proposal I'm inclined to agree
 with, that anything that calls a method on an object reference of the
 base class type should be required to adhere to the base class's API, of
 which the in contract is a part.
 http://d.puremagic.com/issues/show_bug.cgi?id=6857
 (see also the thread "define in contract according to the caller, not
 the callee." on this 'group)

 Still, is there a good way of dealing with the scenario I've brought up
 here?

Library writers shouldn't silently change functionality and/or redefine interfaces. The new stuff should be introduced under a different name and the old name should be deprecated. (As an inferior alternative, the library user could just read the change log and notice that there is now a problem.) Anyway, this second scenario has a similar severity under the current contract inheritance behavior in DMD 2.058 and the right inheritance behavior in a future version of DMD.
 Another thing that's needed for a robust DbC system in D:
 http://d.puremagic.com/issues/show_bug.cgi?id=6549

 Stewart.

Feb 29 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 29/02/2012 14:44, Timon Gehr wrote:
<snip>
 The spec isn't explicit on whether the overridden method retains the
 base's in contract unchanged or loses its in contract altogether.

The front page of the web site is quite explicit about this:

What web site? Certainly not www.digitalmars.com or d-programming-language.org or dlang.org as I look at the moment. <snip>
 Anyway, probably it is not stated explicitly in the relevant parts of the spec
because it
 is assumed that the reader is familiar with similar features in other
languages.

Then it's a hole in the spec. If it's only meant to state how D differs from some other language, it would have to state what language that is. <snip>
 Another scenario I've thought of is:
 - library class defines a method with an in contract
 - application class overrides this method, and has the same argument
 restrictions as the library class
 - a later version of the library class widens the range of acceptable
 inputs
 - however, the app's override is not prepared to deal with inputs that
 are newly allowed by the API

 ...

This is not a contract-related problem. It is a breaking API change, whether or not the library class defines language level contracts.

How do you mean? <snip>
 Library writers shouldn't silently change functionality and/or redefine
interfaces.

So you think that, once a library is written and released, no new functionality should ever be added? Stewart.
Feb 29 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/29/2012 05:26 PM, Stewart Gordon wrote:
 On 29/02/2012 14:44, Timon Gehr wrote:
 <snip>
 The spec isn't explicit on whether the overridden method retains the
 base's in contract unchanged or loses its in contract altogether.

The front page of the web site is quite explicit about this:

What web site? Certainly not www.digitalmars.com or d-programming-language.org or dlang.org as I look at the moment.

The official website, d-programming-language.org or dlang.org. You have to click on "See example". It is at the first bulb of "Power".
 <snip>
 Anyway, probably it is not stated explicitly in the relevant parts of
 the spec because it
 is assumed that the reader is familiar with similar features in other
 languages.

Then it's a hole in the spec. If it's only meant to state how D differs from some other language, it would have to state what language that is.

It surely is a hole in the spec.
 <snip>
 Another scenario I've thought of is:
 - library class defines a method with an in contract
 - application class overrides this method, and has the same argument
 restrictions as the library class
 - a later version of the library class widens the range of acceptable
 inputs
 - however, the app's override is not prepared to deal with inputs that
 are newly allowed by the API

 ...

This is not a contract-related problem. It is a breaking API change, whether or not the library class defines language level contracts.

How do you mean?

Language level contracts as in D are (basically) a way to introduce runtime checks into interfaces. If the interface is redefined then that potentially breaks all code that implements the interface, even if this is not explicitly stated in form of contracts.
 <snip>
 Library writers shouldn't silently change functionality and/or
 redefine interfaces.

So you think that, once a library is written and released, no new functionality should ever be added?

Obviously there can be additional changes without redefining what an existing part of the API does. Timon Gehr wrote:
 The new stuff should be introduced under a different name and the old name
should be deprecated.

Feb 29 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 29/02/2012 17:01, Timon Gehr wrote:
<snip>
 The official website, d-programming-language.org or dlang.org. You have to
click on "See
 example". It is at the first bulb of "Power".

But the comment "// <-- assumed to inherit contract" isn't actually there, so what's "quite explicit" about it? Besides, the interface declaration there isn't allowed by the grammar. <snip>
 Language level contracts as in D are (basically) a way to introduce runtime
checks into
 interfaces. If the interface is redefined then that potentially breaks all
code that
 implements the interface, even if this is not explicitly stated in form of
contracts.

You mean the fault lies on the part of the library creator for widening the in contract of a non-final method? <snip>
 Obviously there can be additional changes without redefining what an existing
part of the
 API does.

So you consider illegal inputs to a function to be part of the API? Stewart.
Feb 29 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/29/2012 07:06 PM, Stewart Gordon wrote:
 On 29/02/2012 17:01, Timon Gehr wrote:
 <snip>
 The official website, d-programming-language.org or dlang.org. You
 have to click on "See
 example". It is at the first bulb of "Power".

But the comment "// <-- assumed to inherit contract" isn't actually there, so what's "quite explicit" about it?

That was just for documentation... The part that is explicit is: // Single inheritance of state class ExtendedWidget : Widget { override void print(uint level) in { /* weakening precondition is okay */ } body { ... level may be 0 here ... } } The fact that this weakens the precondition tells us that it was not weakened before.
 Besides, the interface
 declaration there isn't allowed by the grammar.

The compiler implements it.
 <snip>

 Language level contracts as in D are (basically) a way to introduce
 runtime checks into
 interfaces. If the interface is redefined then that potentially breaks
 all code that
 implements the interface, even if this is not explicitly stated in
 form of contracts.

You mean the fault lies on the part of the library creator for widening the in contract of a non-final method?

Sure. This adds additional requirements that any deriving class needs to fulfill. It is not a backwards-compatible change.
 <snip>
 Obviously there can be additional changes without redefining what an
 existing part of the
 API does.

So you consider illegal inputs to a function to be part of the API?

Yes. Put differently, I consider legal inputs to a overridable virtual function to be part of the API.
Feb 29 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 29/02/2012 19:24, Timon Gehr wrote:
<snip>
 That was just for documentation... The part that is explicit is:

 // Single inheritance of state
 class ExtendedWidget : Widget {
 override void print(uint level)
 in { /* weakening precondition is okay */ } body {
 ... level may be 0 here ...
 }
 }

 The fact that this weakens the precondition tells us that it was not weakened
before.

Of course. I see now.
 Besides, the interface declaration there isn't allowed by the grammar.

The compiler implements it.

So what? It's a bug that compiler behaviour doesn't match the documentation. You seem to be agreed that this is the case with what happens to the contract where the override has no InStatement at all. <snip>
 So you consider illegal inputs to a function to be part of the API?

Yes. Put differently, I consider legal inputs to a overridable virtual function to be part of the API.

That legal inputs are part of the API is something we're agreed on. It's illegal inputs we were debating. But I can see what you really mean: the spec of what inputs to an overridable function are legal and what inputs are illegal is part of the API. Stewart.
Feb 29 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/29/2012 09:30 PM, Stewart Gordon wrote:
 On 29/02/2012 19:24, Timon Gehr wrote:
 <snip>
 That was just for documentation... The part that is explicit is:

 // Single inheritance of state
 class ExtendedWidget : Widget {
 override void print(uint level)
 in { /* weakening precondition is okay */ } body {
 ... level may be 0 here ...
 }
 }

 The fact that this weakens the precondition tells us that it was not
 weakened before.

Of course. I see now.

OK.
 Besides, the interface declaration there isn't allowed by the grammar.

The compiler implements it.

So what? It's a bug that compiler behaviour doesn't match the documentation. You seem to be agreed that this is the case with what happens to the contract where the override has no InStatement at all.

I think in this case the documentation has not been updated yet because the feature is still experimental. Anyway, even with the documentation, many essential parts of the language are documented only on this newsgroup or through the compiler implementation, unspecified completely or only partly specified. I don't think it is currently possible to become completely proficient in D without reading this newsgroup.
 <snip>
 So you consider illegal inputs to a function to be part of the API?

Yes. Put differently, I consider legal inputs to a overridable virtual function to be part of the API.

That legal inputs are part of the API is something we're agreed on. It's illegal inputs we were debating. But I can see what you really mean: the spec of what inputs to an overridable function are legal and what inputs are illegal is part of the API. Stewart.

An input that is not legal is illegal and vice-versa.
Feb 29 2012
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
On 29/02/2012 20:45, Timon Gehr wrote:
 On 02/29/2012 09:30 PM, Stewart Gordon wrote:

 But I can see what you really mean: the spec of what inputs to an
 overridable function are legal and what inputs are illegal is part of
 the API.

 Stewart.

An input that is not legal is illegal and vice-versa.

Well, obviously, but what's your point? Maybe the question is of what's meant by "part of the API". I'd made out that we were talking about what a user of the library would be right to rely on. That a given function call is legal normally falls under that category. But if you call a function, you will call it with legal input, so probably won't care about some other input that is illegal. Of course, if the function in question admits alternative implementations through virtuality, then you're going to get implementations relying on the restrictions on input, so the illegality of some calls to the function becomes part of the API. But that doesn't magically render "the fact that a given function call is legal is part of the API" and "the fact that a given function call is illegal is part of the API" logically equivalent statements. Or does some of your code rely on the fact that a given library class has no method with a given name? Stewart.
Feb 29 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 03/01/2012 01:49 AM, Stewart Gordon wrote:
 On 29/02/2012 20:45, Timon Gehr wrote:
 On 02/29/2012 09:30 PM, Stewart Gordon wrote:

 But I can see what you really mean: the spec of what inputs to an
 overridable function are legal and what inputs are illegal is part of
 the API.

 Stewart.

An input that is not legal is illegal and vice-versa.

Well, obviously, but what's your point? Maybe the question is of what's meant by "part of the API". I'd made out that we were talking about what a user of the library would be right to rely on.

Exactly.
  That a given function call is legal normally falls under that
 category. But if you call a function, you will call it with legal input,
 so probably won't care about some other input that is illegal.

 Of course, if the function in question admits alternative
 implementations through virtuality, then you're going to get
 implementations relying on the restrictions on input, so the illegality
 of some calls to the function becomes part of the API. But that doesn't
 magically render "the fact that a given function call is legal is part
 of the API" and "the fact that a given function call is illegal is part
 of the API" logically equivalent statements.

Well, what is part of the API ultimatively depends on what is in the API documentation. If this documentation does not specify the behavior of a function for a certain set of inputs, then it can add suitable behaviors for those inputs to the API documentation later. In case of virtual overridable functions (as in the scenario we are discussing in this thread), the API must specify sufficient and required conditions that clearly identify the function input validity. It is in this case when the two statements are equivalent. In the general case that includes non-overridable functions, they might be different.
 Or does some of your code rely on the fact that a given library class
 has no method with a given name?

Most meaningful code that inherits a library class does rely on such a fact.
Mar 01 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, February 29, 2012 21:45:19 Timon Gehr wrote:
 On 02/29/2012 09:30 PM, Stewart Gordon wrote:
 So what? It's a bug that compiler behaviour doesn't match the
 documentation. You seem to be agreed that this is the case with what
 happens to the contract where the override has no InStatement at all.

I think in this case the documentation has not been updated yet because the feature is still experimental. Anyway, even with the documentation, many essential parts of the language are documented only on this newsgroup or through the compiler implementation, unspecified completely or only partly specified. I don't think it is currently possible to become completely proficient in D without reading this newsgroup.

It's certainly the case that when the spec does not match the compiler, you _cannot_ assume that it's the spec that's correct. We have the spec, the compiler, _and_ TDPL to worry about. If they don't agree, then TDPL is most likely to be right out of the 3, but there's no guarantee. And definitely between the spec and the compiler, you can't trust that the spec is more correct than the compiler. It depends entirely on what the mismatch is. There have been some recent fixes to the spec, so it's not as bad as it used to be, but you can't really assume that what the spec is correct when it and the compiler disagree. - Jonathan M Davis
Feb 29 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 29/02/2012 17:26, Stewart Gordon a écrit :
 So you think that, once a library is written and released, no new
 functionality should ever be added?

We are reaching the discussion of evolving API. I do think that if change such a thing, you should rename your function to express its new functionality. And mark the previous function as deprecated, with forwarding to the deprecated function, when appropriate, for a transitional period, so code using the old API have some time to adapt before it is removed completely. Additionally, this transition process require to be able to provide a default implementation for interface's function. Java did face the same problem (actually it is worse because of bad choice in generic implementation) and it will be implemented in java 8 or 9 IIRC. I think this problem is way more broad than a contract issue. Both seems orthogonal to me. Any change in the base class can mess up its subclasses, the change being in contract or somewhere else isn't that important. It is an API evolution strategy problem.
Feb 29 2012