www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Proposition for change in D regarding Inheriting overloaded methods

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
OK,

So I originally posted the Overloading/Inheritance question as I was 
confused by the behavior of the current D compiler.  After reading all of 
the responses arguments, I believe there are two main camps in the debate. 
One camp, which I'll name the C++ camp, believes the C++ behavior is the 
best approach.  This camp includes Walter, and believes the current 
implementation of the D compiler (which mimics C++) is best.  The second 
camp includes myself, and several other users of D who were either aware of 
this issue, or unaware and surprised by the current implementation.  I'll 
call this the Java camp, as the behavior I desire is most closely imitating 
Java (although not exactly).  There may be other ways of solving this 
problem, and I welcome those to voice their opinions.  I will try to respond 
to everyone.

Now, I am by no means an expert in anything to do with writing languages, so 
this proposal may come across as a bit stumbly or informal, but I think it's 
important that I start a new thread in which to sort of draw attention to 
the fact that I am no longer asking questions about the current 
implementation, nor am I submitting a bug.  What I believe is that the 
specification itself should be changed.  So I'll make my proposal, and let 
everyone attempt to shoot holes in it/bolster it, until hopefully there is a 
decision by those important enough to make the changes on whether a change 
should be made.

For those of you who were confused by the original question, I'll describe 
the behavior of D as it stands, and why I have issues with it.

When a class inherits from another class, and redefines one of the base 
class' methods, using the same name and parameter types, the derived class 
overrides that method.  However, if the method is overloaded, the base 
class' methods are not considered when calling that method.  This is the 
case even when the derived class has no suitable overrides for the call in 
question.  For example (augmented example from spec):

class A
{
   int foo(int x) { ... }
   int foo(long y) { ... }
   int foo(char[] s) { ... }
}

class B : A
{
  override int foo(long x) { ... }
}

void test()
{
  B b = new B();
  A a = b;
  b.foo(1);   // calls B.foo(long), since A.foo(int) not considered
  a.foo(1);   // calls A.foo(int) because it is not overrided by B
  b.foo("hello");     // generates a compiler error because A is not 
considered for overrides
  a.foo("hello");     // calls A.foo(char[])
}

To have the compiler consider the base class' overloads before considering 
the derived class' overloads, an alias can be added:

class B : A
{
  alias A.foo foo;
  override int foo(long x) { ... }
}

void test()
{
  B b = new B();
  A a = b;
  b.foo(1);   // calls A.foo(int)
  a.foo(1);   // calls A.foo(int)
  b.foo("hello");     // calls A.foo(char[])
  a.foo("hello");     // calls A.foo(char[])
}

Thus ends the definition of the issue.  Here is where my opinion comes in. 
There are two issues in this scenario, and both of them have to do with the 
intentions of the author of class B.  I think everyone agrees that the 
author of B intended to handle the case where foo(long) is called.

However, does the author intend to handle the case where foo(int) is called? 
Let's assume he does (which is what the current compiler assumes).  The 
author has not forbidden the user of the class from calling A.foo, because 
the user can simply cast to an A object, and call A.foo(int) directly. 
Therefore, if the author meant to override foo(int) by defining just a 
foo(long), he has failed.  From these points, I believe that the above code 
is an example of an incorrectly implemented override, and I believe that the 
correct response the compiler should have is to error out, not while 
compiling test(), but while compiling B, indicating to the user that he 
should either declare the alias, or override foo(int).  This is the point in 
which my solution differs from Java.  Java would allow this to proceed, and 
call the base class' foo(int) in all cases, which is not what the author 
intended.

The second issue is how to handle the foo(char[]) case.  In the current 
implementation, because A is not searched for overrides, the compiler 
produces an error indicating that the user tried to call the foo(long) 
method, but there is no implicit conversion from char[] to long.  There are 
two possibilities.  One is that the author did not notice that A.foo(char[]) 
existed, and intended to override all instances of foo() with his foo(long) 
override.  However, the author is NOT notified of this issue, only the user 
of the class is notified of this issue.  So the potential for unambiguous 
code to have escaped exists.  The second possibility is that the author 
fully intended to allow the base class to define foo(char[]), but forgot to 
define the alias.  Again, since the compiler gives no error, he is unaware 
that he is releasing buggy code to the world.  I believe the correct 
assumption of the compiler should be that the user wanted the alias for the 
base class' foo(char[]), and should alias it implicitly if and only if no 
suitable match exists on the derived class.  In the case where the author 
did not notice foo(char[]) existed, he problably doesn't mind that 
foo(char[]) is defined by the base class.  If a suitable match exists that 
is not a direct override of the base class, then the issue reduces to the 
previous case, where an implicit conversion is required, and the compiler 
should error out.

There is one other possibile solution that I would be willing to concede to, 
and that is that the compiler errors out if the base class does not override 
all overloads of a particular method name.  This forces the user to either 
override all overloads of the method, or define the alias.  This would be 
the safest solution, as the author of B must make his intentions perfectly 
clear.

So my proposal is to change the specification so that:

If there is a class A, which is a base class of class B, where A defines a 
method foo(args), and B defines a method foo(args2), such that the types of 
args2 cannot be implicitly converted to args, and B does not define 
foo(args), then the definition of B.foo(args) shall be implicitly aliased to 
A.foo(args).  If, using the same assumptions, args2 can be implicitly 
converted to args, then the compiler should fail to compile B indicating 
that the user must define B.foo(args) by override or by alias.

I believe this will give us the best of both camps, and allow much less code 
to be released with silent bugs than the current implementation.

Let the hole shooting begin...

-Steve
Aug 07 2007
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:
 However, does the author intend to handle the case where foo(int) is called? 
 Let's assume he does (which is what the current compiler assumes).  The 
 author has not forbidden the user of the class from calling A.foo, because 
 the user can simply cast to an A object, and call A.foo(int) directly. 
 Therefore, if the author meant to override foo(int) by defining just a 
 foo(long), he has failed.  From these points, I believe that the above code 
 is an example of an incorrectly implemented override, and I believe that the 
 correct response the compiler should have is to error out, not while 
 compiling test(), but while compiling B, indicating to the user that he 
 should either declare the alias, or override foo(int).  This is the point in 
 which my solution differs from Java.  Java would allow this to proceed, and 
 call the base class' foo(int) in all cases, which is not what the author 
 intended.

The next update of the compiler will throw a runtime exception for this case.
 The second issue is how to handle the foo(char[]) case.  In the current 
 implementation, because A is not searched for overrides, the compiler 
 produces an error indicating that the user tried to call the foo(long) 
 method, but there is no implicit conversion from char[] to long.  There are 
 two possibilities.  One is that the author did not notice that A.foo(char[]) 
 existed, and intended to override all instances of foo() with his foo(long) 
 override.  However, the author is NOT notified of this issue, only the user 
 of the class is notified of this issue.  So the potential for unambiguous 
 code to have escaped exists.

But a compile time error is still generated, so I don't regard this as a big problem. The big problems are silent hijacking of code.
 The second possibility is that the author 
 fully intended to allow the base class to define foo(char[]), but forgot to 
 define the alias.  Again, since the compiler gives no error, he is unaware 
 that he is releasing buggy code to the world.  I believe the correct 
 assumption of the compiler should be that the user wanted the alias for the 
 base class' foo(char[]), and should alias it implicitly if and only if no 
 suitable match exists on the derived class.  In the case where the author 
 did not notice foo(char[]) existed, he problably doesn't mind that 
 foo(char[]) is defined by the base class.

The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.
 If a suitable match exists that 
 is not a direct override of the base class, then the issue reduces to the 
 previous case, where an implicit conversion is required, and the compiler 
 should error out.
 
 There is one other possibile solution that I would be willing to concede to, 
 and that is that the compiler errors out if the base class does not override 
 all overloads of a particular method name.  This forces the user to either 
 override all overloads of the method, or define the alias.  This would be 
 the safest solution, as the author of B must make his intentions perfectly 
 clear.

I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter. I prefer the solution where attempts to call unoverridden base class overloads will result in a runtime exception.
 I believe this will give us the best of both camps, and allow much less code 
 to be released with silent bugs than the current implementation.

I believe that the enforcing of the override attribute, and the runtime exception case as described, closes all the known hijacking issues.
Aug 07 2007
next sibling parent reply Regan Heath <regan netmail.co.nz> writes:
Walter Bright wrote:
 The next update of the compiler will throw a runtime exception for this 
 case.

So, in this case: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception. Is this done at runtime instead of compile time because the parameters cannot always be determined at compile time?
 The second possibility is that the author fully intended to allow the 
 base class to define foo(char[]), but forgot to define the alias.  
 Again, since the compiler gives no error, he is unaware that he is 
 releasing buggy code to the world.  I believe the correct assumption 
 of the compiler should be that the user wanted the alias for the base 
 class' foo(char[]), and should alias it implicitly if and only if no 
 suitable match exists on the derived class.  In the case where the 
 author did not notice foo(char[]) existed, he problably doesn't mind 
 that foo(char[]) is defined by the base class.

The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.

It took me a while (because the example seems to be about something totally different) but I think the argument you're making is that you would prefer an error, requiring the author to specify what they want explicitly, rather than for the compiler to make a potentially incorrect assumption, silently. Is that correct? In the original example (trimmed slightly): class A { int foo(int x) { ... } int foo(long y) { ... } int foo(char[] s) { ... } } class B : A { override int foo(long x) { ... } } void test() { B b = new B(); A a = b; b.foo("hello"); // generates a compiler error a.foo("hello"); // calls A.foo(char[]) } you're already making an assumption, you're assuming the author of B does not want to expose foo(char[]) and it's the fact that this assumption is wrong that has caused this entire debate. As others have mentioned, this assumption destroys the "is-a" relationship of inheritance because "foo(char[])" is a method of A but not a method of B. Meaning B "isn't-a" A any more... unless you've referring to a B with a reference to an A, when suddenly, it is. Crazy idea, could the compiler (when it fails to match this overload) cast the object to it's base class and try again, repeat until you hit Object. I guess this would essentially be a modification of the method lookup rules ;) Making the opposite assumption (implicitly aliasing the "foo(char[])") doesn't introduce any silent bugs (that I am aware of) and restores the "is-a" relationship. If the author really didn't want to expose "foo(char[])" then why were they deriving their class from A? It goes against the whole idea of inheritance, doesn't it? In special cases perhaps this is valid, in which case the author should explicitly define an overload and throw and exception or assert or both. Note that I said "special cases" above, I think the most common case is that the "foo(char[])" should be implicitly aliased into the derived class.
 If a suitable match exists that is not a direct override of the base 
 class, then the issue reduces to the previous case, where an implicit 
 conversion is required, and the compiler should error out.

 There is one other possibile solution that I would be willing to 
 concede to, and that is that the compiler errors out if the base class 
 does not override all overloads of a particular method name.  This 
 forces the user to either override all overloads of the method, or 
 define the alias.  This would be the safest solution, as the author of 
 B must make his intentions perfectly clear.

I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter.

I agree, but I get the impression this was the least favoured suggestion, probably for the very reason you mention here.
 I believe this will give us the best of both camps, and allow much 
 less code to be released with silent bugs than the current 
 implementation.

I believe that the enforcing of the override attribute, and the runtime exception case as described, closes all the known hijacking issues.

You're probably correct, but it doesn't solve the "compiler makes the wrong assumption" problem or the "irritating behaviour" problem ;) Regan
Aug 07 2007
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Regan Heath wrote:
 Walter Bright wrote:
 The next update of the compiler will throw a runtime exception for 
 this case.

So, in this case: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception.

There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).
 Is this done at runtime instead of compile time because the parameters 
 cannot always be determined at compile time?

Yes.
 
 The second possibility is that the author fully intended to allow the 
 base class to define foo(char[]), but forgot to define the alias.  
 Again, since the compiler gives no error, he is unaware that he is 
 releasing buggy code to the world.  I believe the correct assumption 
 of the compiler should be that the user wanted the alias for the base 
 class' foo(char[]), and should alias it implicitly if and only if no 
 suitable match exists on the derived class.  In the case where the 
 author did not notice foo(char[]) existed, he problably doesn't mind 
 that foo(char[]) is defined by the base class.

The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.

It took me a while (because the example seems to be about something totally different) but I think the argument you're making is that you would prefer an error, requiring the author to specify what they want explicitly, rather than for the compiler to make a potentially incorrect assumption, silently. Is that correct?

Yes.
 
 In the original example (trimmed slightly):
 
 class A
 {
    int foo(int x) { ... }
    int foo(long y) { ... }
    int foo(char[] s) { ... }
 }
 
 class B : A
 {
   override int foo(long x) { ... }
 }
 
 void test()
 {
   B b = new B();
   A a = b;
 
   b.foo("hello");     // generates a compiler error
   a.foo("hello");     // calls A.foo(char[])
 }
 
 you're already making an assumption, you're assuming the author of B 
 does not want to expose foo(char[]) and it's the fact that this 
 assumption is wrong that has caused this entire debate.

The language is assuming things on the conservative side, not the expansive side, based on the theory that it is better to generate an error for questionable (and easily correctable) constructs than to make a silent (and erroneous) assumption.
 As others have mentioned, this assumption destroys the "is-a" 
 relationship of inheritance because "foo(char[])" is a method of A but 
 not a method of B.

We should not take rules as absolutes when they don't give us desirable behavior.
 Meaning B "isn't-a" A any more... unless you've 
 referring to a B with a reference to an A, when suddenly, it is.

That will generate a runtime error.
 Crazy idea, could the compiler (when it fails to match this overload) 
 cast the object to it's base class and try again, repeat until you hit 
 Object.  I guess this would essentially be a modification of the method 
 lookup rules ;)
 
 
 Making the opposite assumption (implicitly aliasing the "foo(char[])") 
 doesn't introduce any silent bugs (that I am aware of) and restores the 
 "is-a" relationship.
 
 If the author really didn't want to expose "foo(char[])" then why were 
 they deriving their class from A?  It goes against the whole idea of 
 inheritance, doesn't it?

The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.
Aug 07 2007
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote
 Regan Heath wrote:
 when the call to b.set(3) is made you insert a runtime check which looks 
 for methods called 'set' in <actual type of object>, if none of them take 
 a <insert types of parameters> you throw an exception.

There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).
 Is this done at runtime instead of compile time because the parameters 
 cannot always be determined at compile time?

Yes.

Hm... I'm slightly ignorant on this issue, not being a compiler developer. After reading this, I'm thinking I need to change my proposition to my alternate solution, which is that the compiler should produce an error whenever the derived class does not override the base class's overloads (my alternate solution). From your answer here, it appears that my assumption that the compiler can tell whether a given type of argument could be converted to a base class' argument type might be impossible to tell at compile time. Or is it? In any case, now that I think about it, it produces an O(n^2) run time as the compiler needs to check every argument type in the derived class to see if it can be implicitly converted to every argument type in the base class. This might produce a very slow compiler. I still believe generating a runtime error is no better than the current behavior of the compiler, actually I think it's worse.
 In the original example (trimmed slightly):

 class A
 {
    int foo(int x) { ... }
    int foo(long y) { ... }
    int foo(char[] s) { ... }
 }

 class B : A
 {
   override int foo(long x) { ... }
 }

 void test()
 {
   B b = new B();
   A a = b;

   b.foo("hello");     // generates a compiler error
   a.foo("hello");     // calls A.foo(char[])
 }

 you're already making an assumption, you're assuming the author of B does 
 not want to expose foo(char[]) and it's the fact that this assumption is 
 wrong that has caused this entire debate.

The language is assuming things on the conservative side, not the expansive side, based on the theory that it is better to generate an error for questionable (and easily correctable) constructs than to make a silent (and erroneous) assumption.

The conservative side would be to say that it is an error to generate the class in the first place seeing as how it hasn't defined all the behavior it should. The more I look at it, I think the best solution is not to assume anything, and force the author of the class to define it more clearly.
 As others have mentioned, this assumption destroys the "is-a" 
 relationship of inheritance because "foo(char[])" is a method of A but 
 not a method of B.

We should not take rules as absolutes when they don't give us desirable behavior.

I still would like a real example of how this is desirable.
 If the author really didn't want to expose "foo(char[])" then why were 
 they deriving their class from A?  It goes against the whole idea of 
 inheritance, doesn't it?

The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.

This argument is absurd. How is the author of A's hands tied? If author A changes his API, author B had better take notice. What if author A decides to change the way he stores protected data? How can you prevent author B from having to understand that? Bottom line, if you derive a class, and the base class changes, all bets are off. You may have to change your class. I see no way to prevent this possibility. Even with your exception solution, A can break code which uses instances of B by adding an overload. -Steve
Aug 07 2007
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steven Schveighoffer wrote:
 I still would like a real example of how this is desirable.

When author A wants to extend the API of A without silently breaking derived B of which author A has no knowledge of.
 If the author really didn't want to expose "foo(char[])" then why were 
 they deriving their class from A?  It goes against the whole idea of 
 inheritance, doesn't it?

functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.

This argument is absurd. How is the author of A's hands tied? If author A changes his API, author B had better take notice.

The problem is, author A adds to (not changes) his API, and B starts silently failing. No compile error, no runtime error.
 What if author A decides 
 to change the way he stores protected data?  How can you prevent author B 
 from having to understand that?

Author A should be able to *add* methods and fields to his API without possibly causing silent breakage of derived classes. It is ok to cause them to break with a compile or runtime error, but not ok to silently break them. If author A *changes* his API, that's a different thing entirely.
 Even with your exception solution, A can break code which uses instances of 
 B by adding an overload.

It won't be silent breakage, though. The evil is in the *silent* breakage. The author A shouldn't have his hands tied preventing him from adding to his API out of fear of silently breaking his customers' code.
Aug 07 2007
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:f9bej0$21vn$1 digitalmars.com...
 Steven Schveighoffer wrote:
 I still would like a real example of how this is desirable.

When author A wants to extend the API of A without silently breaking derived B of which author A has no knowledge of.

I meant a piece of code that will not cause compiler error. I believe in the case that A adds an overload that B does not override, the compiler will error when author B tries to compile his code, will it not? Is this not a loud error? I'm assuming the author of B must recompile his code when A is updated. I realize now that we may be arguing the same point. I believe my original solution may not be feasible to imlpement, so I have fallen back on my alternate solution. I'll formally state it here: If there is a class A, which is a base class of class B, where A defines a method foo(args), and B does not define foo(args), then the compiler should fail to compile B indicating that the user must define B.foo(args) by override or by alias.
 This argument is absurd.  How is the author of A's hands tied?  If author 
 A changes his API, author B had better take notice.

The problem is, author A adds to (not changes) his API, and B starts silently failing. No compile error, no runtime error.

I belive that my new solution would produce a compile error upon compiling B. Even in my original solution, you should see no error, but if the arguments of the new method are not implicitly convertable to any of B's methods, then code which uses B would not accidentally call the new method anyways. If they were implicitly convertable, the compiler would error.
 What if author A decides to change the way he stores protected data?  How 
 can you prevent author B from having to understand that?

Author A should be able to *add* methods and fields to his API without possibly causing silent breakage of derived classes. It is ok to cause them to break with a compile or runtime error, but not ok to silently break them.

Your definition of silent error is not the same as mine. If author B does not care about new APIs, he could release his code without testing the new method, and without finding out it throws an exception. This means B can compile without the author knowing that this bomb is sitting in the new method. He then can release his code unknowing that it will fail when someone tries to call the new method on his class. To me, it is a silent error to the author of B, maybe not as "evil" as code hijacking, but still silent.
 If author A *changes* his API, that's a different thing entirely.

If author A adds any methods to his class, it changes the API. Imagine if A added a method with a new name that B did not have an overload for. This is augmenting the class, but will not force an exception because B does not define an override of that method. What if this method "breaks" the class as you have alluded to (but still have not given an example)? B is now forced to take into account this new method. By your argument A's hands are still tied even with your solution. -Steve
Aug 08 2007
parent reply Christopher Wright <dhasenan gmail.com> writes:
Steven Schveighoffer wrote:
 If there is a class A, which is a base class of class B, where A defines a 
 method foo(args),  and B does not define foo(args), then the compiler should 
 fail to compile B indicating that the user must define B.foo(args) by 
 override or by alias.

And if you don't want to provide access to those overloads? If, for instance, the new overload computes something that was previously passed in as an argument, but due to your changes, you cannot compute that argument. With your solution, I'd have to override that method and have it throw an exception. And put it in the docs that the method shouldn't be used. How about having a strong and weak override? A strong override hides all overloads of the function it overrides, while a weak one hides only the overload that matches it. Then it's unambiguous, and only a matter of defining syntax. After all, Walter doesn't implicitly know whether each individual programmer wants to allow inherited overloads to work, and neither does dmd, unless they're explicitly told.
Aug 09 2007
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Christopher Wright" <dhasenan gmail.com> wrote in message 
news:f9evna$284k$1 digitalmars.com...
 Steven Schveighoffer wrote:
 If there is a class A, which is a base class of class B, where A defines 
 a method foo(args),  and B does not define foo(args), then the compiler 
 should fail to compile B indicating that the user must define B.foo(args) 
 by override or by alias.

And if you don't want to provide access to those overloads? If, for instance, the new overload computes something that was previously passed in as an argument, but due to your changes, you cannot compute that argument. With your solution, I'd have to override that method and have it throw an exception. And put it in the docs that the method shouldn't be used.

You can't "hide" it, it can be called by casting to the base class. I see no issue with you having to override the method and throw an exception, why is this solution not good enough? It is the default solution that Walter is trying to promote. However, in my opinion, I find it unlikely that an author would want this behavior, so it should not be the default. I would have to see an example to understand it more, but I think I'd probably suggest an alternative to overriding and throwing an exception (such as not deriving but defining a new class instead). Can you give me an actual example? I still have never seen one.
 How about having a strong and weak override? A strong override hides all 
 overloads of the function it overrides, while a weak one hides only the 
 overload that matches it. Then it's unambiguous, and only a matter of 
 defining syntax. After all, Walter doesn't implicitly know whether each 
 individual programmer wants to allow inherited overloads to work, and 
 neither does dmd, unless they're explicitly told.

As I said, you can't hide the override, but as a compromise, what if there were a way you could alias any undefined overrides to throw exceptions instead of call the base class? Something like: alias not_implemented foo; This saves 1) having to code a silly override for each one you wish to "hide", and 2) having multiple copies of identical code which simply throws an exception compiled into your class. I tried doing this with the current compiler by defining a function: void not_implemented(...) But the problem is that it allows one to compile code which calls overrides that didn't exist in a base class :) I think the compiler would have to handle this case in a special way. You would probably want the exception to say something like "class.foo(arg type) not defined," not sure how one would do this without adding a feature to the compiler. -Steve
Aug 09 2007
prev sibling next sibling parent Manfred Nowak <svv1999 hotmail.com> writes:
Walter Bright wrote in reply to Regan Heath:

[ reciting omitted quote!
 you're already making an assumption, you're assuming the author
 of B does not want to expose foo(char[]) and it's the fact that
 this assumption is wrong that has caused this entire debate.


 As others have mentioned, this assumption destroys the "is-a" 
 relationship of inheritance because "foo(char[])" is a method of
 A but not a method of B.

We should not take rules as absolutes when they don't give us desirable behavior.

 If the author really didn't want to expose "foo(char[])" then why
 were they deriving their class from A?  It goes against the whole
 idea of inheritance, doesn't it? 

The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.

and in reply to Steven Schveighoffer:
 The problem is, author A adds to (not changes) his API, and B
 starts silently failing.

 If author A *changes* his API, that's a different thing entirely.

Please observe that your arguments are showing kind of ambivalence. You want 1) that inheritance gives authors "desirable behaviour". 2) that authors needn't have "ideas about or control over" derived : classes 3) that authors are allowed to add to their API 4) that changing an API is entirely different, where changing is : anything but adding Now assume, that I am author of class C deriving from class B. According to wish 2) I need not have "ideas about or control over" classes D, that derive from my class C. According to wishes 1) and 3) the author of B as well as me can carelessly "add" to the respective APIs, but according to wish 4) are not allowed to "change" those APIs. But there is no other way than to change the API of B, if there is any kind of conflict between the APIs of A and B. This forces me to change the API of my class C. This forces the author of class D ... resulting in a virtual infinite recurrence. Conclusions: 1) I do not believe that in real scenarios virtual infinite recurrences : are considered as "desirable behaviour" according to wish 1). 2) For escaping from this vicious circle use _composition_. 3) This reestablishes the "is-a" relationship of inheritance. -manfred
Aug 08 2007
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Walter Bright wrote:
 Regan Heath wrote:
 
 There is no runtime check or cost for it. The compiler just inserts a 
 call to a library support routine in D's vtbl[] entry for B.set(int).
 
 Is this done at runtime instead of compile time because the parameters 
 cannot always be determined at compile time?

Yes.

Huh? I don't understand this, how come it's not possible to detect it at compile time? Doesn't the full signature of all methods from both parent and child class have to be know at compile time? -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Aug 09 2007
parent reply Regan Heath <regan netmail.co.nz> writes:
Bruno Medeiros wrote:
 Walter Bright wrote:
 Regan Heath wrote:

 There is no runtime check or cost for it. The compiler just inserts a 
 call to a library support routine in D's vtbl[] entry for B.set(int).

 Is this done at runtime instead of compile time because the 
 parameters cannot always be determined at compile time?

Yes.

Huh? I don't understand this, how come it's not possible to detect it at compile time? Doesn't the full signature of all methods from both parent and child class have to be know at compile time?

The feature(1) would be implemented at the call site. In this example: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } When compiling b.set(3) the compiler determines the type of '3' and performs it's checking based on that type. IANACW (I am not a compiler writer) so I can't think of a case but Walters reply seems to indicate that there are cases where the type of the parameter cannot be determined at compile time. Regan (1) The feature being the one described by Steven where the compiler searches all base classes of D for any overload of 'set' not implemented in D accepting 'int' or 'int' by implicit conversion and gives an error.
Aug 09 2007
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Regan Heath wrote:
 Bruno Medeiros wrote:
 Walter Bright wrote:
 Regan Heath wrote:

 There is no runtime check or cost for it. The compiler just inserts a 
 call to a library support routine in D's vtbl[] entry for B.set(int).

 Is this done at runtime instead of compile time because the 
 parameters cannot always be determined at compile time?

Yes.

Huh? I don't understand this, how come it's not possible to detect it at compile time? Doesn't the full signature of all methods from both parent and child class have to be know at compile time?

The feature(1) would be implemented at the call site. In this example:

Ah ok. Indeed, detecting if there is a call to a missing overload can only be done at runtime. But detecting if there are missing overloads can be done at compile (in the other thread Walter acknowledged it, but stated he preferred not to issue a compile error right there). -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Aug 10 2007
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Regan Heath wrote:
 Walter Bright wrote:
 The next update of the compiler will throw a runtime exception for 
 this case.

So, in this case: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception.

I assume the compiler would not throw an exception for the following case? class D : B { long square; alias B.set set; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } Sean
Aug 07 2007
parent Walter Bright <newshound1 digitalmars.com> writes:
Sean Kelly wrote:
 I assume the compiler would not throw an exception for the following case?
 
 class D : B
 {
     long square;
     alias B.set set;
     void set(long i) { B.set(i); square = x * x; }
     long squareIt() { return square; }
 }

Nope, no exception thrown.
Aug 07 2007
prev sibling next sibling parent Tiago Carvalho <merlin3000 portugalmail.pt> writes:
Walter Bright Wrote:

 Steven Schveighoffer wrote:
 However, does the author intend to handle the case where foo(int) is called? 
 Let's assume he does (which is what the current compiler assumes).  The 
 author has not forbidden the user of the class from calling A.foo, because 
 the user can simply cast to an A object, and call A.foo(int) directly. 
 Therefore, if the author meant to override foo(int) by defining just a 
 foo(long), he has failed.  From these points, I believe that the above code 
 is an example of an incorrectly implemented override, and I believe that the 
 correct response the compiler should have is to error out, not while 
 compiling test(), but while compiling B, indicating to the user that he 
 should either declare the alias, or override foo(int).  This is the point in 
 which my solution differs from Java.  Java would allow this to proceed, and 
 call the base class' foo(int) in all cases, which is not what the author 
 intended.

The next update of the compiler will throw a runtime exception for this case.
 The second issue is how to handle the foo(char[]) case.  In the current 
 implementation, because A is not searched for overrides, the compiler 
 produces an error indicating that the user tried to call the foo(long) 
 method, but there is no implicit conversion from char[] to long.  There are 
 two possibilities.  One is that the author did not notice that A.foo(char[]) 
 existed, and intended to override all instances of foo() with his foo(long) 
 override.  However, the author is NOT notified of this issue, only the user 
 of the class is notified of this issue.  So the potential for unambiguous 
 code to have escaped exists.

But a compile time error is still generated, so I don't regard this as a big problem. The big problems are silent hijacking of code.
 The second possibility is that the author 
 fully intended to allow the base class to define foo(char[]), but forgot to 
 define the alias.  Again, since the compiler gives no error, he is unaware 
 that he is releasing buggy code to the world.  I believe the correct 
 assumption of the compiler should be that the user wanted the alias for the 
 base class' foo(char[]), and should alias it implicitly if and only if no 
 suitable match exists on the derived class.  In the case where the author 
 did not notice foo(char[]) existed, he problably doesn't mind that 
 foo(char[]) is defined by the base class.

The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.
 If a suitable match exists that 
 is not a direct override of the base class, then the issue reduces to the 
 previous case, where an implicit conversion is required, and the compiler 
 should error out.
 
 There is one other possibile solution that I would be willing to concede to, 
 and that is that the compiler errors out if the base class does not override 
 all overloads of a particular method name.  This forces the user to either 
 override all overloads of the method, or define the alias.  This would be 
 the safest solution, as the author of B must make his intentions perfectly 
 clear.

I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter. I prefer the solution where attempts to call unoverridden base class overloads will result in a runtime exception.
 I believe this will give us the best of both camps, and allow much less code 
 to be released with silent bugs than the current implementation.

I believe that the enforcing of the override attribute, and the runtime exception case as described, closes all the known hijacking issues.

For the first example I think the exception solution will work fine. Since any predefined choice could hurt the work of the programmer. For the second, where the argument can't be implicitily converted, I think that if the object in question doesn't have the required method, that method should be looked in the base classes, until it reaches the Object class. And it should do this without needing to declare an alias. I think this is similar to how java works. And it's also similar to Regan sugestion.
Aug 07 2007
prev sibling next sibling parent reply Manfred Nowak <svv1999 hotmail.com> writes:
Walter Bright wrote
 closes all the known hijacking issues.

If accessing data declared private in another not imported module can be called hijacking there is at least one more issue: Defining a Base class: module def; class Base{int i;} and defining a Derived class with some operations and some private data `hidden': module def2; import std.stdio; import def; class Derived:Base{ private: int hidden=41; // foreign access possible public: void process( inout Base p){ scope d= new Derived; d.hidden= 666; d.i+=1; p= d; // no upcast needed } void read( Base p){ scope d= cast(Derived)p; //downcast needed assert( d !is null); writefln( "def2:", d.hidden); } } and having a module that does some work: module mod; private import std.stdio, def, def2; public: void process( inout Base p){ scope d= new Derived; d.process= p; // stores Base, drops Derived // do something with d? } void read( Base p){ scope d= new Derived; d.read= p; // drops Derived // do something with d? } the following claim is currently (dmd.1.016,win32) true:: ! One can access the `private' data `hidden' defined in `module def2' ! by having access only to sources of `module def' and `module mod'. This are about 25 LOC and if D is indeed designed to support auditing well, it should be easy to spot that leak. -manfred
Aug 07 2007
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Manfred Nowak wrote:
 Walter Bright wrote
 closes all the known hijacking issues.

If accessing data declared private in another not imported module can be called hijacking there is at least one more issue: Defining a Base class: module def; class Base{int i;} and defining a Derived class with some operations and some private data `hidden': module def2; import std.stdio; import def; class Derived:Base{ private: int hidden=41; // foreign access possible public: void process( inout Base p){ scope d= new Derived; d.hidden= 666; d.i+=1; p= d; // no upcast needed } void read( Base p){ scope d= cast(Derived)p; //downcast needed assert( d !is null); writefln( "def2:", d.hidden); } } and having a module that does some work: module mod; private import std.stdio, def, def2; public: void process( inout Base p){ scope d= new Derived; d.process= p; // stores Base, drops Derived // do something with d? } void read( Base p){ scope d= new Derived; d.read= p; // drops Derived // do something with d? } the following claim is currently (dmd.1.016,win32) true:: ! One can access the `private' data `hidden' defined in `module def2' ! by having access only to sources of `module def' and `module mod'. This are about 25 LOC and if D is indeed designed to support auditing well, it should be easy to spot that leak.

mod does not access hidden, neither does def. I don't understand the issue here.
Aug 07 2007
parent Manfred Nowak <svv1999 hotmail.com> writes:
Walter Bright wrote

 mod does not access hidden, neither does def. I don't understand the 
 issue here.

Not seeing the issue is the issue: D defies code inspection. Although even you are not seeing the leak, one can at least read the value of `hidden', by adding 5 LOC into some `module hijack;'---which _is_ a leak according to my definition of black hat acting. If it is okay for your intentions with D that black hats are able to read private data, then of course there is no issue zhat needs to be understood. -manfred
Aug 10 2007
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Walter Bright" wrote in message news:f9aalt$b2p$1 digitalmars.com...
 Steven Schveighoffer wrote:
 However, does the author intend to handle the case where foo(int) is 
 called? Let's assume he does (which is what the current compiler 
 assumes).  The author has not forbidden the user of the class from 
 calling A.foo, because the user can simply cast to an A object, and call 
 A.foo(int) directly. Therefore, if the author meant to override foo(int) 
 by defining just a foo(long), he has failed.  From these points, I 
 believe that the above code is an example of an incorrectly implemented 
 override, and I believe that the correct response the compiler should 
 have is to error out, not while compiling test(), but while compiling B, 
 indicating to the user that he should either declare the alias, or 
 override foo(int).  This is the point in which my solution differs from 
 Java.  Java would allow this to proceed, and call the base class' 
 foo(int) in all cases, which is not what the author intended.

The next update of the compiler will throw a runtime exception for this case.

How is this better than the current implementation? In the current implementation, the code compiles and creates an obscure bug because the behavior isn't what the user expects. In this new implementation, the obscure bug is only less obscure because an exception is thrown. The code still compiles properly. Why allow compilation at all?
 The second issue is how to handle the foo(char[]) case.  In the current 
 implementation, because A is not searched for overrides, the compiler 
 produces an error indicating that the user tried to call the foo(long) 
 method, but there is no implicit conversion from char[] to long.  There 
 are two possibilities.  One is that the author did not notice that 
 A.foo(char[]) existed, and intended to override all instances of foo() 
 with his foo(long) override.  However, the author is NOT notified of this 
 issue, only the user of the class is notified of this issue.  So the 
 potential for unambiguous code to have escaped exists.

But a compile time error is still generated, so I don't regard this as a big problem. The big problems are silent hijacking of code.

The problem is WHEN the compile time error is generated. I do not mind if the author of the class cannot generate object code for his class because of a compile time error (see my alternate solution). However, because the compile error is generated when someone attempts to USE the class, the error is delivered to the incorrect person. That person may not have the ability to fix the error. Silent hijacking of code is not possible with the solution I propose, so that becomes a moot point.
 The second possibility is that the author fully intended to allow the 
 base class to define foo(char[]), but forgot to define the alias.  Again, 
 since the compiler gives no error, he is unaware that he is releasing 
 buggy code to the world.  I believe the correct assumption of the 
 compiler should be that the user wanted the alias for the base class' 
 foo(char[]), and should alias it implicitly if and only if no suitable 
 match exists on the derived class.  In the case where the author did not 
 notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is 
 defined by the base class.

The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error.

I understand your point of view, and that is why I said that I would concede to a solution where a compiler error is thrown if all overloads are not handled. However, I think it is still incorrect to generate the compiler error on the use of the class rather than the compilation of the class. My preference as I said is to avoid having to specify the alias in the simple case where the arguments are not implicitly convertable, but if there must be a compilation error, I am willing to live with that.
 If a suitable match exists that is not a direct override of the base 
 class, then the issue reduces to the previous case, where an implicit 
 conversion is required, and the compiler should error out.

 There is one other possibile solution that I would be willing to concede 
 to, and that is that the compiler errors out if the base class does not 
 override all overloads of a particular method name.  This forces the user 
 to either override all overloads of the method, or define the alias. 
 This would be the safest solution, as the author of B must make his 
 intentions perfectly clear.

I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter. I prefer the solution where attempts to call unoverridden base class overloads will result in a runtime exception.

The problem is they ARE implemented, and now they will cause runtime exceptions! Also, now this breaks the contract that the base class provides. If you give me an instance of a certain class, and the documentation for that class says that it defines a given method, then that method should be implemented in all derivatives. If you want to derive a class and force the implementation of a base class' method to throw an exception, I think that should be the (for better lack of a word) exception, not the rule. Why derive from a class where you want to shoehorn its functionality into something different? I would recommend to someone trying to do that to define a new class, rather than derive. I challenge anyone to give a real-world example of why this should be possible. As you point out, it is possible to force the implementation that you are specifying by overriding the overloaded method and throwing the exception yourself, and maybe the requirement of extra cluttering functions will discourage people from implementing their code this way. Maybe there could be a specific keyword or something to specify that you want to have the overload throw an exception, so there is less code clutter, but I do not think the default should be throwing an exception. -Steve
Aug 07 2007
prev sibling parent Regan Heath <regan netmail.co.nz> writes:
Steven Schveighoffer wrote:
 So my proposal is to change the specification so that:
 
 If there is a class A, which is a base class of class B, where A defines a 
 method foo(args), and B defines a method foo(args2), such that the types of 
 args2 cannot be implicitly converted to args, and B does not define 
 foo(args), then the definition of B.foo(args) shall be implicitly aliased to 
 A.foo(args).  If, using the same assumptions, args2 can be implicitly 
 converted to args, then the compiler should fail to compile B indicating 
 that the user must define B.foo(args) by override or by alias.
 
 I believe this will give us the best of both camps, and allow much less code 
 to be released with silent bugs than the current implementation.
 
 Let the hole shooting begin...

I like it. In fact it solves the existing problem exhibited by the original example 2! Here are our original examples given by Walter and used to support the current C++ like behaviour. ----------------------------------- class X1 { void f(int); } // chain of derivations X(n) : X(n-1) class X9: X8 { void f(double); } void g(X9 p) { p.f(1); // X1.f or X9.f ? } ----------------------------------- class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } ----------------------------------- Under your proposal these would both be classify as "incorrectly implemented override"'s and fail to compile with an error. In example 1 the error would occur when compiling X9. In example 2 the error would occur when compiling D. In both cases the addition of 'alias' or an exact override for the 'int' method from the base will resolve the error. In example 1, if other overloads existed in X2 thru X8 i.e. "void f(short);" then both "void f(int);" and "void f(short);" would need to be aliased or defined in X9 to resolve the error. This seems to indicate that all base classes all the way back to the root need to be examined, that could be complex... As I mentioned earlier your proposal will solve the existing problem caused by example 2, which under both C++ and Java like implementations calls B.set(int) then D.squareit() resulting in 0 instead of 9. That, combined with the implicit alias of overloads where no implicit conversion is possible and I think it will be a feature which is both safe and intuitive. I'm interested to see what other people think. Regan
Aug 07 2007