www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - visibility vs. accessibility of protected symbols

reply "Martin Nowak" <dawg dawgfoto.de> writes:
I made a pull request to fix private/package access checks.
https://github.com/D-Programming-Language/dmd/pull/669

Now I'm wondering if that goes into the right direction.

The shallow distinction of visibility vs. accessibility breaks the module  
system because
one can't safely add a private symbol without possibly affecting every  
dependent module.
Thus we're back at using underscore names to protect from that.

 From implementing access checks I found the right amount of checking is to  
do them
when resolving identifiers. Thus it's almost equal to visibility based  
protection.

We already hide symbols that are privately imported by an imported module.

martin
Feb 12 2012
next sibling parent David Nadlinger <see klickverbot.at> writes:
On 2/12/12 7:28 PM, Martin Nowak wrote:
 The shallow distinction of visibility vs. accessibility breaks the
 module system because
 one can't safely add a private symbol without possibly affecting every
 dependent module.
 Thus we're back at using underscore names to protect from that.

Yes, and this is exactly why I argued to disregard invisible symbols during overload resolution in the past. Walter seems to be firmly convinced that the current solution is the right thing to do, but I can't recall what his reasons were. David
Feb 12 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Sun, 12 Feb 2012 19:32:28 +0100, David Nadlinger <see klickverbot.at>  
wrote:

 On 2/12/12 7:28 PM, Martin Nowak wrote:
 The shallow distinction of visibility vs. accessibility breaks the
 module system because
 one can't safely add a private symbol without possibly affecting every
 dependent module.
 Thus we're back at using underscore names to protect from that.

Yes, and this is exactly why I argued to disregard invisible symbols during overload resolution in the past. Walter seems to be firmly convinced that the current solution is the right thing to do, but I can't recall what his reasons were. David

Overloading is only the trickier part of it, but it affects all other symbols as well. As far as overloading goes C++ does it one way and Java choses the other. IMHO mixing protection levels for overloads could be disallowed.
Feb 12 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Agreed with David. I find these things a pointless waste of time:

module a;
private {
    struct foo {
    }
}

module b;
void foo() { }

import a;
import b;

void main() {
    foo();
}

main.d(7): Error: a.foo at a.d(3) conflicts with b.foo at b.d(2)
main.d(7): Error: structliteral has no effect in expression (foo())

One of those cases exists in Phobos somewhere, where a private type is
declared with the same name as an unrelated public function in some
other module. I can't recall which module it is though.
Feb 12 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Sunday, February 12, 2012 19:32:28 David Nadlinger wrote:
 On 2/12/12 7:28 PM, Martin Nowak wrote:
 The shallow distinction of visibility vs. accessibility breaks the
 module system because
 one can't safely add a private symbol without possibly affecting every
 dependent module.
 Thus we're back at using underscore names to protect from that.

Yes, and this is exactly why I argued to disregard invisible symbols during overload resolution in the past. Walter seems to be firmly convinced that the current solution is the right thing to do, but I can't recall what his reasons were.

I'm not 100% sure that it's a good idea, depending on what the exact side effects are, but in principle, I'd love it if private symbols were invisible to other modules. Now, this does _not_ fly with private being overridable, as TDPL claims it should be. But I've already argued that that was a mistake due to the performance issues that it causes. I'm not sure what Walter's current plans with that are though. http://d.puremagic.com/issues/show_bug.cgi?id=4542 - Jonathan M Davis
Feb 13 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Mon, 13 Feb 2012 19:07:20 +0100, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Sunday, February 12, 2012 19:32:28 David Nadlinger wrote:
 On 2/12/12 7:28 PM, Martin Nowak wrote:
 The shallow distinction of visibility vs. accessibility breaks the
 module system because
 one can't safely add a private symbol without possibly affecting every
 dependent module.
 Thus we're back at using underscore names to protect from that.

Yes, and this is exactly why I argued to disregard invisible symbols during overload resolution in the past. Walter seems to be firmly convinced that the current solution is the right thing to do, but I can't recall what his reasons were.

I'm not 100% sure that it's a good idea, depending on what the exact side effects are, but in principle, I'd love it if private symbols were invisible to other modules. Now, this does _not_ fly with private being overridable, as TDPL claims it should be. But I've already argued that that was a mistake due to the performance issues that it causes. I'm not sure what Walter's current plans with that are though.

final public method that needs to call the virtual private methods.
Feb 13 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 00:15:40 Martin Nowak wrote:
 Can you elaborate on what issues you see with NVI. After all it's only the
 final public
 method that needs to call the virtual private methods.

As I explain in that bug report, NVI can be acheived with protected rather than private. So, we can do NVI now with private being non-virtual. However, if we make private virtual, then the compiler can no longer inline private functions in classes unless they're marked as final. That's a performance hit to pretty much every class ever written. Yes, the conscientious programmer will mark their classes' private functions final unless they intend to use them for NVI, but the _default_ is then inefficient for little to no gain. All you gain is the ability to do NVI with private instead of protected, and that is not even vaguely worth the performance hit to almost every D class ever IMHO. - Jonathan M Davis
Feb 13 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/14/2012 12:26 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 00:15:40 Martin Nowak wrote:
 Can you elaborate on what issues you see with NVI. After all it's only the
 final public
 method that needs to call the virtual private methods.

As I explain in that bug report, NVI can be acheived with protected rather than private. So, we can do NVI now with private being non-virtual. However, if we make private virtual, then the compiler can no longer inline private functions in classes unless they're marked as final. That's a performance hit to pretty much every class ever written. Yes, the conscientious programmer will mark their classes' private functions final unless they intend to use them for NVI, but the _default_ is then inefficient for little to no gain. All you gain is the ability to do NVI with private instead of protected, and that is not even vaguely worth the performance hit to almost every D class ever IMHO. - Jonathan M Davis

Virtual private does not allow for cross-module NVI, so what would be the point anyway?
Feb 13 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/14/2012 01:28 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 00:36:31 Timon Gehr wrote:
 On 02/14/2012 12:26 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 00:15:40 Martin Nowak wrote:
 Can you elaborate on what issues you see with NVI. After all it's only
 the final public
 method that needs to call the virtual private methods.

As I explain in that bug report, NVI can be acheived with protected rather than private. So, we can do NVI now with private being non-virtual. However, if we make private virtual, then the compiler can no longer inline private functions in classes unless they're marked as final. That's a performance hit to pretty much every class ever written. Yes, the conscientious programmer will mark their classes' private functions final unless they intend to use them for NVI, but the _default_ is then inefficient for little to no gain. All you gain is the ability to do NVI with private instead of protected, and that is not even vaguely worth the performance hit to almost every D class ever IMHO. - Jonathan M Davis

Virtual private does not allow for cross-module NVI, so what would be the point anyway?

Why wouldn't it? The whole point of NVI is that you override it but don't call it - the base class does that. So, the derived class could override the private function but couldn't call it. It _is_ a bit weird though, all the same.

It would break information hiding. A private method declared in a different module is not supposed to be visible (hopefully will get fixed), let alone overrideable.
 Regardless, I'm against making private virtual because of the needless
 performance hit.

There are no performance implications because the compiler has the whole module ready to analyse. Therefore it can de-virtualise any calls to private members that are never overridden. A trivial addition to the class layout could furthermore allow the compiler to remove the vtable entry for such a member function entirely (essentially, virtual private members would get negative vtable offsets). I actually think that should be implemented in order to get a potential TDPL error off the table.
Feb 13 2012
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/14/2012 01:55 AM, Timon Gehr wrote:
 There are no performance implications because the compiler has the whole
 module ready to analyse. Therefore it can de-virtualise any calls to
 private members that are never overridden. A trivial addition to the
 class layout could furthermore allow the compiler to remove the vtable
 entry for such a member function entirely (essentially, virtual private
 members would get negative vtable offsets). I actually think that should
 be implemented in order to get a potential TDPL error off the table.

Nevermind, separate compilation could get tricky.
Feb 13 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/14/2012 02:16 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 01:55:51 Timon Gehr wrote:
 On 02/14/2012 01:28 AM, Jonathan M Davis wrote:
 It would break information hiding. A private method declared in a
 different module is not supposed to be visible (hopefully will get
 fixed), let alone overrideable.

You misunderstand then. Private symbols are most definitely visible. Access modifiers do _not_ affect visibility, only accessibility. Private symbols are visible _by design_. That's how it works in C++, and D inherited that. The whole point of this thread is to discuss whether access modifiers should affect visibility, but right now they _do_, and it's _on purpose_ that they do. It's not a bug.

It is a bug on purpose.
 As it stands, if I declare a private function which conflicts with a public
 function, the compiler will complain. For example, If I were to add a private
 abs function to std.file which took an int, then any module which imported both
 std.math and std.file would have to use std.math.abs instead of just abs,
 because there would be a conflict. The fact that std.file's abs was private
 wouldn't mean anything to overload resolution.

Borken.
 In fact, just recently, a private abs function was added to core.time for what
 it needed, and it caused problems. It had to be changed to _abs to not screw
 up std.math.abs.

Exactly what I am saying. This is laughable (!) and should vanish from Ds otherwise very sane approach at modularity.
 The whole point of this thread is to discuss whether access modifiers should be
 changed to affect visibility rather than access.

Most definitely yes. The only hindrance are implementation issues.
 Regardless, I'm against making private virtual because of the needless
 performance hit.

There are no performance implications because the compiler has the whole module ready to analyse. Therefore it can de-virtualise any calls to private members that are never overridden. A trivial addition to the class layout could furthermore allow the compiler to remove the vtable entry for such a member function entirely (essentially, virtual private members would get negative vtable offsets). I actually think that should be implemented in order to get a potential TDPL error off the table.

No. It can't. Because a derived class in an entirely different module could override the private function. private does _not_ affect visibility.

Well, not being able override what cannot be accesses is a quite basic requirement of security. Private members cannot be overriden in a different module.
 The whole idea of overriding private for NVI is to allow derived classes to
override the
 private function's implementation but not call it.

This is confusing method implementation with overriding. I just read the relevant sections of TDPL: "D fully supports NVI by providing _specific guarantees_ when *interfaces* use access specifiers." It is an explicit special case because private wouldn't have much of a meaning for interfaces otherwise. This means private interface members would not be hidden, they are special _already_.
 If private becomes virtual, then the compiler loses the ability to inline
non-final private functions
 unlessit can guarantee that it's dealing with that exact type, which it 


The optimisation works for all private class members. The issue is the virtual table layout for separate compilation with .di files, therefore it is not a good idea to make private methods virtual. But that is not required to satisfy TDPL anyway: As far as I can see TDPL only requires private *interface* members to be implementable. That is not a problem at all, interface vtbls use thunking anyway. Therefore the private members could just be plugged there and be called virtually through base interface calls and non-virtually otherwise.
Feb 13 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/14/2012 03:05 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:

 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

That article uses C++. It is not really a fair comparison, because in D there is no way to explicitly declare a method "private virtual", and D's private is different from C++'s private, and C++ has a different sub-typing and inheritance model. I think there are different design trade-offs at play. But I can make better sense of your point of view now: When I was talking about virtual I meant just a function that has an entry in a vtbl and is invoked virtually, while you probably had C++'s semantics in mind.
 The whole idea of overriding private for NVI is to allow derived classes
 to override the private function's implementation but not call it.

This is confusing method implementation with overriding. I just read the relevant sections of TDPL: "D fully supports NVI by providing _specific guarantees_ when *interfaces* use access specifiers." It is an explicit special case because private wouldn't have much of a meaning for interfaces otherwise. This means private interface members would not be hidden, they are special _already_.

The fact that TDPL talks about interfaces in specific could be a good argument for _not_ making private virtual for classes. But regardless, as it stands, _nothing_ which is private is hidden. So, there's nothing special going on with interfaces and private, except that you can't override them to actually do NVI, so they're kind of pointless.

The fact that they cannot be implemented in a class that implements the interface seems to be a [TDPL] bug.
 If private becomes virtual, then the compiler loses the ability to
 inline non-final private functions unlessit can guarantee that it's
 dealing with that exact type, which it

The optimisation works for all private class members. The issue is the virtual table layout for separate compilation with .di files, therefore it is not a good idea to make private methods virtual. But that is not required to satisfy TDPL anyway: As far as I can see TDPL only requires private *interface* members to be implementable. That is not a problem at all, interface vtbls use thunking anyway. Therefore the private members could just be plugged there and be called virtually through base interface calls and non-virtually otherwise.

The optimization only works as long as the compiler can guarantee that no derived classes override the private functions, since it doesn't necessarily know about all of the derived classes.

The optimization works under the precondition that derived classes from a different module cannot override private member functions.
 If D allowed NVI with classes the way that C++ does by making private virtual,
then derived classes would _have_ to
 be able to override private functions in base classes which are in separate
 modules; they might even be in separate libraries. And then the compiler could
 not guarantee that no derived classes overrode the private functions, and the
 optimization could not be made. Hence why I'm arguing against making private
 functions virtual.

Here you probably mean virtual as in C++. I agree.
 Personally, I'd very much like to see access level affect visibility, but
 Walter must be convinced.

I think there were some issues with overload sets, but I think overloads with differing accessibility should just be disallowed.
Feb 13 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/14/2012 03:54 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 03:45:54 Timon Gehr wrote:
 On 02/14/2012 03:05 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:

 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

That article uses C++. It is not really a fair comparison, because in D there is no way to explicitly declare a method "private virtual", and D's private is different from C++'s private, and C++ has a different sub-typing and inheritance model. I think there are different design trade-offs at play. But I can make better sense of your point of view now: When I was talking about virtual I meant just a function that has an entry in a vtbl and is invoked virtually, while you probably had C++'s semantics in mind.

I'm not aware of _any_ difference between virtual functions in C++ and D except for the fact that all public and protected member functions are always virtual in D (unless the compiler is able to determine that it can make them non- virtual), and currently private and protected functions are never virtual. What differences are you referring to? - Jonathan M Davis

The main difference is that C++ has an explicit 'virtual' modifier that naturally grants override-access to private symbols.
Feb 13 2012
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/17/12 8:03 AM, Steven Schveighoffer wrote:
 On Mon, 13 Feb 2012 21:05:35 -0500, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:

 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:

 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

This does nothing to protect the implementation. You are deluding yourself if you think you can stop a derived class from calling it's own implementation! In this case, protected is good enough, and I like how private is always non-virtual. As Timon says, it's good for security and module encapsulation. -Steve

I agree. Walter and I chalked that up to a bug in TDPL that will be superseded by a different implementation. Andrei
Feb 17 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 00:36:31 Timon Gehr wrote:
 On 02/14/2012 12:26 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 00:15:40 Martin Nowak wrote:
 Can you elaborate on what issues you see with NVI. After all it's only
 the final public
 method that needs to call the virtual private methods.

As I explain in that bug report, NVI can be acheived with protected rather than private. So, we can do NVI now with private being non-virtual. However, if we make private virtual, then the compiler can no longer inline private functions in classes unless they're marked as final. That's a performance hit to pretty much every class ever written. Yes, the conscientious programmer will mark their classes' private functions final unless they intend to use them for NVI, but the _default_ is then inefficient for little to no gain. All you gain is the ability to do NVI with private instead of protected, and that is not even vaguely worth the performance hit to almost every D class ever IMHO. - Jonathan M Davis

Virtual private does not allow for cross-module NVI, so what would be the point anyway?

Why wouldn't it? The whole point of NVI is that you override it but don't call it - the base class does that. So, the derived class could override the private function but couldn't call it. It _is_ a bit weird though, all the same. Regardless, I'm against making private virtual because of the needless performance hit. - Jonathan M Davis
Feb 13 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Tue, 14 Feb 2012 00:26:14 +0100, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Tuesday, February 14, 2012 00:15:40 Martin Nowak wrote:
 Can you elaborate on what issues you see with NVI. After all it's only  
 the
 final public
 method that needs to call the virtual private methods.

As I explain in that bug report, NVI can be acheived with protected rather than private. So, we can do NVI now with private being non-virtual. However, if we make private virtual, then the compiler can no longer inline private functions in classes unless they're marked as final. That's a performance hit to pretty much every class ever written. Yes, the conscientious programmer will mark their classes' private functions final unless they intend to use them for NVI, but the _default_ is then inefficient for little to no gain. All you gain is the ability to do NVI with private instead of protected, and that is not even vaguely worth the performance hit to almost every D class ever IMHO. - Jonathan M Davis

OK, that was known and I agree with you. I even find the concept of overriding private functions flawed. I thought you had something in mind with overload resolution.
Feb 13 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 01:55:51 Timon Gehr wrote:
 On 02/14/2012 01:28 AM, Jonathan M Davis wrote:
 It would break information hiding. A private method declared in a
 different module is not supposed to be visible (hopefully will get
 fixed), let alone overrideable.

You misunderstand then. Private symbols are most definitely visible. Access modifiers do _not_ affect visibility, only accessibility. Private symbols are visible _by design_. That's how it works in C++, and D inherited that. The whole point of this thread is to discuss whether access modifiers should affect visibility, but right now they _do_, and it's _on purpose_ that they do. It's not a bug. As it stands, if I declare a private function which conflicts with a public function, the compiler will complain. For example, If I were to add a private abs function to std.file which took an int, then any module which imported both std.math and std.file would have to use std.math.abs instead of just abs, because there would be a conflict. The fact that std.file's abs was private wouldn't mean anything to overload resolution. In fact, just recently, a private abs function was added to core.time for what it needed, and it caused problems. It had to be changed to _abs to not screw up std.math.abs. The whole point of this thread is to discuss whether access modifiers should be changed to affect visibility rather than access.
 Regardless, I'm against making private virtual because of the needless
 performance hit.

There are no performance implications because the compiler has the whole module ready to analyse. Therefore it can de-virtualise any calls to private members that are never overridden. A trivial addition to the class layout could furthermore allow the compiler to remove the vtable entry for such a member function entirely (essentially, virtual private members would get negative vtable offsets). I actually think that should be implemented in order to get a potential TDPL error off the table.

No. It can't. Because a derived class in an entirely different module could override the private function. private does _not_ affect visibility. The whole idea of overriding private for NVI is to allow derived classes to override the private function's implementation but not call it. If private becomes virtual, then the compiler loses the ability to inline non-final private functions unless it can guarantee that it's dealing with that exact type, which it usually can't do in the type's own member functions. - Jonathan M Davis
Feb 13 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:

 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm
 The whole idea of overriding private for NVI is to allow derived classes
 to override the private function's implementation but not call it.

This is confusing method implementation with overriding. I just read the relevant sections of TDPL: "D fully supports NVI by providing _specific guarantees_ when *interfaces* use access specifiers." It is an explicit special case because private wouldn't have much of a meaning for interfaces otherwise. This means private interface members would not be hidden, they are special _already_.

The fact that TDPL talks about interfaces in specific could be a good argument for _not_ making private virtual for classes. But regardless, as it stands, _nothing_ which is private is hidden. So, there's nothing special going on with interfaces and private, except that you can't override them to actually do NVI, so they're kind of pointless.
 If private becomes virtual, then the compiler loses the ability to
 inline non-final private functions unlessit can guarantee that it's
 dealing with that exact type, which it

The optimisation works for all private class members. The issue is the virtual table layout for separate compilation with .di files, therefore it is not a good idea to make private methods virtual. But that is not required to satisfy TDPL anyway: As far as I can see TDPL only requires private *interface* members to be implementable. That is not a problem at all, interface vtbls use thunking anyway. Therefore the private members could just be plugged there and be called virtually through base interface calls and non-virtually otherwise.

The optimization only works as long as the compiler can guarantee that no derived classes override the private functions, since it doesn't necessarily know about all of the derived classes. If D allowed NVI with classes the way that C++ does by making private virtual, then derived classes would _have_ to be able to override private functions in base classes which are in separate modules; they might even be in separate libraries. And then the compiler could not guarantee that no derived classes overrode the private functions, and the optimization could not be made. Hence why I'm arguing against making private functions virtual. Personally, I'd very much like to see access level affect visibility, but Walter must be convinced. - Jonathan M Davis
Feb 13 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 03:45:54 Timon Gehr wrote:
 On 02/14/2012 03:05 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:
 
 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

That article uses C++. It is not really a fair comparison, because in D there is no way to explicitly declare a method "private virtual", and D's private is different from C++'s private, and C++ has a different sub-typing and inheritance model. I think there are different design trade-offs at play. But I can make better sense of your point of view now: When I was talking about virtual I meant just a function that has an entry in a vtbl and is invoked virtually, while you probably had C++'s semantics in mind.

I'm not aware of _any_ difference between virtual functions in C++ and D except for the fact that all public and protected member functions are always virtual in D (unless the compiler is able to determine that it can make them non- virtual), and currently private and protected functions are never virtual. What differences are you referring to? - Jonathan M Davis
Feb 13 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 04:39:18 Timon Gehr wrote:
 On 02/14/2012 03:54 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 03:45:54 Timon Gehr wrote:
 On 02/14/2012 03:05 AM, Jonathan M Davis wrote:
 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:
 
 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

That article uses C++. It is not really a fair comparison, because in D there is no way to explicitly declare a method "private virtual", and D's private is different from C++'s private, and C++ has a different sub-typing and inheritance model. I think there are different design trade-offs at play. But I can make better sense of your point of view now: When I was talking about virtual I meant just a function that has an entry in a vtbl and is invoked virtually, while you probably had C++'s semantics in mind.

I'm not aware of _any_ difference between virtual functions in C++ and D except for the fact that all public and protected member functions are always virtual in D (unless the compiler is able to determine that it can make them non- virtual), and currently private and protected functions are never virtual. What differences are you referring to? - Jonathan M Davis

The main difference is that C++ has an explicit 'virtual' modifier that naturally grants override-access to private symbols.

In other words, the only difference is what I said - that public and protected in D are automatically virtual, and private and protected are automatically non-virtual. So, assuming that private were made virtual, it would be the same as a private virtual function in C++ as far as virtuality goes. The only differences would be that D effectively makes all of the functions in the rest of the module friends and has no other way of declaring friends. Regardless, I think that we agree that it would be better if private were non- virtual and access level also affected visibility so that private functions were not visible outside of the module that they're declared in. - Jonathan M Davis
Feb 13 2012
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
 private and protected are automatically non-virtual

You've said that twice now, I think you mean private and package.
Feb 14 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, February 14, 2012 21:13:46 Daniel Murphy wrote:
 private and protected are automatically non-virtual

You've said that twice now, I think you mean private and package.

Yes. You're right. The excess of p's is confusing me, I guess. public and protected are automatically virtual. private and package are automatically non-virtual. - Jonathan M Davis
Feb 14 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 13 Feb 2012 21:05:35 -0500, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:

 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

This does nothing to protect the implementation. You are deluding yourself if you think you can stop a derived class from calling it's own implementation! In this case, protected is good enough, and I like how private is always non-virtual. As Timon says, it's good for security and module encapsulation. -Steve
Feb 17 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, February 17, 2012 10:18:32 Andrei Alexandrescu wrote:
 I agree. Walter and I chalked that up to a bug in TDPL that will be
 superseded by a different implementation.

Good to hear! - Jonathan M Davis
Feb 17 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, February 17, 2012 09:03:32 Steven Schveighoffer wrote:
 On Mon, 13 Feb 2012 21:05:35 -0500, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:
 
 Well, not being able override what cannot be accesses is a quite basic
 requirement of security. Private members cannot be overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

This does nothing to protect the implementation. You are deluding yourself if you think you can stop a derived class from calling it's own implementation! In this case, protected is good enough, and I like how private is always non-virtual. As Timon says, it's good for security and module encapsulation.

I know. That's part of why I'm arguing that protected is enough and that private shouldn't be virtual. But pretty much the whole point of doing NVI with private is the theory that you want to guarantee that the derived classes can't call the function. The fact that you can't actually guarantee that just makes the arguments for making private virtual for NVI that much weaker. But Timon's arguments make it sound like he doesn't understand what NVI is, which was the point of my post. - Jonathan M Davis
Feb 17 2012
prev sibling next sibling parent "Timon Gehr" <timon.gehr gmx.ch> writes:
On Friday, 17 February 2012 at 18:35:38 UTC, Jonathan M Davis 
wrote:
 On Friday, February 17, 2012 09:03:32 Steven Schveighoffer 
 wrote:
 On Mon, 13 Feb 2012 21:05:35 -0500, Jonathan M Davis 
 <jmdavisProg gmx.com>
 
 wrote:
 On Tuesday, February 14, 2012 02:43:29 Timon Gehr wrote:
 On 02/14/2012 02:16 AM, Jonathan M Davis wrote:
 
 Well, not being able override what cannot be accesses is a 
 quite basic
 requirement of security. Private members cannot be 
 overriden in a
 different module.

Have you ever read up on NVI? That's how it's supposed to work. The whole idea is to have derived classes override functions without being able to use them. It makes it so that the public function is non-virtual and contains whatever stuff you want to guarantee is called before or after the private function, which is then virtual. http://www.gotw.ca/publications/mill18.htm

This does nothing to protect the implementation. You are deluding yourself if you think you can stop a derived class from calling it's own implementation! In this case, protected is good enough, and I like how private is always non-virtual. As Timon says, it's good for security and module encapsulation.

I know. That's part of why I'm arguing that protected is enough and that private shouldn't be virtual. But pretty much the whole point of doing NVI with private is the theory that you want to guarantee that the derived classes can't call the function. The fact that you can't actually guarantee that just makes the arguments for making private virtual for NVI that much weaker. But Timon's arguments make it sound like he doesn't understand what NVI is, which was the point of my post. - Jonathan M Davis

I understand what NVI is. It is just that private methods overridable from a different module don't have many merits, regardless of how NVI is apparently sometimes implemented in C++. The fact that C++ has overridable private methods is quite likely just a side-effect of wanting to give some kind of semantics to a method declared as both private and virtual. I could argue that it sounds like you don't understand what NVI is because you assume C++'s private virtual semantics are required for the implementation of NVI. ;) As far as information hiding design goes, we certainly don't want to borrow heavily from C++. Its system is unsound.
Feb 17 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, February 17, 2012 20:02:53 Timon Gehr wrote:
 I understand what NVI is. It is just that private methods
 overridable from a different module don't have many merits,
 regardless of how NVI is apparently sometimes implemented in C++.
 The fact that C++ has overridable private methods is quite likely
 just a side-effect of wanting to give some kind of semantics to a
 method declared as both private and virtual. I could argue that
 it sounds like you don't understand what NVI is because you
 assume C++'s private virtual semantics are required for the
 implementation of NVI. ;)

It's required if you implement NVI the way that's typical in C++ and the way that is requested in the NVI-related bug report. You can get close enough to the same effect using protected, so it's arguably not necessary (which is one of the reasons that I'm arguing against making private virtual), but if we're talking about NVI in the traditional sense (which is what the bug report is looking for and TDPL indicates that you can do in its discussion on NVI), it _is_ required. Fortunately, it looks like Andrei and Walter have agreed that private shouldn't be made virtual, regardless what TDPL says - though what that means for NVI and interfaces, I don't know. Protected may be enough in that case as well. - Jonathan M Davis
Feb 17 2012