www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - traits getProtection

reply "Adam D. Ruppe" <destructionator gmail.com> writes:
I've prepared a dmd pull request to add a new __trait:
getProtection.

It is meant to be used along with getMember() to add to
the reflection capabilities, letting us use the existing
protection qualifiers as strings.

 From the test:
==
class Test {
     public int a;
     private int b;
     export string c;
     protected int d;
     package void e() {}
}

void main() {
     Test t;
     static assert(__traits(getProtection, __traits(getMember, t, 
"a")) == "public");
     static assert(__traits(getProtection, __traits(getMember, t, 
"b")) == "private");
     static assert(__traits(getProtection, __traits(getMember, t, 
"c")) == "export");
     static assert(__traits(getProtection, __traits(getMember, t, 
"d")) == "protected");
     static assert(__traits(getProtection, __traits(getMember, t, 
"e")) == "package");
}
==



This will help D automatically generate things like
external interfaces that use the protections.

For instance, I plan to use it in my web.d to only
make functions marked "export" available via the
web interface. Currently, you have to use a naming
convention to hide functions - a leading underscore -
even on private members. This is ok, but not great.

But with the protection trait, we can mark it with
a much more natural "private", or any of the other
specifiers D has.


I'm sure other uses will come up too.
Apr 01 2012
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
Code: https://github.com/D-Programming-Language/dmd/pull/856
Apr 01 2012
prev sibling next sibling parent James Miller <james aatch.net> writes:
On 2 April 2012 06:27, Adam D. Ruppe <destructionator gmail.com> wrote:
 I've prepared a dmd pull request to add a new __trait:
 getProtection.

 It is meant to be used along with getMember() to add to
 the reflection capabilities, letting us use the existing
 protection qualifiers as strings.

 From the test:
 =3D=3D
 class Test {
 =C2=A0 =C2=A0public int a;
 =C2=A0 =C2=A0private int b;
 =C2=A0 =C2=A0export string c;
 =C2=A0 =C2=A0protected int d;
 =C2=A0 =C2=A0package void e() {}
 }

 void main() {
 =C2=A0 =C2=A0Test t;
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=

 "public");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=

 "private");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=

 "export");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=

 "protected");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=

 "package");
 }
 =3D=3D



 This will help D automatically generate things like
 external interfaces that use the protections.

 For instance, I plan to use it in my web.d to only
 make functions marked "export" available via the
 web interface. Currently, you have to use a naming
 convention to hide functions - a leading underscore -
 even on private members. This is ok, but not great.

 But with the protection trait, we can mark it with
 a much more natural "private", or any of the other
 specifiers D has.


 I'm sure other uses will come up too.

Looks good, adds a lot to the compile-time reflection capabilities. My thoughts for it are similar to yours, generating bindings from "export"ed members, so you don't need to maintain separate lists or use a naming convention. -- James Miller
Apr 01 2012
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01.04.2012 22:27, Adam D. Ruppe wrote:
 I've prepared a dmd pull request to add a new __trait:
 getProtection.

 It is meant to be used along with getMember() to add to
 the reflection capabilities, letting us use the existing
 protection qualifiers as strings.

  From the test:
 ==
 class Test {
 public int a;
 private int b;
 export string c;
 protected int d;
 package void e() {}
 }

 void main() {
 Test t;
 static assert(__traits(getProtection, __traits(getMember, t, "a")) ==
 "public");
 static assert(__traits(getProtection, __traits(getMember, t, "b")) ==
 "private");
 static assert(__traits(getProtection, __traits(getMember, t, "c")) ==
 "export");
 static assert(__traits(getProtection, __traits(getMember, t, "d")) ==
 "protected");
 static assert(__traits(getProtection, __traits(getMember, t, "e")) ==
 "package");
 }
 ==



 This will help D automatically generate things like
 external interfaces that use the protections.

 For instance, I plan to use it in my web.d to only
 make functions marked "export" available via the
 web interface. Currently, you have to use a naming
 convention to hide functions - a leading underscore -
 even on private members. This is ok, but not great.

 But with the protection trait, we can mark it with
 a much more natural "private", or any of the other
 specifiers D has.


 I'm sure other uses will come up too.

It's all nice and well, but I believe part of the reason of say private protection is that user is never ever able to see(!) it. Thus it user can't depend on private members being there, which is a good thing. If I read it right, the technique you present allows user code to depend on private functions being there. I argue that we shouldn't even provide a _possibility_ for external stuff to depend on private members. Same argument in limited scope goes for protected. As for export, I thinks it looks OK. -- Dmitry Olshansky
Apr 02 2012
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 02.04.2012 16:04, Adam D. Ruppe wrote:
 On Monday, 2 April 2012 at 09:26:07 UTC, Dmitry Olshansky wrote:
 It's all nice and well, but I believe part of the reason of say
 private protection is that user is never ever able to see(!) it. Thus
 it user can't depend on private members being there, which is a good
 thing.

That's not the way it is right now in D: if you use the reflection traits, you can see and access private members. Worse yet, the way it is now, there's no way to tell they are marked private when using reflection!

Awful.
 That's one of the reasons why I want to add this - it
 lets us check.


 I remember asking if private members should be hidden from
 reflection before, and the answer was no - I think it had
 to do with them being useful for serialization and stuff
 like that.

Now I get it, traits hasIndirections would need it too, and it's too big a feat to lose it. -- Dmitry Olshansky
Apr 02 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 03:04, Robert Jacques a ťcrit :
 On Tue, 03 Apr 2012 08:30:25 -0500, Adam D. Ruppe
 <destructionator gmail.com> wrote:

 On Tuesday, 3 April 2012 at 13:14:00 UTC, Robert Jacques wrote:
 As someone who has implemented a runtime reflection library in
 D, it is entirely possible to detect whether a function is
 marked private/protected or not using __traits today.

How did you do it? __traits(compiles) is the best I could find, and that breaks down if you are friends (easy to happen with mixins, even if the code is in a separate file) and is kinda fragile in general since it will swallow unrelated errors too.

I did it by having all the inspection routines in a private template inside a struct in another module.

That is neat. It should be provided as lib by std.traits . And no need for language changes.
Apr 04 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 15:52, Adam D. Ruppe a écrit :
 On Wednesday, 4 April 2012 at 07:05:59 UTC, deadalnix wrote:
 It should be provided as lib by std.traits . And no need for language
 changes.

It wouldn't account for the case I actually want: checking for export.

Good point. You are right, it is needed in reflection.
Apr 06 2012
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 02/04/2012 19:10, Jonathan M Davis a écrit :
 On Monday, April 02, 2012 13:26:05 Dmitry Olshansky wrote:
 It's all nice and well, but I believe part of the reason of say private
 protection is that user is never ever able to see(!) it. Thus it user
 can't depend on private members being there, which is a good thing.
 If I read it right, the technique you present allows user code to depend
 on private functions being there.
 I argue that we shouldn't even provide a _possibility_ for external
 stuff to depend on private members.
 Same argument in limited scope goes for protected.

As it stands, private has _no_ effect on symbol visibility. All it affects is symbol accessibility. For instance, if you create a private alias in a module, it affects every module that imports your module, or if you create a function which causes an overload conflict, it creates an overload conflict regardless of whether it's private or not. C++ is the same way. Access modifiers are just that, _access_ modifieres. They _only_ affect accessibility, not visibility.

Reflection should reflect what the reflectee is. Private stuff are part of the reflectee.
 Now, there are a number of people very unhappy about this state of affairs and
 want private to hide symbols as well (personally, I think that the fact that
 it makes private aliases effectively useless is reason enough to seriously
 reconsider the current behavior), but I don't know if there's any real chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossible if symbol is inaccessible. NVI for instance would be impossible in such a situation.
Apr 02 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/02/2012 10:59 PM, Simen Kjærås wrote:
 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com> wrote:

 Now, there are a number of people very unhappy about this state of
 affairs and
 want private to hide symbols as well (personally, I think that the
 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any
 real chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossible if symbol is inaccessible. NVI for instance would be impossible in such a situation.


Making private symbols invisible to other modules or at least excluding them from symbol clashes is necessary. The current behaviour is not useful or desirable in any way.
 NVI is perfectly possible with protected.

Exactly. Furthermore, private implies final anyway.
Apr 02 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 03/04/2012 00:00, Timon Gehr a écrit :
 Making private symbols invisible to other modules or at least excluding
 them from symbol clashes is necessary. The current behaviour is not
 useful or desirable in any way.

This smells like religious coding to me.
 NVI is perfectly possible with protected.

Exactly. Furthermore, private implies final anyway.

Which is complete bullshit. Even in a module you may want to override a method.
Apr 02 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 04/03/2012 08:27 AM, deadalnix wrote:
 Le 03/04/2012 00:00, Timon Gehr a écrit :
 Making private symbols invisible to other modules or at least excluding
 them from symbol clashes is necessary. The current behaviour is not
 useful or desirable in any way.

This smells like religious coding to me.

Nope.
 NVI is perfectly possible with protected.

Exactly. Furthermore, private implies final anyway.

Which is complete bullshit.

It is a questionable design decision. It might be reasonable to allow overriding private members, but that would mess with the vtable layout.
 Even in a module you may want to override a
 method.

... which is excluded from the public interface. There are better ways to hide a method than to mark it private in an implementation class.
Apr 03 2012
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 02/04/2012 22:59, Simen Kjærås a écrit :
 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com> wrote:

 Now, there are a number of people very unhappy about this state of
 affairs and
 want private to hide symbols as well (personally, I think that the
 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any
 real chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossible if symbol is inaccessible. NVI for instance would be impossible in such a situation.

NVI is perfectly possible with protected.

You'll loose the ability to define a function, without being able to call it. It is something you want if function have to be called by pair for example (using protected here would introduce temporal coupling). It make even more sense if you consider combining it with contract programing.
Apr 02 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 04:48, Jonathan M Davis a écrit :
 On Tuesday, April 03, 2012 08:23:49 deadalnix wrote:
 Le 02/04/2012 22:59, Simen Kjærås a écrit :
 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix<deadalnix gmail.com>  wrote:
 Now, there are a number of people very unhappy about this state of
 affairs and
 want private to hide symbols as well (personally, I think that the
 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any
 real chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossible if symbol is inaccessible. NVI for instance would be impossible in such a situation.

NVI is perfectly possible with protected.

You'll loose the ability to define a function, without being able to call it.

Except that that doesn't even actually work as discussed in this thread: http://www.digitalmars.com/d/archives/digitalmars/D/Re_Module- level_accessibility_118209.html#N118329 The derived class overrode the function, so it can call it. It may not be able to call the base class version, but it can call its own.

It shouldn't be able to do so. This should be reserved for protected methods.
 Sure, NVI is great, but it works just as well with protected. private doesn't
 actually prevent you from calling the function in the derived class, and even
 if it did, it's _not_ worth the cost of making private virtual by default. And
 as D lets the compiler control virtuality, unlike C++, it doesn't make any
 sense to make it so that the programmer can specifically make a private
 function virtual to work with NVI. So, it just makes more sense to use
 protected to do NVI.

Visibility and virtuality are 2 completely orthogonal concerns. Mixing both is a bad design decision, what ever is the rational behind it. Separation of concerns is more important. Plus, this isn't a real issue, because the final keyword exists. At the end, for performance concerns, what we want is that the compiler or the linker were able to finalize methods that have no override, not some dirty trick that break larger more important conception principles.
 Now, it may be different with interfaces. TDPL specifically talks about using
 private for NVI with _interfaces_, not classes. Doing that sort of thing with
 interfaces requires special treatment already, and it doesn't affect efficiency
 like making private always virtual would, so that should be okay. In the
 general case though, it's just far better to use protected to do NVI with
 classes and let private be non-virtual and therefore efficient by default
rather
 than inefficient by default.

 - Jonathan M Davis

It is not that simple. First, it introduce an inconsistency between interfaces and classes for no real reasons. The only difference between classes and interfaces should be that interface cannot have member data, and you can inherit from multiple interfaces. Interface have been created to solve the problems that exists for multiple inheritance, and that is enough to solve that problem. Everything else is, again, lack of separation of concerns. Second, the same method can be declared in the base class and in an interface, and in this case, we cause compile error for nothing.
Apr 04 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 10:41, Jonathan M Davis a écrit :
 Plus, this isn't a real issue, because the final keyword exists.

It's a _huge_ issue, because it means that nearly every single private function in a class in D will need to be marked with final in order for it to be non-virtual and inlinable. The default becomes inefficient, and all just for NVI.

OK, let me restate this. This isn't a language design issue, this is an implementation issue. Java and C# finalize function for ages. See below.
 At the end, for performance concerns, what we want is that the compiler
 or the linker were able to finalize methods that have no override, not
 some dirty trick that break larger more important conception principles.

Because of how the compilation model works, that's impossible. No class can know all of its derived classes. Not only are classes compiled completely independently of their derived classes, but derived classes could be linked in dynamically at runtime, and those classes could have been written long after the base class was written and compiled. So, unless the programmer explicitly marks a function as final, there's no way that the compiler can know that that function won't be overridden.

Unless something is marked as export, no code can be linked to it at runtime. So the linker could finalize any function that isn't export - a vast majority. It doesn't, I admit. But this is definitively a implementation issue, and fixing it by language design decision is a mistake.
 It is not that simple. First, it introduce an inconsistency between
 interfaces and classes for no real reasons. The only difference between
 classes and interfaces should be that interface cannot have member data,
 and you can inherit from multiple interfaces. Interface have been
 created to solve the problems that exists for multiple inheritance, and
 that is enough to solve that problem. Everything else is, again, lack of
 separation of concerns.

private is _not_ going to become virtual in classes. I believe that Walter has stated that it will remain non-virtual (though I'd have to dig through the newsgroup for the exact quote). The only question is how to handle it in interfaces, and as far as I know, the intention is to make interfaces follow TDPL with regards to private.

I know, but Andrei seems to disagree. I also have to dig, to find it, but I guess we can trust each other on that. In general, I'm not convinced by the authoritative argument. Both Andrei and Walter are skilled persons, but they can be wrong, especially when they don't agree. If I restate the question, what is the benefit of having classes and interfaces behaving differently on this point ? TDPL make it pretty clear how it should be handled in interfaces. Classes should work the same way. Fixing an implementation issue isn't a valid reason.
 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.

Why would there be a compilation error? The base class cannot call the interface's private function, so any reference to that function would be the base class' function.

You have a class C and an interface I, both define the private method foo. In MyClass : C, I, if I define foo, I get an error because foo isn't virtual in C, and if I don't, I don't implement I.
Apr 04 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 19:43, Jonathan M Davis a écrit :
 In Posix-land, _everything_ is exported. AFAK, this whole idea of marking
 functions as exported or not is a Windows-only thing. And honestly, it's one
 of my biggest complaints about programming in Windows. It's seriously annoying
 to have to deal with constantly exporting stuff on Windows. On Linux, it
 _just works_. IMHO, shared libraries is one area where Windows seriously
 dropped the ball in comparison to *nix. In my experience, there are _far_
 fewer issues with them on Linux than Windows with its export and its nonsense
 of having to link against a specific version of a static library to be able
 to link against a dynamic one.

New version of gcc implemented extensions to not export everything in posix world. IIRC, clang provide this as well. Again, this is an implementation problem and shouldn't be fixed with language design decisions.
 Regardless, export is a Windows-only thing and is completely unusable for any
 kind of optimizations in the general case.

This is why extensions are made for C/C++ compilers on posix systems.
 Allowing private in interfaces is already getting weird, because of what
 interfaces are for and how they work, and doing so is already arguably special
 casing things. Certainly, the reasons for having private in interfaces is
 _completely_ different from having it in classes. private in interfaces is
 _only_ intended for NVI, whereas NVI is _not_ the reason for having private in
 classes, and in fact is fairly rare thing to do with private. I don't think
 that it's _possible_ to overload private in any language other than C++, and I
 believe that a large number of C++ programmers are completely unaware that
 it's possible in C++. Making private in classes work the same way as it does
 in interfaces would be detrimental to classes just for an idiom which many
 programmers _never_ use (though it _is_ a useful one) - and one which works
 just fine with protected.

I understand your point about this being weird in interfaces. Now, granted that it is likely to be like this in interfaces, let's make things consistent.
 I don't believe that it's currently the plan, but I don't see any problem with
 just making it so that protected is used in interfaces for NVI instead of
 private. As I already pointed out, in C++, the derived classes can already
 call their versions of the NVI functions, since they're private to _them_, not
 the base class, even if they can't call the base class implementation (if
 there even is one), but interfaces don't _have_ a base class implementation,
 so there's nothing to protect from being called anyway. So, using protected
 wouldn't really change anything, and it would be completely consistent with
 how protected is used normally.

I have to agree. The difference, as stated in TDPL, is that you cannot call the private one, only override it.
 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.

Why would there be a compilation error? The base class cannot call the interface's private function, so any reference to that function would be the base class' function.

You have a class C and an interface I, both define the private method foo. In MyClass : C, I, if I define foo, I get an error because foo isn't virtual in C, and if I don't, I don't implement I.

All it has to do is ignore C's private function in MyClass, because it's non- virtual and _can't_ be overloaded.

This is possible, but this have quite a lot of implication, on reflection for instance. What about both being defined in the same module, where private stuff are accessible.
Apr 05 2012
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 2 April 2012 at 09:26:07 UTC, Dmitry Olshansky wrote:
 It's all nice and well, but I believe part of the reason of say 
 private protection is that user is never ever able to see(!) 
 it. Thus it user can't depend on private members being there, 
 which is a good thing.

That's not the way it is right now in D: if you use the reflection traits, you can see and access private members. Worse yet, the way it is now, there's no way to tell they are marked private when using reflection! That's one of the reasons why I want to add this - it lets us check. I remember asking if private members should be hidden from reflection before, and the answer was no - I think it had to do with them being useful for serialization and stuff like that.
Apr 02 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, April 02, 2012 13:26:05 Dmitry Olshansky wrote:
 It's all nice and well, but I believe part of the reason of say private
 protection is that user is never ever able to see(!) it. Thus it user
 can't depend on private members being there, which is a good thing.
 If I read it right, the technique you present allows user code to depend
 on private functions being there.
 I argue that we shouldn't even provide a _possibility_ for external
 stuff to depend on private members.
 Same argument in limited scope goes for protected.

As it stands, private has _no_ effect on symbol visibility. All it affects is symbol accessibility. For instance, if you create a private alias in a module, it affects every module that imports your module, or if you create a function which causes an overload conflict, it creates an overload conflict regardless of whether it's private or not. C++ is the same way. Access modifiers are just that, _access_ modifieres. They _only_ affect accessibility, not visibility. Now, there are a number of people very unhappy about this state of affairs and want private to hide symbols as well (personally, I think that the fact that it makes private aliases effectively useless is reason enough to seriously reconsider the current behavior), but I don't know if there's any real chance of convincing Walter or not. - Jonathan M Davis
Apr 02 2012
prev sibling next sibling parent =?utf-8?Q?Simen_Kj=C3=A6r=C3=A5s?= <simen.kjaras gmail.com> writes:
On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com> wrote:

 Now, there are a number of people very unhappy about this state of  
 affairs and
 want private to hide symbols as well (personally, I think that the fact  
 that
 it makes private aliases effectively useless is reason enough to  
 seriously
 reconsider the current behavior), but I don't know if there's any real  
 chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossible if symbol is inaccessible. NVI for instance would be impossible in such a situation.

NVI is perfectly possible with protected.
Apr 02 2012
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Mon, 02 Apr 2012 08:05:09 -0500, Dmitry Olshansky <dmitry.olsh gmail.com>
wrote:

 On 02.04.2012 16:04, Adam D. Ruppe wrote:
 On Monday, 2 April 2012 at 09:26:07 UTC, Dmitry Olshansky wrote:
 It's all nice and well, but I believe part of the reason of say
 private protection is that user is never ever able to see(!) it. Thus
 it user can't depend on private members being there, which is a good
 thing.

That's not the way it is right now in D: if you use the reflection traits, you can see and access private members. Worse yet, the way it is now, there's no way to tell they are marked private when using reflection!

Awful.

As someone who has implemented a runtime reflection library in D, it is entirely possible to detect whether a function is marked private/protected or not using __traits today. In fact, IIRC, there is no way to break protection using __traits; what the op is referring to is .tupleof, which can be used to break protections of fields if you have the full source code.
Apr 03 2012
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 3 April 2012 at 13:14:00 UTC, Robert Jacques wrote:
 As someone who has implemented a runtime reflection library in 
 D, it is entirely possible to detect whether a function is 
 marked private/protected or not using __traits today.

How did you do it? __traits(compiles) is the best I could find, and that breaks down if you are friends (easy to happen with mixins, even if the code is in a separate file) and is kinda fragile in general since it will swallow unrelated errors too.
Apr 03 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 4/3/12, Adam D. Ruppe <destructionator gmail.com> wrote:
 is kinda fragile in general since it will swallow unrelated
 errors too.

That's an issue I've noticed recently. msgpack-d allows me to define a custom serialization routine in my structs/classes, but it has to be a template. Then msgpack tries to instantiate it via __traits(compiles), and if it doesn't work it just skips calling the routine alltogether. So if I have invalid code in my template I'll never know because the external library just swallows the errors. It took me a long time to figure out why my routine wasn't called. I really wish DMD would semantically check templates at least partially even if they're never instantiated.
Apr 03 2012
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 03 Apr 2012 08:30:25 -0500, Adam D. Ruppe <destructionator gmail.com>
wrote:

 On Tuesday, 3 April 2012 at 13:14:00 UTC, Robert Jacques wrote:
 As someone who has implemented a runtime reflection library in
 D, it is entirely possible to detect whether a function is
 marked private/protected or not using __traits today.

How did you do it? __traits(compiles) is the best I could find, and that breaks down if you are friends (easy to happen with mixins, even if the code is in a separate file) and is kinda fragile in general since it will swallow unrelated errors too.

I did it by having all the inspection routines in a private template inside a struct in another module.
Apr 03 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, April 03, 2012 08:23:49 deadalnix wrote:
 Le 02/04/2012 22:59, Simen Kj=C3=A6r=C3=A5s a =C3=A9crit :
 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com>=


 Now, there are a number of people very unhappy about this state o=




 affairs and
 want private to hide symbols as well (personally, I think that th=




 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any=




 real chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossible=



 symbol is inaccessible.
=20
 NVI for instance would be impossible in such a situation.

NVI is perfectly possible with protected.

You'll loose the ability to define a function, without being able to call it.

Except that that doesn't even actually work as discussed in this thread= : http://www.digitalmars.com/d/archives/digitalmars/D/Re_Module- level_accessibility_118209.html#N118329 The derived class overrode the function, so it can call it. It may not = be able=20 to call the base class version, but it can call its own. Sure, NVI is great, but it works just as well with protected. private d= oesn't=20 actually prevent you from calling the function in the derived class, an= d even=20 if it did, it's _not_ worth the cost of making private virtual by defau= lt. And=20 as D lets the compiler control virtuality, unlike C++, it doesn't make = any=20 sense to make it so that the programmer can specifically make a private= =20 function virtual to work with NVI. So, it just makes more sense to use=20= protected to do NVI. Now, it may be different with interfaces. TDPL specifically talks about= using=20 private for NVI with _interfaces_, not classes. Doing that sort of thin= g with=20 interfaces requires special treatment already, and it doesn't affect ef= ficiency=20 like making private always virtual would, so that should be okay. In th= e=20 general case though, it's just far better to use protected to do NVI wi= th=20 classes and let private be non-virtual and therefore efficient by defau= lt rather=20 than inefficient by default. - Jonathan M Davis
Apr 03 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, April 04, 2012 09:04:41 deadalnix wrote:
 Le 04/04/2012 04:48, Jonathan M Davis a =C3=A9crit :
 On Tuesday, April 03, 2012 08:23:49 deadalnix wrote:
 Le 02/04/2012 22:59, Simen Kj=C3=A6r=C3=A5s a =C3=A9crit :
 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix<deadalnix gmail.com=




 Now, there are a number of people very unhappy about this state=






 affairs and
 want private to hide symbols as well (personally, I think that =






 fact that
 it makes private aliases effectively useless is reason enough t=






 seriously
 reconsider the current behavior), but I don't know if there's a=






 real chance
 of convincing Walter or not.

This would be a huge mistake. For instance, private method are sometime meant to be overridden in subclasses, which is impossib=





 symbol is inaccessible.
=20
 NVI for instance would be impossible in such a situation.

NVI is perfectly possible with protected.

You'll loose the ability to define a function, without being able =



 call it.

Except that that doesn't even actually work as discussed in this th=


=20
 http://www.digitalmars.com/d/archives/digitalmars/D/Re_Module-
 level_accessibility_118209.html#N118329
=20
 The derived class overrode the function, so it can call it. It may =


 able to call the base class version, but it can call its own.

It shouldn't be able to do so. This should be reserved for protected methods.

Well, that's the way that it works in C++. And it makes sense if you th= ink=20 about it. The derived class' method is private to _it_, not the base cl= ass, so=20 it can still call it, even if it can't call the base class version.
 Sure, NVI is great, but it works just as well with protected. priva=


 doesn't actually prevent you from calling the function in the deriv=


 class, and even if it did, it's _not_ worth the cost of making priv=


 virtual by default. And as D lets the compiler control virtuality, =


 C++, it doesn't make any sense to make it so that the programmer ca=


 specifically make a private function virtual to work with NVI. So, =


 just makes more sense to use protected to do NVI.

Visibility and virtuality are 2 completely orthogonal concerns. Mixin=

 both is a bad design decision, what ever is the rational behind it.
 Separation of concerns is more important.

They are and they aren't. There are major issues in allowing the overri= ding of=20 non-virtual functions. It's far less error-prone to simply make it so t= hat a=20 class' member functions are always virtual unless the compiler can dete= rmine=20 that it can make them non-virtual (which can only happen when they're f= inal and=20 don't override anything). D took the same route as Java in this regard.= And=20 because it took that route, the programmer has _zero_ control over the=20= virtuality of a function, and it's very much tied to the access level o= f a=20 function. That design decision is _not_ going to be changed, so the vir= tuality=20 of a function is defined by its access level.
 Plus, this isn't a real issue, because the final keyword exists.

It's a _huge_ issue, because it means that nearly every single private=20= function in a class in D will need to be marked with final in order for= it to=20 be non-virtual and inlinable. The default becomes inefficient, and all = just for=20 NVI.
 At the end, for performance concerns, what we want is that the compil=

 or the linker were able to finalize methods that have no override, no=

 some dirty trick that break larger more important conception principl=

Because of how the compilation model works, that's impossible. No class= can=20 know all of its derived classes. Not only are classes compiled complete= ly=20 independently of their derived classes, but derived classes could be li= nked in=20 dynamically at runtime, and those classes could have been written long = after=20 the base class was written and compiled. So, unless the programmer expl= icitly=20 marks a function as final, there's no way that the compiler can know th= at that=20 function won't be overridden.
 Now, it may be different with interfaces. TDPL specifically talks a=


 using private for NVI with _interfaces_, not classes. Doing that so=


 thing with interfaces requires special treatment already, and it do=


 affect efficiency like making private always virtual would, so that=


 should be okay. In the general case though, it's just far better to=


 protected to do NVI with classes and let private be non-virtual and=


 therefore efficient by default rather than inefficient by default.
=20
 - Jonathan M Davis

It is not that simple. First, it introduce an inconsistency between interfaces and classes for no real reasons. The only difference betwe=

 classes and interfaces should be that interface cannot have member da=

 and you can inherit from multiple interfaces. Interface have been
 created to solve the problems that exists for multiple inheritance, a=

 that is enough to solve that problem. Everything else is, again, lack=

 separation of concerns.

private is _not_ going to become virtual in classes. I believe that Wal= ter has=20 stated that it will remain non-virtual (though I'd have to dig through = the=20 newsgroup for the exact quote). The only question is how to handle it i= n=20 interfaces, and as far as I know, the intention is to make interfaces f= ollow=20 TDPL with regards to private.
 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.

Why would there be a compilation error? The base class cannot call the=20= interface's private function, so any reference to that function would b= e the=20 base class' function. - Jonathan M Davis
Apr 04 2012
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 4 April 2012 at 07:05:59 UTC, deadalnix wrote:
 It should be provided as lib by std.traits . And no need for 
 language changes.

It wouldn't account for the case I actually want: checking for export.
Apr 04 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, April 04, 2012 15:49:59 deadalnix wrote:
 Le 04/04/2012 10:41, Jonathan M Davis a écrit :
 Plus, this isn't a real issue, because the final keyword exists.

It's a _huge_ issue, because it means that nearly every single private function in a class in D will need to be marked with final in order for it to be non-virtual and inlinable. The default becomes inefficient, and all just for NVI.

OK, let me restate this. This isn't a language design issue, this is an implementation issue. Java and C# finalize function for ages. See below.
 At the end, for performance concerns, what we want is that the compiler
 or the linker were able to finalize methods that have no override, not
 some dirty trick that break larger more important conception principles.

Because of how the compilation model works, that's impossible. No class can know all of its derived classes. Not only are classes compiled completely independently of their derived classes, but derived classes could be linked in dynamically at runtime, and those classes could have been written long after the base class was written and compiled. So, unless the programmer explicitly marks a function as final, there's no way that the compiler can know that that function won't be overridden.

Unless something is marked as export, no code can be linked to it at runtime. So the linker could finalize any function that isn't export - a vast majority. It doesn't, I admit. But this is definitively a implementation issue, and fixing it by language design decision is a mistake.

In Posix-land, _everything_ is exported. AFAK, this whole idea of marking functions as exported or not is a Windows-only thing. And honestly, it's one of my biggest complaints about programming in Windows. It's seriously annoying to have to deal with constantly exporting stuff on Windows. On Linux, it _just works_. IMHO, shared libraries is one area where Windows seriously dropped the ball in comparison to *nix. In my experience, there are _far_ fewer issues with them on Linux than Windows with its export and its nonsense of having to link against a specific version of a static library to be able to link against a dynamic one. Regardless, export is a Windows-only thing and is completely unusable for any kind of optimizations in the general case.
 It is not that simple. First, it introduce an inconsistency between
 interfaces and classes for no real reasons. The only difference between
 classes and interfaces should be that interface cannot have member data,
 and you can inherit from multiple interfaces. Interface have been
 created to solve the problems that exists for multiple inheritance, and
 that is enough to solve that problem. Everything else is, again, lack of
 separation of concerns.

private is _not_ going to become virtual in classes. I believe that Walter has stated that it will remain non-virtual (though I'd have to dig through the newsgroup for the exact quote). The only question is how to handle it in interfaces, and as far as I know, the intention is to make interfaces follow TDPL with regards to private.

I know, but Andrei seems to disagree. I also have to dig, to find it, but I guess we can trust each other on that.

Actually, I believe that Andrei agreed with Walter in the same thread that Walter agreed not to make private virtual, but I'd have to go dig it up, and unfortunately, I always have a horrible time tracking down posts. I think that it was within the last few months though.
 If I restate the question, what is the benefit of having classes and
 interfaces behaving differently on this point ? TDPL make it pretty
 clear how it should be handled in interfaces. Classes should work the
 same way. Fixing an implementation issue isn't a valid reason.

Allowing private in interfaces is already getting weird, because of what interfaces are for and how they work, and doing so is already arguably special casing things. Certainly, the reasons for having private in interfaces is _completely_ different from having it in classes. private in interfaces is _only_ intended for NVI, whereas NVI is _not_ the reason for having private in classes, and in fact is fairly rare thing to do with private. I don't think that it's _possible_ to overload private in any language other than C++, and I believe that a large number of C++ programmers are completely unaware that it's possible in C++. Making private in classes work the same way as it does in interfaces would be detrimental to classes just for an idiom which many programmers _never_ use (though it _is_ a useful one) - and one which works just fine with protected. I don't believe that it's currently the plan, but I don't see any problem with just making it so that protected is used in interfaces for NVI instead of private. As I already pointed out, in C++, the derived classes can already call their versions of the NVI functions, since they're private to _them_, not the base class, even if they can't call the base class implementation (if there even is one), but interfaces don't _have_ a base class implementation, so there's nothing to protect from being called anyway. So, using protected wouldn't really change anything, and it would be completely consistent with how protected is used normally.
 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.

Why would there be a compilation error? The base class cannot call the interface's private function, so any reference to that function would be the base class' function.

You have a class C and an interface I, both define the private method foo. In MyClass : C, I, if I define foo, I get an error because foo isn't virtual in C, and if I don't, I don't implement I.

All it has to do is ignore C's private function in MyClass, because it's non- virtual and _can't_ be overloaded. - Jonathan M Davis
Apr 04 2012
prev sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Mon, 02 Apr 2012 19:10:01 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Monday, April 02, 2012 13:26:05 Dmitry Olshansky wrote:
 It's all nice and well, but I believe part of the reason of say private
 protection is that user is never ever able to see(!) it. Thus it user
 can't depend on private members being there, which is a good thing.
 If I read it right, the technique you present allows user code to depend
 on private functions being there.
 I argue that we shouldn't even provide a _possibility_ for external
 stuff to depend on private members.
 Same argument in limited scope goes for protected.

As it stands, private has _no_ effect on symbol visibility. All it affects is symbol accessibility. For instance, if you create a private alias in a module, it affects every module that imports your module, or if you create a function which causes an overload conflict, it creates an overload conflict regardless of whether it's private or not. C++ is the same way. Access modifiers are just that, _access_ modifieres. They _only_ affect accessibility, not visibility.

After all you can have access to a value of a protected type. In this case changing the visibility could lead to accidental mismatches. This behavior is taken from C++ and it makes sense. Due to the .h/.cpp separation you'll also have most of the implementation symbols invisible. That's where the C++ model doesn't really map to D if 'import everything' remains the default.
Apr 04 2012