www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Export and Protected Scoping in Dynamic Libraries

reply "Adam Wilson" <flyboynw gmail.com> writes:
Hello Everyone,

I want to start this conversation by pointing out that I come from a  
C/C++/C# background and my ideas and frustrations in this post will be  
colored by that history.

When I first approached D, the idea of an 'export' confused me. I've since  
figured out, at least in libraries, that in C# terms D's 'export' means  
'public'. However, this raise a problem, specifically, how do I export a  
protected member from a dynamic library?

Here is a table of D's scoping system mapped to C#'s scoping system as I  
understand it:

D		     C#	
export		public
public		internal
protected     internal protected
private	      private

I believe this illustrates the problem clearly. There is no protected  
scope that is available to the outside world. The ONLY way to export a  
function in D is to mark is as an export. I have tested this against 2.056  
and without the export identifier, the function never makes it into the  
.lib file. But Protected functions are clearly intended to be used outside  
of the library that they are built in. Currently, D says no; but without  
protected functions, inheritance becomes very troublesome. Any library of  
decent size is going to want to allow it's consumers to inherit from it at  
some point. For example, just yesterday I wrote four functions that  
overrode protected functions in .NET's WPF library. Without access to  
these protected functions I would have been simply unable to solve the  
problem. And my boss would have been VERY unhappy about that.

I think the way export should work is more like an attribute or note to  
the compiler that says "Hey Mr. Compiler, here is this public function,  
compile it. And while you're at it, I set this little flag over here that  
says you should export it as well." It would look something like 'export  
public' or 'export protected'.

So the new D->C# scope mapping table might look like this:

D		     		 C#	
export public		  public
public				 internal
export protected	protected
protected     		 internal protected
private	      		 private

This would completely solve the issue of protected scoping in dynamic  
libraries. As a note, 'export private' should be a compiler error.

I know there is a lot of resistance to making design changes to D2 at this  
point, and rightly so. The biggest issue I see here is that it breaks with  
TDPL. However, this is a major issue and one that will block many people  
 from picking up D for serious work. Support for inheritance from dynamic  
libraries is a MUST HAVE for dynamic libraries to be truly useful. As for  
maintaining backwards compatibility, a function with just 'export' could  
be easily mapped to 'export public' as that is it's current behavior.  
Also, please note that I am not married to that syntax. But a solution is  
required and I think that this one represents a good compromise. It  
doesn't introduce any new keywords, fixes a major design issue, and  
increases expressiveness and readability.

Any thoughts, objections, rants, raves? Walter, Andrei, thoughts?

-- 
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Dec 14 2011
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 14/12/2011 20:41, Adam Wilson a écrit :
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a
 C/C++/C# background and my ideas and frustrations in this post will be
 colored by that history.

 When I first approached D, the idea of an 'export' confused me. I've
 since figured out, at least in libraries, that in C# terms D's 'export'
 means 'public'. However, this raise a problem, specifically, how do I
 export a protected member from a dynamic library?

 Here is a table of D's scoping system mapped to C#'s scoping system as I
 understand it:

 D C#
 export public
 public internal
 protected internal protected
 private private

 I believe this illustrates the problem clearly. There is no protected
 scope that is available to the outside world. The ONLY way to export a
 function in D is to mark is as an export. I have tested this against
 2.056 and without the export identifier, the function never makes it
 into the .lib file. But Protected functions are clearly intended to be
 used outside of the library that they are built in. Currently, D says
 no; but without protected functions, inheritance becomes very
 troublesome. Any library of decent size is going to want to allow it's
 consumers to inherit from it at some point. For example, just yesterday
 I wrote four functions that overrode protected functions in .NET's WPF
 library. Without access to these protected functions I would have been
 simply unable to solve the problem. And my boss would have been VERY
 unhappy about that.

 I think the way export should work is more like an attribute or note to
 the compiler that says "Hey Mr. Compiler, here is this public function,
 compile it. And while you're at it, I set this little flag over here
 that says you should export it as well." It would look something like
 'export public' or 'export protected'.

 So the new D->C# scope mapping table might look like this:

 D C#
 export public public
 public internal
 export protected protected
 protected internal protected
 private private

 This would completely solve the issue of protected scoping in dynamic
 libraries. As a note, 'export private' should be a compiler error.

 I know there is a lot of resistance to making design changes to D2 at
 this point, and rightly so. The biggest issue I see here is that it
 breaks with TDPL. However, this is a major issue and one that will block
 many people from picking up D for serious work. Support for inheritance
 from dynamic libraries is a MUST HAVE for dynamic libraries to be truly
 useful. As for maintaining backwards compatibility, a function with just
 'export' could be easily mapped to 'export public' as that is it's
 current behavior. Also, please note that I am not married to that
 syntax. But a solution is required and I think that this one represents
 a good compromise. It doesn't introduce any new keywords, fixes a major
 design issue, and increases expressiveness and readability.

 Any thoughts, objections, rants, raves? Walter, Andrei, thoughts?

The same goes for private virtual methods actually. They cannot be called, but they can be reimplemented in subclasses. If you put that detail asside, this is a neat idea. I'd love to see that in D.
Dec 15 2011
parent deadalnix <deadalnix gmail.com> writes:
Le 15/12/2011 09:52, Jonathan M Davis a écrit :
 On Thursday, December 15, 2011 09:32:37 deadalnix wrote:
 The same goes for private virtual methods actually. They cannot be
 called, but they can be reimplemented in subclasses.

 If you put that detail asside, this is a neat idea. I'd love to see that
 in D.

http://d.puremagic.com/issues/show_bug.cgi?id=4542

This is not the same problem, but close. I think both are important to solve. I do agree with your comments on this bugs.
Dec 15 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, December 15, 2011 09:32:37 deadalnix wrote:
 The same goes for private virtual methods actually. They cannot be
 called, but they can be reimplemented in subclasses.
 
 If you put that detail asside, this is a neat idea. I'd love to see that
 in D.

http://d.puremagic.com/issues/show_bug.cgi?id=4542
Dec 15 2011
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a C/C++/C#
 background and my ideas and frustrations in this post will be colored by that
 history.

 When I first approached D, the idea of an 'export' confused me. I've since
 figured out, at least in libraries, that in C# terms D's 'export' means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.
 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?
Dec 16 2011
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-12-16 10:34, Walter Bright wrote:
 On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a
 C/C++/C#
 background and my ideas and frustrations in this post will be colored
 by that
 history.

 When I first approached D, the idea of an 'export' confused me. I've
 since
 figured out, at least in libraries, that in C# terms D's 'export' means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.
 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?

What I think he wants is a member that is overrideable in a subclass outside of the DLL but otherwise only reachable within the module its defined in. // foo.dll/foo.d class Foo { export protected void foo () {} } // foo.dll/bar.d auto foo = new Foo; foo.foo; // error can't access protected member // bar.dll/bar.d class Bar : Foo { export protected override void foo () {} // it's fine to override the method since it's protected, it's reachable in this dll since it's exported } -- /Jacob Carlborg
Dec 16 2011
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/16/2011 9:01 PM, Adam Wilson wrote:
 On Fri, 16 Dec 2011 01:34:32 -0800, Walter Bright <newshound2 digitalmars.com>
 wrote:

 On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a C/C++/C#
 background and my ideas and frustrations in this post will be colored by that
 history.

 When I first approached D, the idea of an 'export' confused me. I've since
 figured out, at least in libraries, that in C# terms D's 'export' means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.

Huh, I didn't know that. It makes sense though, I recall using __declspec(dllexport) in MSVC and hating the unwieldy syntax. Does it perform a similar function on other OS'es?

No, as such magic isn't necessary. Any public symbol will be automatically accessible from a shared library.
 C#'s public means "this function can be used anyone, including other libraries
 or executables", which according to the D docs, is what D's export means. C#
 also has 'protected' and 'internal protected'. 'protected' is available to any
 subclass, even those outside of the library, and 'internal protected' is only
 available to subclasses within the same library.

 It's actually quite a useful to have that distinction. For example, in WPF you
 end up subclassing the system classes (which are in separate DLL's) repeatedly
 and it is an encouraged pattern to extend existing functionality with your own.
 The reason I went with the syntax I did is that it doesn't add any new keywords
 and is, I think, even more clear about the programmers intentionality than C#'s
 scoping model. If 'export' really is just the equivalent of
 __declspec(dllexport) then it makes even more sense as an attribute and not a
 scope in it's own right.

 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?

What I am after is a member that can be overridden in a subclass outside of the DLL but is otherwise only reachable within the module it's defined in. Mr. Carlborg's code example is spot on.

It isn't necessary to export a protected symbol in order to override it.
Dec 17 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/17/2011 11:22 AM, Adam Wilson wrote:
 On Sat, 17 Dec 2011 01:33:33 -0800, Walter Bright <newshound2 digitalmars.com>
 wrote:

 On 12/16/2011 9:01 PM, Adam Wilson wrote:
 On Fri, 16 Dec 2011 01:34:32 -0800, Walter Bright <newshound2 digitalmars.com>
 wrote:

 On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a C/C++/C#
 background and my ideas and frustrations in this post will be colored by that
 history.

 When I first approached D, the idea of an 'export' confused me. I've since
 figured out, at least in libraries, that in C# terms D's 'export' means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.

Huh, I didn't know that. It makes sense though, I recall using __declspec(dllexport) in MSVC and hating the unwieldy syntax. Does it perform a similar function on other OS'es?

No, as such magic isn't necessary. Any public symbol will be automatically accessible from a shared library.

So my understanding then is that, for dynamic libs on OS'es like Linux/OSX, DI files also need to include members marked public?

The member function will be called through the vtbl[]. But for the compiler to know that member even exists, and where it is in the vtbl[], it needs to be in the .di file.
 And how does the compiler
 treat the export keyword on systems that don't need it? I am assuming they are
 treated as publics?

Yes.
 C#'s public means "this function can be used anyone, including other libraries
 or executables", which according to the D docs, is what D's export means. C#
 also has 'protected' and 'internal protected'. 'protected' is available to any
 subclass, even those outside of the library, and 'internal protected' is only
 available to subclasses within the same library.

 It's actually quite a useful to have that distinction. For example, in WPF you
 end up subclassing the system classes (which are in separate DLL's) repeatedly
 and it is an encouraged pattern to extend existing functionality with your own.
 The reason I went with the syntax I did is that it doesn't add any new keywords
 and is, I think, even more clear about the programmers intentionality than C#'s
 scoping model. If 'export' really is just the equivalent of
 __declspec(dllexport) then it makes even more sense as an attribute and not a
 scope in it's own right.

 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?

What I am after is a member that can be overridden in a subclass outside of the DLL but is otherwise only reachable within the module it's defined in. Mr. Carlborg's code example is spot on.

It isn't necessary to export a protected symbol in order to override it.

I see, that's a neat little trick. +1 for D!

C++ works the same way.
 So for DI file generation I should
 just include the protected members in the file and let the compiler sort it
out?

Yes.
 I know DLL's are relatively new to D so not much documentation exists about how
 they work in D. I think this would be something good to document, especially
the
 special behaviors for Windows. I have to admit that, coming from a C# and C++
 background, I've been a little confused by how the scope system works in
 relation to dynamic libraries and have been shooting the dark trying to figure
 it out. Thanks for the clarifications! Hopefully this lesson will become part
of
 the D lore surrounding dynamic libs. :-)

DLL's have been supported in D forever, but hardly anyone uses them, and sometimes they get bit rotted. I strongly recommend Jeffrey Richter's book "Advanced Windows" for a low level and lucid explanation of how DLLs work.
Dec 17 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2011 4:26 AM, Andrej Mitrovic wrote:
 I'd hardly call it "support" when you can't even implicitly link to a
 D DLL that has certain phobos imports
 http://dl.dropbox.com/u/9218759/dll_bug.zip .

Bug reports need to be in bugzilla, not dropbox. Please add to bugzilla.
Dec 18 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2011 10:46 AM, Andrej Mitrovic wrote:
 On 12/18/11, Walter Bright<newshound2 digitalmars.com>  wrote:
 On 12/18/2011 4:26 AM, Andrej Mitrovic wrote:
 I'd hardly call it "support" when you can't even implicitly link to a
 D DLL that has certain phobos imports
 http://dl.dropbox.com/u/9218759/dll_bug.zip .

Bug reports need to be in bugzilla, not dropbox. Please add to bugzilla.

This was filed in May (you can ignore the first few comments, but the files in attachment are all you need to know) http://d.puremagic.com/issues/show_bug.cgi?id=6019

Thank you.
 I also made a topic but nobody replied back then:
 http://www.digitalmars.com/d/archives/digitalmars/D/What_to_do_about_implicit_linking_with_D_DLLs_that_import_Phobos_136659.html

There are several people going through the bugzilla issues, hopefully we can get to yours soon.
Dec 19 2011
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 17.12.2011 20:50, Walter Bright wrote:
 On 12/17/2011 11:22 AM, Adam Wilson wrote:
 On Sat, 17 Dec 2011 01:33:33 -0800, Walter Bright
 <newshound2 digitalmars.com>
 wrote:
 It isn't necessary to export a protected symbol in order to override it.

I see, that's a neat little trick. +1 for D!

C++ works the same way.

You still need the protected symbol if you want to call the base class implementation through "super", or if you do not override the protected function in the derived class, so the base class member function needs to be placed into the vtbl of the derived class.
 I know DLL's are relatively new to D so not much documentation exists
 about how
 they work in D. I think this would be something good to document,
 especially the
 special behaviors for Windows. I have to admit that, coming from a C#
 and C++
 background, I've been a little confused by how the scope system works in
 relation to dynamic libraries and have been shooting the dark trying
 to figure
 it out. Thanks for the clarifications! Hopefully this lesson will
 become part of
 the D lore surrounding dynamic libs. :-)

DLL's have been supported in D forever, but hardly anyone uses them, and sometimes they get bit rotted. I strongly recommend Jeffrey Richter's book "Advanced Windows" for a low level and lucid explanation of how DLLs work.

sharing D objects across DLLs (as you will obviously need when deriving classes in another DLL) is not well supported, a stab at sharing phobos as a test bed can be found here: http://d.puremagic.com/issues/show_bug.cgi?id=4071, but it is probably dated.
Dec 19 2011
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/19/2011 4:08 AM, Rainer Schuetze wrote:
 On 17.12.2011 20:50, Walter Bright wrote:
 On 12/17/2011 11:22 AM, Adam Wilson wrote:
 On Sat, 17 Dec 2011 01:33:33 -0800, Walter Bright
 <newshound2 digitalmars.com>
 wrote:
 It isn't necessary to export a protected symbol in order to override it.

I see, that's a neat little trick. +1 for D!

C++ works the same way.

You still need the protected symbol if you want to call the base class implementation through "super", or if you do not override the protected function in the derived class, so the base class member function needs to be placed into the vtbl of the derived class.

That's right. I recommend in those cases to make such members "export" rather than "protected".
 I know DLL's are relatively new to D so not much documentation exists
 about how
 they work in D. I think this would be something good to document,
 especially the
 special behaviors for Windows. I have to admit that, coming from a C#
 and C++
 background, I've been a little confused by how the scope system works in
 relation to dynamic libraries and have been shooting the dark trying
 to figure
 it out. Thanks for the clarifications! Hopefully this lesson will
 become part of
 the D lore surrounding dynamic libs. :-)

DLL's have been supported in D forever, but hardly anyone uses them, and sometimes they get bit rotted. I strongly recommend Jeffrey Richter's book "Advanced Windows" for a low level and lucid explanation of how DLLs work.

sharing D objects across DLLs (as you will obviously need when deriving classes in another DLL) is not well supported, a stab at sharing phobos as a test bed can be found here: http://d.puremagic.com/issues/show_bug.cgi?id=4071, but it is probably dated.

Dec 19 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/11 1:22 PM, Adam Wilson wrote:
 I know DLL's are relatively new to D so not much documentation exists
 about how they work in D. I think this would be something good to
 document, especially the special behaviors for Windows. I have to admit
 that, coming from a C# and C++ background, I've been a little confused
 by how the scope system works in relation to dynamic libraries and have
 been shooting the dark trying to figure it out. Thanks for the
 clarifications! Hopefully this lesson will become part of the D lore
 surrounding dynamic libs. :-)

The best thing you'd do is to write an article - an RFC - about it, which we publish on the website. While in draft mode, the article documents what needs be done, and after the implementation of shared libs is done (and there's a lot of momentum behind it), the article will become a great description of it. Andrei
Dec 17 2011
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/17/11 5:14 PM, Adam Wilson wrote:
 This is a fantastic idea Andrei, I'll see what I can come up with. It
 shouldn't be that hard, mostly i'll just have document the process, talk
 about the limitations of dynamic libraries, and make sure that the
 scoping issues are clearly defined and explained.

Great. Please format your RFC drafts as pull requests for d-p-l.org (use any existing page as a template, e.g. https://github.com/D-Programming-Language/d-programming-language.org/ lob/master/abi.dd), and I'll upload them as we make progress. Andrei
Dec 17 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Fri, 16 Dec 2011 01:34:32 -0800, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a  
 C/C++/C#
 background and my ideas and frustrations in this post will be colored  
 by that
 history.

 When I first approached D, the idea of an 'export' confused me. I've  
 since
 figured out, at least in libraries, that in C# terms D's 'export' means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.

Huh, I didn't know that. It makes sense though, I recall using __declspec(dllexport) in MSVC and hating the unwieldy syntax. Does it perform a similar function on other OS'es? C#'s public means "this function can be used anyone, including other libraries or executables", which according to the D docs, is what D's export means. C# also has 'protected' and 'internal protected'. 'protected' is available to any subclass, even those outside of the library, and 'internal protected' is only available to subclasses within the same library. It's actually quite a useful to have that distinction. For example, in WPF you end up subclassing the system classes (which are in separate DLL's) repeatedly and it is an encouraged pattern to extend existing functionality with your own. The reason I went with the syntax I did is that it doesn't add any new keywords and is, I think, even more clear about the programmers intentionality than C#'s scoping model. If 'export' really is just the equivalent of __declspec(dllexport) then it makes even more sense as an attribute and not a scope in it's own right.
 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?

What I am after is a member that can be overridden in a subclass outside of the DLL but is otherwise only reachable within the module it's defined in. Mr. Carlborg's code example is spot on. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 16 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 17 Dec 2011 01:33:33 -0800, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 12/16/2011 9:01 PM, Adam Wilson wrote:
 On Fri, 16 Dec 2011 01:34:32 -0800, Walter Bright  
 <newshound2 digitalmars.com>
 wrote:

 On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from a  
 C/C++/C#
 background and my ideas and frustrations in this post will be colored  
 by that
 history.

 When I first approached D, the idea of an 'export' confused me. I've  
 since
 figured out, at least in libraries, that in C# terms D's 'export'  
 means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.

Huh, I didn't know that. It makes sense though, I recall using __declspec(dllexport) in MSVC and hating the unwieldy syntax. Does it perform a similar function on other OS'es?

No, as such magic isn't necessary. Any public symbol will be automatically accessible from a shared library.

So my understanding then is that, for dynamic libs on OS'es like Linux/OSX, DI files also need to include members marked public? And how does the compiler treat the export keyword on systems that don't need it? I am assuming they are treated as publics?
 C#'s public means "this function can be used anyone, including other  
 libraries
 or executables", which according to the D docs, is what D's export  
 means. C#
 also has 'protected' and 'internal protected'. 'protected' is available  
 to any
 subclass, even those outside of the library, and 'internal protected'  
 is only
 available to subclasses within the same library.

 It's actually quite a useful to have that distinction. For example, in  
 WPF you
 end up subclassing the system classes (which are in separate DLL's)  
 repeatedly
 and it is an encouraged pattern to extend existing functionality with  
 your own.
 The reason I went with the syntax I did is that it doesn't add any new  
 keywords
 and is, I think, even more clear about the programmers intentionality  
 than C#'s
 scoping model. If 'export' really is just the equivalent of
 __declspec(dllexport) then it makes even more sense as an attribute and  
 not a
 scope in it's own right.

 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?

What I am after is a member that can be overridden in a subclass outside of the DLL but is otherwise only reachable within the module it's defined in. Mr. Carlborg's code example is spot on.

It isn't necessary to export a protected symbol in order to override it.

I see, that's a neat little trick. +1 for D! So for DI file generation I should just include the protected members in the file and let the compiler sort it out? I know DLL's are relatively new to D so not much documentation exists about how they work in D. I think this would be something good to document, especially the special behaviors for Windows. I have to admit that, coming from a C# and C++ background, I've been a little confused by how the scope system works in relation to dynamic libraries and have been shooting the dark trying to figure it out. Thanks for the clarifications! Hopefully this lesson will become part of the D lore surrounding dynamic libs. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 17 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 17 Dec 2011 14:27:12 -0800, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 On 12/17/11 1:22 PM, Adam Wilson wrote:
 I know DLL's are relatively new to D so not much documentation exists
 about how they work in D. I think this would be something good to
 document, especially the special behaviors for Windows. I have to admit
 that, coming from a C# and C++ background, I've been a little confused
 by how the scope system works in relation to dynamic libraries and have
 been shooting the dark trying to figure it out. Thanks for the
 clarifications! Hopefully this lesson will become part of the D lore
 surrounding dynamic libs. :-)

The best thing you'd do is to write an article - an RFC - about it, which we publish on the website. While in draft mode, the article documents what needs be done, and after the implementation of shared libs is done (and there's a lot of momentum behind it), the article will become a great description of it. Andrei

This is a fantastic idea Andrei, I'll see what I can come up with. It shouldn't be that hard, mostly i'll just have document the process, talk about the limitations of dynamic libraries, and make sure that the scoping issues are clearly defined and explained. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 17 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 17 Dec 2011 11:50:39 -0800, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 12/17/2011 11:22 AM, Adam Wilson wrote:
 On Sat, 17 Dec 2011 01:33:33 -0800, Walter Bright  
 <newshound2 digitalmars.com>
 wrote:

 On 12/16/2011 9:01 PM, Adam Wilson wrote:
 On Fri, 16 Dec 2011 01:34:32 -0800, Walter Bright  
 <newshound2 digitalmars.com>
 wrote:

 On 12/14/2011 11:41 AM, Adam Wilson wrote:
 Hello Everyone,

 I want to start this conversation by pointing out that I come from  
 a C/C++/C#
 background and my ideas and frustrations in this post will be  
 colored by that
 history.

 When I first approached D, the idea of an 'export' confused me.  
 I've since
 figured out, at least in libraries, that in C# terms D's 'export'  
 means
 'public'.

I'm not too familiar with C#'s public, but what D 'export' means is a function is an entry point for a DLL. In the Windows world, that means it gets an extra level of indirection when calling it, and it corresponds to: __declspec(export) in Windows compilers.

Huh, I didn't know that. It makes sense though, I recall using __declspec(dllexport) in MSVC and hating the unwieldy syntax. Does it perform a similar function on other OS'es?

No, as such magic isn't necessary. Any public symbol will be automatically accessible from a shared library.

So my understanding then is that, for dynamic libs on OS'es like Linux/OSX, DI files also need to include members marked public?

The member function will be called through the vtbl[]. But for the compiler to know that member even exists, and where it is in the vtbl[], it needs to be in the .di file.
 And how does the compiler
 treat the export keyword on systems that don't need it? I am assuming  
 they are
 treated as publics?

Yes.

Understood. I'll update my DI gen changes to bring those in.
 C#'s public means "this function can be used anyone, including other  
 libraries
 or executables", which according to the D docs, is what D's export  
 means. C#
 also has 'protected' and 'internal protected'. 'protected' is  
 available to any
 subclass, even those outside of the library, and 'internal protected'  
 is only
 available to subclasses within the same library.

 It's actually quite a useful to have that distinction. For example,  
 in WPF you
 end up subclassing the system classes (which are in separate DLL's)  
 repeatedly
 and it is an encouraged pattern to extend existing functionality with  
 your own.
 The reason I went with the syntax I did is that it doesn't add any  
 new keywords
 and is, I think, even more clear about the programmers intentionality  
 than C#'s
 scoping model. If 'export' really is just the equivalent of
 __declspec(dllexport) then it makes even more sense as an attribute  
 and not a
 scope in it's own right.

 However, this raise a problem, specifically, how do I export a
 protected member from a dynamic library?

Do you mean how does one directly call a protected function inside a DLL from outside that DLL, without going through the virtual table?

What I am after is a member that can be overridden in a subclass outside of the DLL but is otherwise only reachable within the module it's defined in. Mr. Carlborg's code example is spot on.

It isn't necessary to export a protected symbol in order to override it.

I see, that's a neat little trick. +1 for D!

C++ works the same way.

I may have know that at some point, but for the most part in the classes I took, they said "it just works" and left it at that without any further explanation.
 So for DI file generation I should
 just include the protected members in the file and let the compiler  
 sort it out?

Yes.
 I know DLL's are relatively new to D so not much documentation exists  
 about how
 they work in D. I think this would be something good to document,  
 especially the
 special behaviors for Windows. I have to admit that, coming from a C#  
 and C++
 background, I've been a little confused by how the scope system works in
 relation to dynamic libraries and have been shooting the dark trying to  
 figure
 it out. Thanks for the clarifications! Hopefully this lesson will  
 become part of
 the D lore surrounding dynamic libs. :-)

DLL's have been supported in D forever, but hardly anyone uses them, and sometimes they get bit rotted.

Well, as long as I'm around I'll be sure to pester someone when it starts to get stale. Dynamic libraries are fundamental to my projects.
 I strongly recommend Jeffrey Richter's book "Advanced Windows" for a low  
 level and lucid explanation of how DLLs work.

I'll look into it ... my bookshelf is already creaking under the strain of all the programming books I should have. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 17 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/17/11, Walter Bright <newshound2 digitalmars.com> wrote:
 DLL's have been supported in D forever.

I'd hardly call it "support" when you can't even implicitly link to a D DLL that has certain phobos imports http://dl.dropbox.com/u/9218759/dll_bug.zip .
Dec 18 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/18/11, Walter Bright <newshound2 digitalmars.com> wrote:
 On 12/18/2011 4:26 AM, Andrej Mitrovic wrote:
 I'd hardly call it "support" when you can't even implicitly link to a
 D DLL that has certain phobos imports
 http://dl.dropbox.com/u/9218759/dll_bug.zip .

Bug reports need to be in bugzilla, not dropbox. Please add to bugzilla.

This was filed in May (you can ignore the first few comments, but the files in attachment are all you need to know) http://d.puremagic.com/issues/show_bug.cgi?id=6019 I also made a topic but nobody replied back then: http://www.digitalmars.com/d/archives/digitalmars/D/What_to_do_about_implicit_linking_with_D_DLLs_that_import_Phobos_136659.html
Dec 18 2011
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 12/19/11, Walter Bright <newshound2 digitalmars.com> wrote:
 There are several people going through the bugzilla issues, hopefully we can
 get
 to yours soon.

Ok, but note that it's not a high-priority bug for me. There's also a workaround, ModuleInfo can be faked by adding a stub global extern(C) variable inside the main file, e.g.: extern(C) int D6EdrLib12__ModuleInfoZ;
Dec 19 2011