www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DMD 0.142 release

reply "Walter Bright" <newshound digitalmars.com> writes:
This incorporates a new 'header' generator capability, written by Dave
Fladebo, now working!

http://www.digitalmars.com/d/changelog.html
Dec 29 2005
next sibling parent reply Deewiant <deewiant.doesnotlike.spam gmail.com> writes:
Walter Bright wrote:

 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!
 


Thanks for the release, but "partly" is missing from before "working" (as usual
when it comes to programming <g>). I presume bugs related to the header
generator still belong in digitalmars.D.bugs?

I'll post my report here anyway since it's so short. The following crashes DMD
completely when compiled with -H, on Windows (XP) at least:

this() {}

In or outside a class, doesn't matter, it still doesn't work.
Dec 29 2005
parent reply Deewiant <deewiant.doesnotlike.spam gmail.com> writes:
Deewiant wrote:

 this() {}
 
 In or outside a class, doesn't matter, it still doesn't work.


I should probably correct myself somewhat: if it's inside a class the header
file is correctly generated, but DMD still crashes.
Dec 29 2005
parent "Walter Bright" <newshound digitalmars.com> writes:
"Deewiant" <deewiant.doesnotlike.spam gmail.com> wrote in message 
news:dp1epm$nta$1 digitaldaemon.com...

 Deewiant wrote:

 this() {}

 In or outside a class, doesn't matter, it still doesn't work.


 I should probably correct myself somewhat: if it's inside a class the 
 header
 file is correctly generated, but DMD still crashes.


Thanks, I'll take care of it.
Dec 29 2005
prev sibling next sibling parent reply "Derek Parnell" <derek psych.ward> writes:
On Fri, 30 Dec 2005 06:11:54 +1100, Walter Bright  
<newshound digitalmars.com> wrote:


 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!


Well it might be working, but it doesn't do what I was expecting.

This source file ...

//--------------------------------
import std.stdio;

private int qwerty;
class A
{
     private int x;
     void ths()
     {
       x = 1;
       }
}

void xpub()
{
     A a = new A;
     qwerty = 1;
}

private void xpriv()
{
     A b = new A;
     qwerty = 2;
}
//--------------------------------


was turned into this 'header' ...

// D import file generated from 'test.d'
import std.stdio;
private
{
     int qwerty;
}
class A
{
     private
{
     int x;
}
     void ths()
{
x = 1;
}
}
void xpub()
{
A a = new A;
qwerty = 1;
}
private
{
     void xpriv()
{
A b = new A;
qwerty = 2;
}
}
//-------------------------

but I was expecting something more like this ...

// D import file generated by hand
import std.stdio;
class A
{
     void ths(){}
}
void xpub(){}
//-------------------------


In other words, why does the generated header show implementation code and  
why does it show private members? Currently, all it seems to do is  
reformat the original source by placing braces around private sections.
-- 
Derek Parnell
Melbourne, Australia
Dec 29 2005
next sibling parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Derek Parnell" <derek psych.ward> wrote in message 
news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 In other words, why does the generated header show implementation code and 
 why does it show private members?


It shows implementation code for functions that are candidates for inlining. 
It shows private members because they might be used in const initializers 
and in inline functions.

If a function cannot be inlined, its implementation will not be included.
Dec 29 2005
next sibling parent reply John Reimer <terminal.node gmail.com> writes:
Walter Bright wrote:

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 In other words, why does the generated header show implementation code and 
 why does it show private members?


 
 It shows implementation code for functions that are candidates for inlining. 
 It shows private members because they might be used in const initializers 
 and in inline functions.
 
 If a function cannot be inlined, its implementation will not be included. 
 
 


This really doesn't look right.  I realize the need to make private data 
visible for inlining purposes, but this appears to defeat the main 
purpose of the interface file -- hiding private members and 
implementation details.

The need to accommodate inlining seems wrong too.  Don't you think it's 
better, at this point, to disable inlining for interface files and 
document it well so nobody is surprised by the absence of it.

Maybe, later on, a better solution can be concocted.

-JJR
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"John Reimer" <terminal.node gmail.com> wrote in message 
news:dp1ja1$rh2$1 digitaldaemon.com...

 I realize the need to make private data visible for inlining purposes, but 
 this appears to defeat the main purpose of the interface file -- hiding 
 private members and implementation details.

 The need to accommodate inlining seems wrong too.  Don't you think it's 
 better, at this point, to disable inlining for interface files and 
 document it well so nobody is surprised by the absence of it.


This would be a perfectly valid solution, but Dave and I decided to come at 
it from the direction of having the results be identical whether importing a 
.d or a .di file.

Note that template bodies also wind up being totally reproduced in a .di 
file. Private data members are also needed in order to get the class member 
offsets right. In this way, it's like a C++ header file, or a Java .class 
file.

I could switch it to a binary format, and that would provide some level of 
obfuscation, but that shouldn't be confused with a secure format. Binary 
symbol formats are easilly hacked and reversed. Even if the file format was 
encrypted, the decryption keys would have to be supplied to the compiler, 
and once DVD Jon reverse engineers it, that's the end of any security.

The best way to achieve hiding is to use interface classes, not an interface 
file. The main advantage of a D interface file is compile performance.

(P.S. binary symbol table formats have been compared to object files as far 
as difficulty of reverse engineering. I believe this is a misguided and 
completely false analogy. Reverse engineering an object file is converting 
the proverbial hamburger back into a cow, because most semantic information 
is thrown away in the process of making an object file. Not so with a symbol 
file; all the semantic information *must* be present.

This thinking is the result of my discussions with C++ experts on having 
binary exported template symbol files, which many otherwise well informed 
people mistakenly imagine will make for a securely hidden implementation.
)

(P.P.S. A D interface file can also be compared with a Java .class file, 
which can be imported by Java compilers rather than source code. But many 
tools exist which automatically turn .class files into reasonable Java 
source code. .class files don't really hide information at all.)
Dec 29 2005
parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 29 Dec 2005 13:38:48 -0800, Walter Bright wrote:


 The main advantage of a D interface file is compile performance.


This is the *only* advantage and that's not such a problem anyhow. It seems
then that I won't be needing the -H switch anytime soon.

Excuse me for sounding impertinent and selfish, but are you spending your
time on D wisely? We get something of little value and still not receive
things that are higher on the priority list.

How can I help? I know I can /donate/ library code and documentation, but
the core language is still where the pre-1.0 issues lie. And even when I
have given you library fixes, they are still to appear. I guess I don't
really understand your organization's development processes.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 9:41:09 AM
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Derek Parnell" <derek psych.ward> wrote in message 
news:1uxk73kt2jabq.1w7j11urctxbb.dlg 40tude.net...

 On Thu, 29 Dec 2005 13:38:48 -0800, Walter Bright wrote:

 The main advantage of a D interface file is compile performance.


 This is the *only* advantage


No, it does wind up hiding the implementation details of most functions.


 and that's not such a problem anyhow.


Build speed isn't a problem now, but it will become one once D projects 
exceed a certain size.


 It seems then that I won't be needing the -H switch anytime soon.


It's certainly a forward looking capability.


 Excuse me for sounding impertinent and selfish, but are you spending your
 time on D wisely? We get something of little value and still not receive
 things that are higher on the priority list.


It is regularly asked for. The specific impetus in this case was that Dave 
Fladebo went ahead and implemented it.


 How can I help? I know I can /donate/ library code and documentation, but
 the core language is still where the pre-1.0 issues lie. And even when I
 have given you library fixes, they are still to appear. I guess I don't
 really understand your organization's development processes.


It's mainly driven by 1) what people feel like contributing and 2) things 
that are blocking use of D.
Dec 29 2005
next sibling parent Chris Lajoie <ctlajoie___remove___this___ ___gmail.com> writes:
Walter Bright wrote:

 How can I help? I know I can /donate/ library code and documentation, but
 the core language is still where the pre-1.0 issues lie. And even when I
 have given you library fixes, they are still to appear. I guess I don't
 really understand your organization's development processes.


 
 It's mainly driven by 1) what people feel like contributing and 2) things 
 that are blocking use of D. 


I believe Derek is asking how he can help the development of D. There is 
no indication that many of the things that need to be implemented/fixed 
are even being considered yet. Also, I have seen complaints from a few 
people now that their fixes aren't making it into phobos.

Chris
Dec 30 2005
prev sibling parent reply "Dave" <Dave_member pathlink.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote in message
news:dp1s4k$12ev$2 digitaldaemon.com...

 "Derek Parnell" <derek psych.ward> wrote in message
 news:1uxk73kt2jabq.1w7j11urctxbb.dlg 40tude.net...

 On Thu, 29 Dec 2005 13:38:48 -0800, Walter Bright wrote:

 The main advantage of a D interface file is compile performance.


 This is the *only* advantage


 No, it does wind up hiding the implementation details of most functions.


 and that's not such a problem anyhow.


 Build speed isn't a problem now, but it will become one once D projects
 exceed a certain size.


 It seems then that I won't be needing the -H switch anytime soon.


 It's certainly a forward looking capability.


 Excuse me for sounding impertinent and selfish, but are you spending your
 time on D wisely? We get something of little value and still not receive
 things that are higher on the priority list.




Maybe it isn't a big step, at least at this stage in D's lifetime, but if so 
it may have cost me a bunch of time, not Walter (AFAIK). He integrated (and 
improved) this over the Holiday season and I reckon he deserves a break once 
in a while <g>

It looks like there may be a few issues right now, but once -H is stable, 
build the phobos and DWT .di's and then compare and perhaps you'll see more 
value: 1) lots less code for the compiler to cut through and 2) the inlined 
functions really don't give all that much away, IMO.

During my testing, for phobos the code reduction was ~50% and for DWT ~80%. 
Not to mention that skipping the semantic analysis for large function bodies 
should save a good chunk of compiler time.

Once I saw how DWT was put together, one of the things I thought this would 
be useful for would be things like RAD IDE's. RAD IDE's should be a lot 
easier to build if you can import a couple of 'master headers' that then 
import the rest. If you can reduce the avg. 'header' by 80% for a GUI 
library (like DWT), the edit-compile-debug cycles should be much shorter, 
plus you simplify the IDE with respect to maintaining imports and the like.


 It is regularly asked for. The specific impetus in this case was that Dave
 Fladebo went ahead and implemented it.


The -H stuff was built using dmdfe(*) and then I wrote Walter regarding what 
it did and how it was tested to see if it made sense to include in the
compiler before posting it as a seperate tool. It was intentionally built to 
be "easy" to plug-in to the dmd front end, but I also found out that it
pretty much needs a full-blown compiler to do things right.

The original dmdfe implementation was tested by producing 'headers' for 
phobos, DWT and several other test cases covering all of the examples in the 
documentation.

I experimented with the "implementation hiding" details previously mentioned 
in other posts, but that was abandoned for the reasons also previously
mentioned in other posts (see: 
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).

* dmdfe: http://home.comcast.net/~benhinkle/dmdfe/


 How can I help? I know I can /donate/ library code and documentation, but
 the core language is still where the pre-1.0 issues lie. And even when I
 have given you library fixes, they are still to appear. I guess I don't
 really understand your organization's development processes.


 It's mainly driven by 1) what people feel like contributing and 2) things
 that are blocking use of D.
Dec 30 2005
parent reply John Reimer <terminal.node gmail.com> writes:
 Maybe it isn't a big step, at least at this stage in D's lifetime, but if so 
 it may have cost me a bunch of time, not Walter (AFAIK). He integrated (and 
 improved) this over the Holiday season and I reckon he deserves a break once 
 in a while <g>
 
 It looks like there may be a few issues right now, but once -H is stable, 
 build the phobos and DWT .di's and then compare and perhaps you'll see more 
 value: 1) lots less code for the compiler to cut through and 2) the inlined 
 functions really don't give all that much away, IMO.
 
 During my testing, for phobos the code reduction was ~50% and for DWT ~80%. 
 Not to mention that skipping the semantic analysis for large function bodies 
 should save a good chunk of compiler time.
 
 Once I saw how DWT was put together, one of the things I thought this would 
 be useful for would be things like RAD IDE's. RAD IDE's should be a lot 
 easier to build if you can import a couple of 'master headers' that then 
 import the rest. If you can reduce the avg. 'header' by 80% for a GUI 
 library (like DWT), the edit-compile-debug cycles should be much shorter, 
 plus you simplify the IDE with respect to maintaining imports and the like.
 

 It is regularly asked for. The specific impetus in this case was that Dave
 Fladebo went ahead and implemented it.


 
 The -H stuff was built using dmdfe(*) and then I wrote Walter regarding what 
 it did and how it was tested to see if it made sense to include in the
 compiler before posting it as a seperate tool. It was intentionally built to 
 be "easy" to plug-in to the dmd front end, but I also found out that it
 pretty much needs a full-blown compiler to do things right.
 
 The original dmdfe implementation was tested by producing 'headers' for 
 phobos, DWT and several other test cases covering all of the examples in the 
 documentation.
 
 I experimented with the "implementation hiding" details previously mentioned 
 in other posts, but that was abandoned for the reasons also previously
 mentioned in other posts (see: 
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).
 
 * dmdfe: http://home.comcast.net/~benhinkle/dmdfe/



Walter is not always the best at "selling" these things to us.  You've 
done a lot better job here. :)

Still, I wonder how these headers look for DWT, which I'm pretty sure 
has lots of opportunities for inlining and private data.

-JJR
Dec 30 2005
parent "Dave" <Dave_member pathlink.com> writes:
"John Reimer" <terminal.node gmail.com> wrote in message 
news:dp4lhd$4m1$1 digitaldaemon.com...


 During my testing, for phobos the code reduction was ~50% and for DWT 
 ~80%.





 Still, I wonder how these headers look for DWT, which I'm pretty sure has 
 lots of opportunities for inlining and private data.


I just re-checked my numbers and it looks to be >= 80% ;)

But, it is pretty hard to do really accurately with a quick grep -v | wc 
type of thing because of different whitespace delimiters, etc. To do that 
really well you'd almost need to add it to the compiler.. hmmm... <JUST 
KIDDING <g>>

What I've noticed is that there are fewer opportunities for inlining then 
you might think in DWT and in general for anything non-trivial... For 
example, any fbody with a loop in it is not a candidate, which generally is 
a real good thing, I think. OTOH, Phobos std.math[2] has just about 
everything in it inlined, which is also a good thing, since most of those 
are return statements basically codifying what the name of the function 
implies.

There was a great article a while back in Dr. Dobbs with an inlining 
algorithm that made a lot of sense, and I think the DMD frontend heuristics 
and the article coincide quite a bit (for the record, DMD had implemented 
the inlining long before the article came out ;). The reason I mention this 
is because I think compiler developers are starting to nail down what should 
be inlined pretty well, so the inlining is good to have in the reference 
compiler and reasonable to have in the -H tool too.


 -JJR
Dec 31 2005
prev sibling next sibling parent "Derek Parnell" <derek psych.ward> writes:
On Fri, 30 Dec 2005 07:40:24 +1100, Walter Bright  
<newshound digitalmars.com> wrote:


 "Derek Parnell" <derek psych.ward> wrote in message
 news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 In other words, why does the generated header show implementation code  
 and
 why does it show private members?


 It shows implementation code for functions that are candidates for  
 inlining.


Thanks for explaining this. I don't like it one bit, though. The DI file  
can be used instead of the original source when the implementation is in a  
library right? How does the compiler know if something has been inlined or  
not in the library object? Just because a specific edition of DMD reckons  
that a function is a candidate for inlining, it cannot know that some  
other edition of some other D compiler has actually inlined it. Also I  
thought that a DI file is not being used to generate object code, but just  
to get information about how to compile the other (non) DI fi


 It shows private members because they might be used in const initializers
 and in inline functions.


But 'private' means that I don't want other modules knowing about this.


 If a function cannot be inlined, its implementation will not be included.


So we must make functions we want to hide deliberately complex to prevent  
inlining? This just sounds wrong. Isn't there a way to turn off inlining  
if I don't want it?

I wanted to use DI files to *HIDE* implementation details. Your definition  
of DI does not do this.

-- 
Derek Parnell
Melbourne, Australia
Dec 29 2005
prev sibling parent reply "Chris Miller" <chris dprogramming.com> writes:
On Thu, 29 Dec 2005 15:40:24 -0500, Walter Bright  
<newshound digitalmars.com> wrote:


 "Derek Parnell" <derek psych.ward> wrote in message
 news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 In other words, why does the generated header show implementation code  
 and
 why does it show private members?


 It shows implementation code for functions that are candidates for  
 inlining.
 It shows private members because they might be used in const initializers
 and in inline functions.

 If a function cannot be inlined, its implementation will not be included.


Perhaps it should only do this if -inline is present.
Dec 29 2005
parent reply John Reimer <terminal.node gmail.com> writes:
Chris Miller wrote:

 On Thu, 29 Dec 2005 15:40:24 -0500, Walter Bright 
 <newshound digitalmars.com> wrote:
 

 "Derek Parnell" <derek psych.ward> wrote in message
 news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 In other words, why does the generated header show implementation 
 code and
 why does it show private members?


 It shows implementation code for functions that are candidates for 
 inlining.
 It shows private members because they might be used in const initializers
 and in inline functions.

 If a function cannot be inlined, its implementation will not be included.


 
 Perhaps it should only do this if -inline is present.


What happens, then, if you use "-inline" on the library and no "-inline" 
on the project that uses the library?  You get an interface file that's 
prepared for inline code per the library build (with all the extra 
private and function implementation details), but a project file that 
uses an interface file that won't be inlined in the project.  Kind of 
complicated.  I think it's just better to disable inlining on libraries 
that generate an interface file and on projects that uses that interface 
file.

-JJR
Dec 29 2005
parent John Reimer <terminal.node gmail.com> writes:
John Reimer wrote:

 Chris Miller wrote:

 On Thu, 29 Dec 2005 15:40:24 -0500, Walter Bright 
 <newshound digitalmars.com> wrote:


 "Derek Parnell" <derek psych.ward> wrote in message
 news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 In other words, why does the generated header show implementation 
 code and
 why does it show private members?


 It shows implementation code for functions that are candidates for 
 inlining.
 It shows private members because they might be used in const 
 initializers
 and in inline functions.

 If a function cannot be inlined, its implementation will not be 
 included.


 Perhaps it should only do this if -inline is present.


 
 What happens, then, if you use "-inline" on the library and no "-inline" 
 on the project that uses the library?  You get an interface file that's 
 prepared for inline code per the library build (with all the extra 
 private and function implementation details), but a project file that 
 uses an interface file that won't be inlined in the project.  Kind of 
 complicated.  I think it's just better to disable inlining on libraries 
 that generate an interface file and on projects that uses that interface 
 file.
 
 -JJR


I should qualify the above.  "-inline" should be allowed on the project 
file as usual, but only for items that don't pertain to the interface file.
Dec 29 2005
prev sibling next sibling parent reply "Kris" <fu bar.com> writes:
Regarding Derek's example: what exactly is the value in an "import file" 
like that, Walter? It bears little or no resemblance to what any related 
posts over the last two years have been requesting, and seems to break all 
understood rules vis-a-vis visability.

It certainly appears to have little bearing on the primary request point ~ 
that is, taking a non OSS component and exposing the minimal attributes to 
make it usable/linkable. The example below explicitly exposes the inner 
workings (private attributes also!), which is totally contrary to said 
requested purpose.

I really don't get the point of this at all, and it seems like others are in 
the same boat. Can you enlighten us, please? It's not April 1st yet, right? 
:)

- Kris


"Derek Parnell" <derek psych.ward> wrote in message 
news:op.s2kab6y06b8z09 ginger.vic.bigpond.net.au...

 On Fri, 30 Dec 2005 06:11:54 +1100, Walter Bright 
 <newshound digitalmars.com> wrote:


 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!


 Well it might be working, but it doesn't do what I was expecting.

 This source file ...

 //--------------------------------
 import std.stdio;

 private int qwerty;
 class A
 {
     private int x;
     void ths()
     {
       x = 1;
       }
 }

 void xpub()
 {
     A a = new A;
     qwerty = 1;
 }

 private void xpriv()
 {
     A b = new A;
     qwerty = 2;
 }
 //--------------------------------


 was turned into this 'header' ...

 // D import file generated from 'test.d'
 import std.stdio;
 private
 {
     int qwerty;
 }
 class A
 {
     private
 {
     int x;
 }
     void ths()
 {
 x = 1;
 }
 }
 void xpub()
 {
 A a = new A;
 qwerty = 1;
 }
 private
 {
     void xpriv()
 {
 A b = new A;
 qwerty = 2;
 }
 }
 //-------------------------

 but I was expecting something more like this ...

 // D import file generated by hand
 import std.stdio;
 class A
 {
     void ths(){}
 }
 void xpub(){}
 //-------------------------


 In other words, why does the generated header show implementation code and 
 why does it show private members? Currently, all it seems to do is 
 reformat the original source by placing braces around private sections.
 -- 
 Derek Parnell
 Melbourne, Australia
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Kris" <fu bar.com> wrote in message news:dp1on5$vsn$1 digitaldaemon.com...

 I really don't get the point of this at all, and it seems like others are 
 in the same boat. Can you enlighten us, please? It's not April 1st yet, 
 right? :)


Sure. See my other post in this thread.
Dec 29 2005
parent reply "Kris" <fu bar.com> writes:
You suggest that using Interfaces is the right way to hide implementation 
detail. I'd agree, but how does one expose the means of accessing an 
instance? Does one provide an interface factory, and expose a header for 
that?

How is it supposed to work?


"Walter Bright" <newshound digitalmars.com> wrote in message 
news:dp1pbv$10i7$2 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp1on5$vsn$1 digitaldaemon.com...

 I really don't get the point of this at all, and it seems like others are 
 in the same boat. Can you enlighten us, please? It's not April 1st yet, 
 right? :)


 Sure. See my other post in this thread.
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Kris" <fu bar.com> wrote in message news:dp1sn8$12rr$1 digitaldaemon.com...

 You suggest that using Interfaces is the right way to hide implementation 
 detail. I'd agree, but how does one expose the means of accessing an 
 instance? Does one provide an interface factory, and expose a header for 
 that?


That's the classic way of doing it.
Dec 29 2005
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 29 Dec 2005 16:03:22 -0800, Walter Bright wrote:


 "Kris" <fu bar.com> wrote in message news:dp1sn8$12rr$1 digitaldaemon.com...

 You suggest that using Interfaces is the right way to hide implementation 
 detail. I'd agree, but how does one expose the means of accessing an 
 instance? Does one provide an interface factory, and expose a header for 
 that?


 
 That's the classic way of doing it.


Can someone translate this into an example or two?

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 11:18:52 AM
Dec 29 2005
parent reply "Kris" <fu bar.com> writes:
Here's an arbitrary example, Derek ~ for a closed-source SearchEngine. I 
think Walter is the only one who can explain how -H can help here:

~~~~~~~~~~~~~~~~~~~
module SearchFactory;

// these interfaces would typically live in other modules

// the hidden implementation
interface ISearchEngine
{
        ISearchItem[] lookup (char[] terms);
}

// more hidden implementation
interface ISearchItem
{
        int relevancy();
        char[] retrievalUrl();
        char[] displaySnippet();
}


// import a.bunch.of.proprietary.stuff;


// a factory for hooking up a client to the hidden implementation
ISearchEngine createSearchEngine (char[] someAttributes)
{
        // potentially use a.bunch.of.proprietary.stuff;
}


// do some proprietary setup
static this()
{
        // potentially use a.bunch.of.proprietary.stuff;
}

~~~~~~~~~~~~~~~~~~~~~

unfortunately, this is what I get when using the -H option:
dmd.exe has encountered a problem and needs to close. We are sorry for the 
inconvenience.


GPF's aside, I imagine that -H would expose the content of both the 
createSearchEngine() factory method, and whatever else is in there (such as 
the static ctor). This means that the all relevant proprietary import 
modules will also need to be shipped with this module :-)

The idea of an Interface is to cleanly isolate a "contractual obligation" 
from any and all implementation detail. I could easily be wrong, but I fail 
to see how this can be achieved with D (even in a trivial example) without 
rather careful attention to detail.

If I were to create a "di" file by hand, then it might look something like 
this:

~~~~~~~~~~~~~~~~~~~~~~~
module importSearchEngine;

// expose the interfaces
public import search.ISearchEngine;
public import search.ISearchItem;

ISearchEngine createSearchEngine (char[] someAttributes);
~~~~~~~~~~~~~~~~~~~~~~~~

This should work correctly (note the lack of implementation detail). 
However, it bears little or no resemblance to the -H option. Thus -H is 
worthless for such things ~ I think you have to build them by hand instead 
...

:-(





"Derek Parnell" <derek psych.ward> wrote in message 
news:bm7jarnr2vgq$.ddyfy4fndls1.dlg 40tude.net...

 On Thu, 29 Dec 2005 16:03:22 -0800, Walter Bright wrote:


 "Kris" <fu bar.com> wrote in message 
 news:dp1sn8$12rr$1 digitaldaemon.com...

 You suggest that using Interfaces is the right way to hide 
 implementation
 detail. I'd agree, but how does one expose the means of accessing an
 instance? Does one provide an interface factory, and expose a header for
 that?


 That's the classic way of doing it.


 Can someone translate this into an example or two?

 -- 
 Derek
 (skype: derek.j.parnell)
 Melbourne, Australia
 "A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
 30/12/2005 11:18:52 AM
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Kris" <fu bar.com> wrote in message news:dp20nn$15ns$1 digitaldaemon.com...

 Here's an arbitrary example, Derek ~ for a closed-source SearchEngine. I 
 think Walter is the only one who can explain how -H can help here:

 ~~~~~~~~~~~~~~~~~~~
 module SearchFactory;

 // these interfaces would typically live in other modules

 // the hidden implementation
 interface ISearchEngine
 {
        ISearchItem[] lookup (char[] terms);
 }

 // more hidden implementation
 interface ISearchItem
 {
        int relevancy();
        char[] retrievalUrl();
        char[] displaySnippet();
 }


 // import a.bunch.of.proprietary.stuff;


 // a factory for hooking up a client to the hidden implementation
 ISearchEngine createSearchEngine (char[] someAttributes)
 {
        // potentially use a.bunch.of.proprietary.stuff;
 }


 // do some proprietary setup
 static this()
 {
        // potentially use a.bunch.of.proprietary.stuff;
 }

 ~~~~~~~~~~~~~~~~~~~~~

 unfortunately, this is what I get when using the -H option:
 dmd.exe has encountered a problem and needs to close. We are sorry for the 
 inconvenience.


I'd like an example of that, so I can fix it.


 GPF's aside, I imagine that -H would expose the content of both the 
 createSearchEngine() factory method, and whatever else is in there (such 
 as the static ctor). This means that the all relevant proprietary import 
 modules will also need to be shipped with this module :-)


No, it won't if the function is too complex to be inlined, which is the 
usual case.


 The idea of an Interface is to cleanly isolate a "contractual obligation" 
 from any and all implementation detail. I could easily be wrong, but I 
 fail to see how this can be achieved with D (even in a trivial example) 
 without rather careful attention to detail.


A good example of using interfaces is COM programming under Windows. The 
general idea is:

interface Foo
{
    T method1(args);
    U method2(args);
 }

Foo FooFactory(args);

and you call FooFactory(args) to create an instance of Foo, instead of using 
new.
Dec 29 2005
next sibling parent reply "Kris" <fu bar.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote
[snip]

 unfortunately, this is what I get when using the -H option:
 dmd.exe has encountered a problem and needs to close. We are sorry for 
 the inconvenience.


 I'd like an example of that, so I can fix it.


Uh, the very example was included right above your comment?



 GPF's aside, I imagine that -H would expose the content of both the 
 createSearchEngine() factory method, and whatever else is in there (such 
 as the static ctor). This means that the all relevant proprietary import 
 modules will also need to be shipped with this module :-)


 No, it won't if the function is too complex to be inlined, which is the 
 usual case.


I see. But, surely that makes it non-deterministic in a variety of manners? 
Thus, one could not always depend on such a header being generated 
correctly?
Dec 29 2005
parent "Walter Bright" <newshound digitalmars.com> writes:
"Kris" <fu bar.com> wrote in message news:dp27jl$19bc$1 digitaldaemon.com...

 "Walter Bright" <newshound digitalmars.com> wrote
 [snip]

 unfortunately, this is what I get when using the -H option:
 dmd.exe has encountered a problem and needs to close. We are sorry for 
 the inconvenience.


 I'd like an example of that, so I can fix it.


 Uh, the very example was included right above your comment?


Ok. I didn't realize that. Thanks!


 GPF's aside, I imagine that -H would expose the content of both the 
 createSearchEngine() factory method, and whatever else is in there (such 
 as the static ctor). This means that the all relevant proprietary import 
 modules will also need to be shipped with this module :-)


 No, it won't if the function is too complex to be inlined, which is the 
 usual case.


 I see. But, surely that makes it non-deterministic in a variety of 
 manners? Thus, one could not always depend on such a header being 
 generated correctly?


The code and header are still correct whether it is inlined or not.
Dec 29 2005
prev sibling next sibling parent Derek Parnell <derek psych.ward> writes:
On Thu, 29 Dec 2005 18:13:03 -0800, Walter Bright wrote:


 I'd like an example of that, so I can fix it.


  module cf;
  interface iFoo{}
  class Foo: iFoo{}

C:\> dmd -c -H cf.d

... ABEND ...

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 3:36:47 PM
Dec 29 2005
prev sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 29 Dec 2005 18:13:03 -0800, Walter Bright wrote:

 

 I'd like an example of that, so I can fix it.


Here are two more examples that cause DMD to crash ...
 
 module cf;
 class Foo
 {
     this(){}
 }


-----------------

  module cf;
  static this()
  {
  }


-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 3:40:41 PM
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
Thanks for the examples. I'll take care of it.
Dec 29 2005
parent reply JT <JT_member pathlink.com> writes:
Thanks Walter this import file is a nice feature. :D

And I agree - the pretty printing is usefull to have in the front end. I noticed
while playing around in the front end that you have stubbed out the posibility
of using 'precompiled symbol files' as imports - do you still have plans for
this and do you expect it to be in version 1?
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"JT" <JT_member pathlink.com> wrote in message 
news:dp2ins$1gn3$1 digitaldaemon.com...

 And I agree - the pretty printing is usefull to have in the front end. I 
 noticed
 while playing around in the front end that you have stubbed out the 
 posibility
 of using 'precompiled symbol files' as imports - do you still have plans 
 for
 this and do you expect it to be in version 1?


.di files replace precompiled symbol files.
Dec 30 2005
parent JT <JT_member pathlink.com> writes:
In article <dp50o3$bt6$1 digitaldaemon.com>, Walter Bright says...

"JT" <JT_member pathlink.com> wrote in message 
news:dp2ins$1gn3$1 digitaldaemon.com...

 And I agree - the pretty printing is usefull to have in the front end. I 
 noticed
 while playing around in the front end that you have stubbed out the 
 posibility
 of using 'precompiled symbol files' as imports - do you still have plans 
 for
 this and do you expect it to be in version 1?


.di files replace precompiled symbol files. 


very cool. .di files make much more sense now, in fact I think its a very good
solution for this part. Thanks for everything you do Walter.
Dec 30 2005
prev sibling parent reply "Kris" <fu bar.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote

 "Kris" <fu bar.com> wrote in message 
 news:dp1sn8$12rr$1 digitaldaemon.com...

 You suggest that using Interfaces is the right way to hide implementation 
 detail. I'd agree, but how does one expose the means of accessing an 
 instance? Does one provide an interface factory, and expose a header for 
 that?


 That's the classic way of doing it.


Then, this related "di" would not expose anything private or potentially 
"sensitive" about the interface-factory itself?

Can you perhaps illustrate with a working example, please? I think that 
would help a lot :)
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Kris" <fu bar.com> wrote in message news:dp1ufk$142q$1 digitaldaemon.com...

 "Walter Bright" <newshound digitalmars.com> wrote

 "Kris" <fu bar.com> wrote in message 
 news:dp1sn8$12rr$1 digitaldaemon.com...

 You suggest that using Interfaces is the right way to hide 
 implementation detail. I'd agree, but how does one expose the means of 
 accessing an instance? Does one provide an interface factory, and expose 
 a header for that?


 That's the classic way of doing it.


 Then, this related "di" would not expose anything private or potentially 
 "sensitive" about the interface-factory itself?

 Can you perhaps illustrate with a working example, please? I think that 
 would help a lot :)


.di files aren't necessarilly going to help you here. But in another post in 
this thread, I gave an example of FooFactory() that is the usual way of 
generating an instance of an interface.
Dec 29 2005
parent "Kris" <fu bar.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote

 "Kris" <fu bar.com> wrote

 Can you perhaps illustrate with a working example, please? I think that 
 would help a lot :)


 .di files aren't necessarilly going to help you here. But in another post 
 in this thread, I gave an example of FooFactory() that is the usual way of 
 generating an instance of an interface.



As did I, which showed two things:

a) -H is of little realistic value (if any) for exposing factories used for 
purposes of code hiding. In fact, it just gets in the way and should not be 
used for said purpose.

b) -H currently fails if you try to do the above (example in the prior 
post).

I think there's been a total miscommunication here. Seems like most everyone 
was expecting some form of code-hiding per the long-standing requests for 
closed-source support (certainly I was thoroughly misguided). Instead, 
what -H does is some form of source-munging that makes it questionably 
faster for the compiler to parse?

I recall something from a long time back noting that just such a facility 
would be implemented via a binary format? As such, it would very likely be 
much faster than the current -H approach (which requires full lexing and 
parsing of the exposed source).

Without wishing to offend anyone, it seems like what's been implemented will 
be unlikely to make a /blind bit of difference/ to compile times (in 
contrast to that binary-format version you had mentioned so long ago). Yes, 
it's great that you took an opportunity to clean up a bunch of 
output-formatting issues, but  ....  this new 'feature' appears to exist 
simply and only because it can? I guess one could split hairs over that, all 
day.

What this feature /does/ do is give the user yet another darned file-type to 
bother about, and provides a non-deterministic means to reformat ones 
source-code. That just doesn't seem like such a great idea, somehow. Oh 
well.
Dec 29 2005
prev sibling parent reply Derek Parnell <derek psych.ward> writes:
On Fri, 30 Dec 2005 07:10:44 +1100, Derek Parnell wrote:

Ok, how about we try to get both camps happy about this DI file concept.
Here is a suggestion that does mean a change to the language though...
 
Given this source file ...
 
 //--------------------------------
 import std.stdio;
 
 private int  qwerty;
 private real asdfgh;

 pragma(generate): // Turn inlining off from here on.
 
 class A
 {
      private int x;
      void ths()
      {
        x = 1;
        }
 }
 
 void xpub()
 {
      A a = new A;
      qwerty = 1;
 }
 
 private void xpriv()
 {
      A b = new A;
      qwerty = 2;
 }
 //--------------------------------
 

It could be turned into this 'header' ...

 // D import file generated from 'test.d'
 import std.stdio;
 reserve:14; // fourteen bytes reserved
 class A
 {
  reserve:4; // four bytes reserved
  void ths() { }
 }
 void xpub() { }
 //-------------------------


In other words, let's enhance the language so we can support the needs of
developers who want closed-source development *and* the needs of the
compiler to generate correctly performing code. This would mean we need to
resolve the issue of class member offsets, inlining, and const
declarations. These are not impossible to resolve.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 9:56:10 AM
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Derek Parnell" <derek psych.ward> wrote in message 
news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


There's more than that, there are template bodies. At some point, we have to 
ask if we've just got the wrong approach to implementation hiding. I suggest 
that interface classes are the way to get true implementation hiding, not di 
files. di files' main advantage is improving build speeds for large 
projects.

They work a lot like "precompiled headers" do in C++.
Dec 29 2005
next sibling parent reply James Dunne <james.jdunne gmail.com> writes:
Walter Bright wrote:

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...
 

In other words, let's enhance the language so we can support the needs of
developers who want closed-source development *and* the needs of the
compiler to generate correctly performing code. This would mean we need to
resolve the issue of class member offsets, inlining, and const
declarations. These are not impossible to resolve.


 
 
 There's more than that, there are template bodies. At some point, we have to 
 ask if we've just got the wrong approach to implementation hiding. I suggest 
 that interface classes are the way to get true implementation hiding, not di 
 files. di files' main advantage is improving build speeds for large 
 projects.
 
 They work a lot like "precompiled headers" do in C++. 
 
 


Understood.

How about a separate interface class generation feature then?

If it were up to me I would rename the 'import file' feature to 
'precompiled header', hoping that the term is not 
copyrighted/trademarked by Microsoft. =)  Then again, if they work 'a 
lot like' precompiled headers in C++ then what are the main differences 
if the two are not exactly alike?

For the closed source community, one could also invent a specially 
formatted D import source file which declares only the non-private 
methods of classes *and* at which offsets they can be found at in the 
vtable.  Of course include the public fields and at which offsets they 
are found at as well.
Dec 29 2005
next sibling parent "Walter Bright" <newshound digitalmars.com> writes:
"James Dunne" <james.jdunne gmail.com> wrote in message 
news:dp1t3o$1383$1 digitaldaemon.com...

 Walter Bright wrote:

 They work a lot like "precompiled headers" do in C++.


 Understood.
 How about a separate interface class generation feature then?
 If it were up to me I would rename the 'import file' feature to 
 'precompiled header', hoping that the term is not copyrighted/trademarked 
 by Microsoft. =)  Then again, if they work 'a lot like' precompiled 
 headers in C++ then what are the main differences if the two are not 
 exactly alike?


The main difference is that precompiled headers are an abominable kludge 
<g>. Precompiled headers are order dependent, and cannot be included 
multiple times. Some things cannot be put in headers to be precompiled. 
Precompiled headers tend to aggregate all the headers together. They are 
very sensitive to compiler switch settings, meaning they need to be 
constantly rebuilt.

None of these apply to D interface files.


 For the closed source community, one could also invent a specially 
 formatted D import source file which declares only the non-private methods 
 of classes *and* at which offsets they can be found at in the vtable.  Of 
 course include the public fields and at which offsets they are found at as 
 well.


Because of templates, const initializers, etc., I suspect that although this 
is doable, it will be of marginal utility.
Dec 29 2005
prev sibling parent reply BCS <BCS_member pathlink.com> writes:
Why dose the order of the function in a class define the order of there 
entries in the vtbl? This seems to me like an unnecessary complication 
that could create some vary difficult to find bugs.

Example:
Say an imported file gets changed but never gets recompiled (maybe it's 
a header file to a closed library)

1) compile main so that it imports new a.
2) compile old a
3) link

This won't have any errors but will end up calling the wrong fn at run time.
----------------
module a;	// old a

class C
{
	int fn1() {return 2;}
	int fn2() {return 3;}
}
----------------
module a;	// new a

class C
{
	int fn2() {return 3;}	// note: order reversed
	int fn1() {return 2;}
}
----------------
module main;
import a;
import std.stdio;

void main()
{
	auto c = new C;
	writef("hello world",\n);
	int i = c.fn1();
	writef("hello world ", i,\n);
}
----------------


I know next to nothing about COFF/ELF/etc. so this suggestion may be 
impossible to implement but here goes.

How about only calculate the offsets for the vtbl once, when you compile 
the class they are in. Then in the generated object file you store some 
symbols that are the offsets. When another module imports the file, the 
code is generated to make the virtual function call but the offset is 
left as an unresolved external that must be resolved at link time. 
Furthermore, this would solve many of the problems about code hiding in 
much the same way as James Dunne's suggestion



James Dunne wrote:
[...]

 
 For the closed source community, one could also invent a specially 
 formatted D import source file which declares only the non-private 
 methods of classes *and* at which offsets they can be found at in the 
 vtable.  Of course include the public fields and at which offsets they 
 are found at as well.
Dec 30 2005
parent BCS <BCS_member pathlink.com> writes:
By the way, this bug is still around.

http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/5671
Dec 30 2005
prev sibling next sibling parent reply "Kris" <fu bar.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote in
[snip]

 di files' main advantage is improving build speeds for large projects.


I regularly compile 200+ files on a mediocre desktop. Takes about 3 seconds 
when using Build (which eats perhaps a third or a half of that time). I 
would not consider it worthwhile using "di" files until the file count were 
up in the thousands ... from that perspective, the current value of "di" 
appears truly nebulous. It will likely remain so for the overwhelming 
majority of users; for eternity.

While I always applaud your progress with the language and the compiler, 
don't you think this stands alone as blatant feature-creep? With respect, I 
think we'd all be better off without -H until an actual need for it arises ~ 
at that point we might even have some /useful/ requirements for it, since 
implementation hiding (what we all requested) is apparently not what it's 
intended for :-)

Respectfully;
Dec 29 2005
parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Kris" <fu bar.com> wrote in message news:dp1tf9$13ds$1 digitaldaemon.com...

 While I always applaud your progress with the language and the compiler, 
 don't you think this stands alone as blatant feature-creep? With respect, 
 I think we'd all be better off without -H until an actual need for it 
 arises ~ at that point we might even have some /useful/ requirements for 
 it, since implementation hiding (what we all requested) is apparently not 
 what it's intended for :-)


Since Dave had already implemented it, and it would be needed eventually 
anyway, there was good reason to put it in. Furthermore, most of the work 
involved improving the pretty-printing capability of the compiler, much of 
which was stubbed out or kludged and needed to be done properly anyway. The 
pretty-printing capability is used for things like formatting of error 
messages and the doc generator.

(For example, now expressions get properly parenthesized when printed, and 
numerous problems printing numeric and string literals were fixed.)

These fixes weren't necessary for DMD to compile programs successfully, but 
they do contribute to it looking like a polished, presentable product.
Dec 29 2005
parent John Reimer <terminal.node gmail.com> writes:
Walter Bright wrote:

 "Kris" <fu bar.com> wrote in message news:dp1tf9$13ds$1 digitaldaemon.com...

 While I always applaud your progress with the language and the compiler, 
 don't you think this stands alone as blatant feature-creep? With respect, 
 I think we'd all be better off without -H until an actual need for it 
 arises ~ at that point we might even have some /useful/ requirements for 
 it, since implementation hiding (what we all requested) is apparently not 
 what it's intended for :-)


 
 Since Dave had already implemented it, and it would be needed eventually 
 anyway, there was good reason to put it in. Furthermore, most of the work 
 involved improving the pretty-printing capability of the compiler, much of 
 which was stubbed out or kludged and needed to be done properly anyway. The 
 pretty-printing capability is used for things like formatting of error 
 messages and the doc generator.
 
 (For example, now expressions get properly parenthesized when printed, and 
 numerous problems printing numeric and string literals were fixed.)
 
 These fixes weren't necessary for DMD to compile programs successfully, but 
 they do contribute to it looking like a polished, presentable product. 
 
 


That's definitely a plus.  If these are the only benefits of the 
addition, I can appreciate them.

-JJR
Dec 29 2005
prev sibling next sibling parent reply "Zz" <junkie noware.com> writes:
"Walter Bright" <newshound digitalmars.com> wrote in message 
news:dp1s4k$12ev$1 digitaldaemon.com...

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need 
 to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


 There's more than that, there are template bodies. At some point, we have 
 to ask if we've just got the wrong approach to implementation hiding. I 
 suggest that interface classes are the way to get true implementation 
 hiding, not di files. di files' main advantage is improving build speeds 
 for large projects.

 They work a lot like "precompiled headers" do in C++.


I'm not suggesting that the solution below in worth using but it's a good 
idea to look into.

Quite some time ago I used to do some work in Modula2 and the way 
implementation and interface were divided was very intutive, over time I've 
got used to other methods and eventually felt comfortable with them.

This link gives a short overview
http://www.csc.twu.ca/rsbook/Ch6/Ch6.5.html

Oberon2 which came later on removed this concept but it was later on 
re-introduced.

for eg in Modula2:
DEFINITION MODULE MyRealMath;

CONST
  root2 = 1.414213562;

PROCEDURE log (x : REAL) : REAL;

END MyRealMath.and in another file:IMPLEMENTATION MODULE MyRealMath;  (* 
note same name *)

FROM RealMath IMPORT
  ln;

PROCEDURE log (x : REAL) : REAL;
(* returns the base 10 logarithm of x *)
BEGIN
  RETURN (ln (x)) / (ln (10.0) )
END log;

END MyRealMath.user file:MODULE Client;

FROM MyRealMath IMPORT
  root2, log;
Dec 29 2005
parent reply John Reimer <terminal.node gmail.com> writes:
Zz wrote:

 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:dp1s4k$12ev$1 digitaldaemon.com...

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need 
 to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


 There's more than that, there are template bodies. At some point, we have 
 to ask if we've just got the wrong approach to implementation hiding. I 
 suggest that interface classes are the way to get true implementation 
 hiding, not di files. di files' main advantage is improving build speeds 
 for large projects.

 They work a lot like "precompiled headers" do in C++.


 
 I'm not suggesting that the solution below in worth using but it's a good 
 idea to look into.
 
 Quite some time ago I used to do some work in Modula2 and the way 
 implementation and interface were divided was very intutive, over time I've 
 got used to other methods and eventually felt comfortable with them.
 
 This link gives a short overview
 http://www.csc.twu.ca/rsbook/Ch6/Ch6.5.html
 
 Oberon2 which came later on removed this concept but it was later on 
 re-introduced.
 
 for eg in Modula2:
 DEFINITION MODULE MyRealMath;
 
 CONST
   root2 = 1.414213562;
 
 PROCEDURE log (x : REAL) : REAL;
 
 END MyRealMath.and in another file:IMPLEMENTATION MODULE MyRealMath;  (* 
 note same name *)
 
 FROM RealMath IMPORT
   ln;
 
 PROCEDURE log (x : REAL) : REAL;
 (* returns the base 10 logarithm of x *)
 BEGIN
   RETURN (ln (x)) / (ln (10.0) )
 END log;
 
 END MyRealMath.user file:MODULE Client;
 
 FROM MyRealMath IMPORT
   root2, log; 
 
 


I studied some Modula 2 more than 10 years ago, so I don't remember too 
much.  But I do remember the implementation/definition file separation. 
  Were those given different file endings? I think Modula 3 (which does 
somewhat the same, I believe) uses i3 for interface and m3 for 
implementation.  It does look like an effective solution, although I 
don't know how such a solution would fit into a modern language like D, 
that has features that none of these early languages ever had.

-JJR
Dec 29 2005
parent reply "Zz" <junkie noware.com> writes:
"John Reimer" <terminal.node gmail.com> wrote in message 
news:dp29ai$1a9l$1 digitaldaemon.com...

 Zz wrote:

 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:dp1s4k$12ev$1 digitaldaemon.com...

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs 
 of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need 
 to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


 There's more than that, there are template bodies. At some point, we 
 have to ask if we've just got the wrong approach to implementation 
 hiding. I suggest that interface classes are the way to get true 
 implementation hiding, not di files. di files' main advantage is 
 improving build speeds for large projects.

 They work a lot like "precompiled headers" do in C++.


 I'm not suggesting that the solution below in worth using but it's a good 
 idea to look into.

 Quite some time ago I used to do some work in Modula2 and the way 
 implementation and interface were divided was very intutive, over time 
 I've got used to other methods and eventually felt comfortable with them.

 This link gives a short overview
 http://www.csc.twu.ca/rsbook/Ch6/Ch6.5.html

 Oberon2 which came later on removed this concept but it was later on 
 re-introduced.

 for eg in Modula2:
 DEFINITION MODULE MyRealMath;

 CONST
   root2 = 1.414213562;

 PROCEDURE log (x : REAL) : REAL;

 END MyRealMath.and in another file:IMPLEMENTATION MODULE MyRealMath;  (* 
 note same name *)

 FROM RealMath IMPORT
   ln;

 PROCEDURE log (x : REAL) : REAL;
 (* returns the base 10 logarithm of x *)
 BEGIN
   RETURN (ln (x)) / (ln (10.0) )
 END log;

 END MyRealMath.user file:MODULE Client;

 FROM MyRealMath IMPORT
   root2, log;


 I studied some Modula 2 more than 10 years ago, so I don't remember too 
 much.  But I do remember the implementation/definition file separation. 
 Were those given different file endings? I think Modula 3 (which does 
 somewhat the same, I believe) uses i3 for interface and m3 for 
 implementation.  It does look like an effective solution, although I don't 
 know how such a solution would fit into a modern language like D, that has 
 features that none of these early languages ever had.


Stony Brook and Topspeed Modula2 used .MOD for Implementation and .DEF for 
Definition.
As for fitting within a modern language like D, I wonder why not (at least 
it's better than the -H option).

In Stonnt Brook M2 the libs were generated from the DEF and lib and all the 
user needed was the lib and def to do any thing, the def module did not 
contain anything (just like D's interface but in a separate file with the 
option to include other def's).

Here is an example of  ctrl3d.

First the DEF module:

DEFINITION MODULE CTL3D;

(* Notice that you can include other Modules here*)
FROM WIN32 IMPORT
    HWND;

(* functions to interface with the CTL3D32.DLL for Windows NT *)
(* most function do nothing on WIN95, actually any Windows version 4.x *)

PROCEDURE Ctl3dLoaded() : BOOLEAN;
(* returns TRUE if CTL3D32 is loaded and initialized *)
(* CTL3D32 will not be loaded if system supports DS_3DLOOK dialogs *)

PROCEDURE MakeAllDialogs3d;
(* if Ctl3dLoaded() = TRUE, then forces all dialogs to 3D *)

END CTL3D.

and now the MOD (implementation) module.
Here I pasted more just to give you more detail on what is going on.

IMPLEMENTATION MODULE CTL3D;
<*/NOWARN:F/INLINE:N*>

FROM SYSTEM IMPORT
    ADDRESS;

FROM WIN32 IMPORT
    HWND, DWORD, BOOL, HINSTANCE, FARPROC,
    LoadLibrary, FreeLibrary, GetProcAddress,
    OSVERSIONINFO, GetVersionEx;

FROM WINUSER IMPORT
    GetWindowLong, GWL_STYLE, DS_3DLOOK;

FROM WINX IMPORT
    Instance, NULL_HINSTANCE;

VAR
    Have3dDll  : BOOLEAN;
    Dll   : HINSTANCE;

    C3DRegister  : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DUnregister : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DAutoSubclass : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DSubclassDlgEx : PROCEDURE(HWND, DWORD) : BOOL [WIN32SYSTEM];
    C3DColorChange : PROCEDURE() : BOOL [WIN32SYSTEM];

PROCEDURE Load3dDll;
BEGIN
    Dll := LoadLibrary("CTL3D32");
PROCEDURE MakeAllDialogs3d;
BEGIN
    IF Have3dDll THEN
 C3DAutoSubclass(Instance);
    END;
END MakeAllDialogs3d;

BEGIN
    Have3dDll := FALSE;
    IF NOT Builtin3d() THEN
 Load3dDll;
    END;

FINALLY
    IF Have3dDll THEN
 C3DUnregister(Instance);
 FreeLibrary(Dll);
    END;
END CTL3D.

Regards
Zz
Dec 30 2005
next sibling parent reply rko <rko_member pathlink.com> writes:
In article <dp35ru$1vle$1 digitaldaemon.com>, Zz says...

"John Reimer" <terminal.node gmail.com> wrote in message 
news:dp29ai$1a9l$1 digitaldaemon.com...

 Zz wrote:

 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:dp1s4k$12ev$1 digitaldaemon.com...

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs 
 of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need 
 to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


 There's more than that, there are template bodies. At some point, we 
 have to ask if we've just got the wrong approach to implementation 
 hiding. I suggest that interface classes are the way to get true 
 implementation hiding, not di files. di files' main advantage is 
 improving build speeds for large projects.

 They work a lot like "precompiled headers" do in C++.


 I'm not suggesting that the solution below in worth using but it's a good 
 idea to look into.

 Quite some time ago I used to do some work in Modula2 and the way 
 implementation and interface were divided was very intutive, over time 
 I've got used to other methods and eventually felt comfortable with them.

 This link gives a short overview
 http://www.csc.twu.ca/rsbook/Ch6/Ch6.5.html

 Oberon2 which came later on removed this concept but it was later on 
 re-introduced.

 for eg in Modula2:
 DEFINITION MODULE MyRealMath;

 CONST
   root2 = 1.414213562;

 PROCEDURE log (x : REAL) : REAL;

 END MyRealMath.and in another file:IMPLEMENTATION MODULE MyRealMath;  (* 
 note same name *)

 FROM RealMath IMPORT
   ln;

 PROCEDURE log (x : REAL) : REAL;
 (* returns the base 10 logarithm of x *)
 BEGIN
   RETURN (ln (x)) / (ln (10.0) )
 END log;

 END MyRealMath.user file:MODULE Client;

 FROM MyRealMath IMPORT
   root2, log;


 I studied some Modula 2 more than 10 years ago, so I don't remember too 
 much.  But I do remember the implementation/definition file separation. 
 Were those given different file endings? I think Modula 3 (which does 
 somewhat the same, I believe) uses i3 for interface and m3 for 
 implementation.  It does look like an effective solution, although I don't 
 know how such a solution would fit into a modern language like D, that has 
 features that none of these early languages ever had.


Stony Brook and Topspeed Modula2 used .MOD for Implementation and .DEF for 
Definition.
As for fitting within a modern language like D, I wonder why not (at least 
it's better than the -H option).

In Stonnt Brook M2 the libs were generated from the DEF and lib and all the 
user needed was the lib and def to do any thing, the def module did not 
contain anything (just like D's interface but in a separate file with the 
option to include other def's).

Here is an example of  ctrl3d.

First the DEF module:

DEFINITION MODULE CTL3D;

(* Notice that you can include other Modules here*)
FROM WIN32 IMPORT
    HWND;

(* functions to interface with the CTL3D32.DLL for Windows NT *)
(* most function do nothing on WIN95, actually any Windows version 4.x *)

PROCEDURE Ctl3dLoaded() : BOOLEAN;
(* returns TRUE if CTL3D32 is loaded and initialized *)
(* CTL3D32 will not be loaded if system supports DS_3DLOOK dialogs *)

PROCEDURE MakeAllDialogs3d;
(* if Ctl3dLoaded() = TRUE, then forces all dialogs to 3D *)

END CTL3D.

and now the MOD (implementation) module.
Here I pasted more just to give you more detail on what is going on.

IMPLEMENTATION MODULE CTL3D;
<*/NOWARN:F/INLINE:N*>

FROM SYSTEM IMPORT
    ADDRESS;

FROM WIN32 IMPORT
    HWND, DWORD, BOOL, HINSTANCE, FARPROC,
    LoadLibrary, FreeLibrary, GetProcAddress,
    OSVERSIONINFO, GetVersionEx;

FROM WINUSER IMPORT
    GetWindowLong, GWL_STYLE, DS_3DLOOK;

FROM WINX IMPORT
    Instance, NULL_HINSTANCE;

VAR
    Have3dDll  : BOOLEAN;
    Dll   : HINSTANCE;

    C3DRegister  : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DUnregister : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DAutoSubclass : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DSubclassDlgEx : PROCEDURE(HWND, DWORD) : BOOL [WIN32SYSTEM];
    C3DColorChange : PROCEDURE() : BOOL [WIN32SYSTEM];

PROCEDURE Load3dDll;
BEGIN
    Dll := LoadLibrary("CTL3D32");
PROCEDURE MakeAllDialogs3d;
BEGIN
    IF Have3dDll THEN
 C3DAutoSubclass(Instance);
    END;
END MakeAllDialogs3d;

BEGIN
    Have3dDll := FALSE;
    IF NOT Builtin3d() THEN
 Load3dDll;
    END;

FINALLY
    IF Have3dDll THEN
 C3DUnregister(Instance);
 FreeLibrary(Dll);
    END;
END CTL3D.

Regards
Zz



i am his opinion. boy, would it be nice getting someting like the modula
definition file system.

rko
Dec 30 2005
parent "Zz" <junkie noware.com> writes:
 i am his opinion. boy, would it be nice getting someting like the modula
 definition file system.

 rko


Good to see another person with M2 background.

It would be a good feature to have but I don't believe it will ever be 
available in D.
Whenever someone hears of a language that Wirth designed they assume that we 
are talking about Pascal and don't look any furher.

Stonny Brook M2 used this system to do their cross platform stuff and in use 
it worked perfectly, and I don't ever recall having a problem with it.

On the other hand D is a new language which and I really do enjoy doing 
small quick and dirty things with it (I would never dream of doing a large 
project with it witout some form of separation, maybe it's just me but I 
can't see it happening).

Note: I have some stuff done in D running in a commercial production 
enviroment, but they were small programs.

Regards,
Zz
Dec 30 2005
prev sibling next sibling parent rko <rko_member pathlink.com> writes:
In article <dp35ru$1vle$1 digitaldaemon.com>, Zz says...

"John Reimer" <terminal.node gmail.com> wrote in message 
news:dp29ai$1a9l$1 digitaldaemon.com...

 Zz wrote:

 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:dp1s4k$12ev$1 digitaldaemon.com...

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs 
 of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need 
 to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


 There's more than that, there are template bodies. At some point, we 
 have to ask if we've just got the wrong approach to implementation 
 hiding. I suggest that interface classes are the way to get true 
 implementation hiding, not di files. di files' main advantage is 
 improving build speeds for large projects.

 They work a lot like "precompiled headers" do in C++.


 I'm not suggesting that the solution below in worth using but it's a good 
 idea to look into.

 Quite some time ago I used to do some work in Modula2 and the way 
 implementation and interface were divided was very intutive, over time 
 I've got used to other methods and eventually felt comfortable with them.

 This link gives a short overview
 http://www.csc.twu.ca/rsbook/Ch6/Ch6.5.html

 Oberon2 which came later on removed this concept but it was later on 
 re-introduced.

 for eg in Modula2:
 DEFINITION MODULE MyRealMath;

 CONST
   root2 = 1.414213562;

 PROCEDURE log (x : REAL) : REAL;

 END MyRealMath.and in another file:IMPLEMENTATION MODULE MyRealMath;  (* 
 note same name *)

 FROM RealMath IMPORT
   ln;

 PROCEDURE log (x : REAL) : REAL;
 (* returns the base 10 logarithm of x *)
 BEGIN
   RETURN (ln (x)) / (ln (10.0) )
 END log;

 END MyRealMath.user file:MODULE Client;

 FROM MyRealMath IMPORT
   root2, log;


 I studied some Modula 2 more than 10 years ago, so I don't remember too 
 much.  But I do remember the implementation/definition file separation. 
 Were those given different file endings? I think Modula 3 (which does 
 somewhat the same, I believe) uses i3 for interface and m3 for 
 implementation.  It does look like an effective solution, although I don't 
 know how such a solution would fit into a modern language like D, that has 
 features that none of these early languages ever had.


Stony Brook and Topspeed Modula2 used .MOD for Implementation and .DEF for 
Definition.
As for fitting within a modern language like D, I wonder why not (at least 
it's better than the -H option).

In Stonnt Brook M2 the libs were generated from the DEF and lib and all the 
user needed was the lib and def to do any thing, the def module did not 
contain anything (just like D's interface but in a separate file with the 
option to include other def's).

Here is an example of  ctrl3d.

First the DEF module:

DEFINITION MODULE CTL3D;

(* Notice that you can include other Modules here*)
FROM WIN32 IMPORT
    HWND;

(* functions to interface with the CTL3D32.DLL for Windows NT *)
(* most function do nothing on WIN95, actually any Windows version 4.x *)

PROCEDURE Ctl3dLoaded() : BOOLEAN;
(* returns TRUE if CTL3D32 is loaded and initialized *)
(* CTL3D32 will not be loaded if system supports DS_3DLOOK dialogs *)

PROCEDURE MakeAllDialogs3d;
(* if Ctl3dLoaded() = TRUE, then forces all dialogs to 3D *)

END CTL3D.

and now the MOD (implementation) module.
Here I pasted more just to give you more detail on what is going on.

IMPLEMENTATION MODULE CTL3D;
<*/NOWARN:F/INLINE:N*>

FROM SYSTEM IMPORT
    ADDRESS;

FROM WIN32 IMPORT
    HWND, DWORD, BOOL, HINSTANCE, FARPROC,
    LoadLibrary, FreeLibrary, GetProcAddress,
    OSVERSIONINFO, GetVersionEx;

FROM WINUSER IMPORT
    GetWindowLong, GWL_STYLE, DS_3DLOOK;

FROM WINX IMPORT
    Instance, NULL_HINSTANCE;

VAR
    Have3dDll  : BOOLEAN;
    Dll   : HINSTANCE;

    C3DRegister  : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DUnregister : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DAutoSubclass : PROCEDURE(HINSTANCE) : BOOL [WIN32SYSTEM];
    C3DSubclassDlgEx : PROCEDURE(HWND, DWORD) : BOOL [WIN32SYSTEM];
    C3DColorChange : PROCEDURE() : BOOL [WIN32SYSTEM];

PROCEDURE Load3dDll;
BEGIN
    Dll := LoadLibrary("CTL3D32");
PROCEDURE MakeAllDialogs3d;
BEGIN
    IF Have3dDll THEN
 C3DAutoSubclass(Instance);
    END;
END MakeAllDialogs3d;

BEGIN
    Have3dDll := FALSE;
    IF NOT Builtin3d() THEN
 Load3dDll;
    END;

FINALLY
    IF Have3dDll THEN
 C3DUnregister(Instance);
 FreeLibrary(Dll);
    END;
END CTL3D.

Regards
Zz



here is a def file for a class (i took out some of the stuff):

DEFINITION MODULE database;
FROM SQL IMPORT *;
IMPORT EXCEPTIONS;

TYPE
SQL_ENV;
CONST
XALLOCERROR         = -2001;

CLASS DATABASE;
REVEAL StrBaseToCard64, .... , Commit;

TYPE
NumberType = (Annex, Filter);

VAR
myConnection: SQL_ENV;
dbUsed      : DBTYPE;
ExSource    : EXCEPTIONS.ExceptionSource;

PROCEDURE MakeGenericNewNumber(tableName: ARRAY OF CHAR) : CARDINAL64;
....
PROCEDURE Commit() : INTEGER32;
END DATABASE;
END database.
Dec 30 2005
prev sibling parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Zz" <junkie noware.com> wrote in message 
news:dp35ru$1vle$1 digitaldaemon.com...

 In Stonnt Brook M2 the libs were generated from the DEF and lib and all 
 the user needed was the lib and def to do any thing, the def module did 
 not contain anything (just like D's interface but in a separate file with 
 the option to include other def's).


The trouble with that is the usual trouble with interfaces - they 
necessarilly introduce an extra level of indirection to get at the 
methods/data. If there's no way around it, the language implementation is 
never going to attain the needed performance.
Jan 03 2006
parent ZZ <ZZ zz.com> writes:
Walter Bright wrote:

 "Zz" <junkie noware.com> wrote in message 
 news:dp35ru$1vle$1 digitaldaemon.com...
 

In Stonnt Brook M2 the libs were generated from the DEF and lib and all 
the user needed was the lib and def to do any thing, the def module did 
not contain anything (just like D's interface but in a separate file with 
the option to include other def's).


 
 
 The trouble with that is the usual trouble with interfaces - they 
 necessarilly introduce an extra level of indirection to get at the 
 methods/data. If there's no way around it, the language implementation is 
 never going to attain the needed performance. 
 
 


Hi Walter

You have a point about the extra level of indirection, SB M2 allowed one 
to put procedures into the DEF (interface file) and they would be 
inlined in case someone needed that. Just like C++ inline functions.

I guess it all depends on background, I introduced one of the people I 
work with to D (he has to re-write something small I wrote in D to C++) 
and at first glance the thing that he loved was that it didn't have 
header files (I wrote the whole thing in one file).

There is a new thread "Decoupling implementation from usage" that helped 
clear some things up for me.

All in all I've come to really like D and will continue using it.

Thanks
Zz
Jan 03 2006
prev sibling parent reply John Reimer <terminal.node gmail.com> writes:
Walter Bright wrote:

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...

 In other words, let's enhance the language so we can support the needs of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we need to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.


 
 There's more than that, there are template bodies. At some point, we have to 
 ask if we've just got the wrong approach to implementation hiding. I suggest 
 that interface classes are the way to get true implementation hiding, not di 
 files. di files' main advantage is improving build speeds for large 
 projects.
 
 They work a lot like "precompiled headers" do in C++. 
 
 



Okay, now I see why there is so much confusion going on with the 
addition of this feature.  There are two problems to be solved: one is 
the issue of implementation hiding; the other is the issue of compiler 
performance.  I think most of us mistook the di file generator to be a 
solution for implementation hiding for closed source projects. 
Obviously we got the wrong idea about the whole thing.

If di files serve the same purpose as "precompiled headers," obviously 
there is no consideration whatsoever for implementation hiding (and 
there never was any such intention).

Good to know.  Personally, I was less interested in compile performance 
and more interested in implementation hiding.  The -H flag, therefore, 
will not be of much use to me.

-JJR
Dec 29 2005
next sibling parent reply J C Calvarese <technocrat7 gmail.com> writes:
In article <dp27tj$19f5$1 digitaldaemon.com>, John Reimer says...

Okay, now I see why there is so much confusion going on with the 
addition of this feature.  There are two problems to be solved: one is 
the issue of implementation hiding; the other is the issue of compiler 
performance.  I think most of us mistook the di file generator to be a 
solution for implementation hiding for closed source projects. 
Obviously we got the wrong idea about the whole thing.

If di files serve the same purpose as "precompiled headers," obviously 
there is no consideration whatsoever for implementation hiding (and 
there never was any such intention).

Good to know.  Personally, I was less interested in compile performance 
and more interested in implementation hiding.  The -H flag, therefore, 
will not be of much use to me.

-JJR


I have to say as fast as DMD compiles files, I was more interested in seeing
headers stripped to the bone than optimally stripped, too (I really don't want
that to sound as weird as it does, but I think "stripped aggressively" would be
worse). I doubt that I would personally use it for hiding code (my code isn't
interesting enough to bother hiding), but it seems like a neat thing to do. 

But I was thinking... If the magic happens in the (open source) front-end, it
might not be that hard for an enterprising individual to alter the compiler to
leave out the optimizations. I found in "mars.c" where the command line option
"-H" is detected, and I followed the program flow to "hdrgen.c". But that's as
far as I got before I lost interest. Time for sleep anyway. ;)

jcc7
Dec 29 2005
parent John Reimer <terminal.node gmail.com> writes:
J C Calvarese wrote:

 In article <dp27tj$19f5$1 digitaldaemon.com>, John Reimer says...

 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. 
 Obviously we got the wrong idea about the whole thing.

 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).

 Good to know.  Personally, I was less interested in compile performance 
 and more interested in implementation hiding.  The -H flag, therefore, 
 will not be of much use to me.

 -JJR


 
 I have to say as fast as DMD compiles files, I was more interested in seeing
 headers stripped to the bone than optimally stripped, too (I really don't want
 that to sound as weird as it does, but I think "stripped aggressively" would be
 worse). I doubt that I would personally use it for hiding code (my code isn't
 interesting enough to bother hiding), but it seems like a neat thing to do. 
 
 But I was thinking... If the magic happens in the (open source) front-end, it
 might not be that hard for an enterprising individual to alter the compiler to
 leave out the optimizations. I found in "mars.c" where the command line option
 "-H" is detected, and I followed the program flow to "hdrgen.c". But that's as
 far as I got before I lost interest. Time for sleep anyway. ;)
 
 jcc7


It's probably possible.  Or they could just start on a project separate 
from the compiler that creates the headers.  Like Carlos mentioned, digc 
may already do this.

-JJR
Dec 30 2005
prev sibling next sibling parent reply Kyle Furlong <kylefurlong gmail.com> writes:
John Reimer wrote:

 Walter Bright wrote:
 

 "Derek Parnell" <derek psych.ward> wrote in message 
 news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg 40tude.net...


 In other words, let's enhance the language so we can support the 
 needs of
 developers who want closed-source development *and* the needs of the
 compiler to generate correctly performing code. This would mean we 
 need to
 resolve the issue of class member offsets, inlining, and const
 declarations. These are not impossible to resolve.



 There's more than that, there are template bodies. At some point, we 
 have to ask if we've just got the wrong approach to implementation 
 hiding. I suggest that interface classes are the way to get true 
 implementation hiding, not di files. di files' main advantage is 
 improving build speeds for large projects.

 They work a lot like "precompiled headers" do in C++.


 
 
 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. Obviously 
 we got the wrong idea about the whole thing.
 
 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).
 
 Good to know.  Personally, I was less interested in compile performance 
 and more interested in implementation hiding.  The -H flag, therefore, 
 will not be of much use to me.
 
 -JJR


Right on John. Walter, what is your answer for closed source 
implementation hiding?
Dec 30 2005
parent reply John Reimer <terminal.node gmail.com> writes:
Kyle Furlong wrote:

 John Reimer wrote:



 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. 
 Obviously we got the wrong idea about the whole thing.

 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).

 Good to know.  Personally, I was less interested in compile 
 performance and more interested in implementation hiding.  The -H 
 flag, therefore, will not be of much use to me.

 -JJR


 
 Right on John. Walter, what is your answer for closed source 
 implementation hiding?


I think Walter already answered this question in a previous post in this 
thread.  If I understand correctly, he basically detailed how 
implementation hiding is nigh to impossible to do effectively.  I'm not 
sure if he meant that it's not worth trying to program a solution 
because data can't be effectively hidden or that the difficultly is not 
worth the effort.

I just want to clarify that my idea of implementation-hiding really 
didn't entail encrypted libraries.  I know somebody could find out 
implementation details if they really wanted to.  The idea, much like 
the Modula 2 and 3 interface definition files, is merely to separate the 
details: (1) so that it isn't /easy/ to see implementation details and 
(2) so that we have a summary of the features a library provides.  A 
nifty side effect is that there's probably increase compiler performance.

In the end though, maybe something like DDL (www.dsource.org) will solve 
all our problems in some way.  Eliminating the need for the compiler to 
parse a header altogether may be a big step forward.

Also, while I feel that the -H option seems to be more of distraction 
from fixing higher priority items in D, it may do well for us at some 
point; I'm just hoping, though, that all the bugs it introduces don't 
take up Walter's time.

A tool like this could theoretically be separate from the compiler. That 
may be the best solution in the end if it's too much work to maintain. 
I vaguely remember arguments to this end being made months to a year or 
so before.

-JJR
Dec 30 2005
parent reply "Kris" <fu bar.com> writes:
Just to clarify a couple of minor things:

As John noted, implementation hiding is not about encryption ~ it's about 
supporting closed-source development.

D /does/ support implementation hiding, and has done for a long time. What's 
been missing is a nice, simple, automated, standard, and compiler-supported 
way to generate a bridge between the "hidden" implementation, and client 
usage thereof. It's always been done manually, and it's easy to get confused 
over which file is which since there's common module names involved etc.

One way to approach information hiding is to use Interfaces in conjunction 
with a Factory pattern. There's been a couple of trivial examples posted in 
this thread, but the general idea is that Interfaces are used to abstract 
away all implementation details, and one or more public functions are 
exposed in a "bridging module" to create instances on a clients behalf.

Another approach to implementation hiding is to simply strip out existing 
method bodies and leave the bare declarations, along with any enums and/or 
typedefs they may use. Many requests have been made for an automated tool to 
do exactly this: which would also work well for the Interface approach noted 
above. The end result would be the same in both cases ~ a secondary 
"declaration only" file would be produced ~ which would be used by a client 
as a means to get the appropriate code linked. In all cases, the client is 
expected to know nothing of, nor care about, the implementation under the 
hood.

This is where a misunderstanding arose with -H which, as it turns out, has 
no real value with respect to the above. Thus, D does not have a tool to 
help produce a client-binding from an implementation ~ you /still/ have to 
do that manually. That doesn't mean it can't be done, but it can certainly 
be a chore, and it would be good to get some standardization in the 
mechanics of it (such as a standard file-extension, for example, such that 
the conflict over module names can be better managed).

So ~

1) D still needs a much more solid *story* for closed-source development. 
Without it, one could argue it deserves to remain a niche language. Note 
that this is not a technical hurdle, but it's one that has repeatedly failed 
to be passed. All it needs is a little TLC and attention.

2) The -H flag is apparently for something else entirely.


Hope that helps?

- Kris



"John Reimer" <terminal.node gmail.com> wrote

 Kyle Furlong wrote:

 John Reimer wrote:



 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. Obviously 
 we got the wrong idea about the whole thing.

 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).

 Good to know.  Personally, I was less interested in compile performance 
 and more interested in implementation hiding.  The -H flag, therefore, 
 will not be of much use to me.

 -JJR


 Right on John. Walter, what is your answer for closed source 
 implementation hiding?


 I think Walter already answered this question in a previous post in this 
 thread.  If I understand correctly, he basically detailed how 
 implementation hiding is nigh to impossible to do effectively.  I'm not 
 sure if he meant that it's not worth trying to program a solution because 
 data can't be effectively hidden or that the difficultly is not worth the 
 effort.

 I just want to clarify that my idea of implementation-hiding really didn't 
 entail encrypted libraries.  I know somebody could find out implementation 
 details if they really wanted to.  The idea, much like the Modula 2 and 3 
 interface definition files, is merely to separate the details: (1) so that 
 it isn't /easy/ to see implementation details and (2) so that we have a 
 summary of the features a library provides.  A nifty side effect is that 
 there's probably increase compiler performance.

 In the end though, maybe something like DDL (www.dsource.org) will solve 
 all our problems in some way.  Eliminating the need for the compiler to 
 parse a header altogether may be a big step forward.

 Also, while I feel that the -H option seems to be more of distraction from 
 fixing higher priority items in D, it may do well for us at some point; 
 I'm just hoping, though, that all the bugs it introduces don't take up 
 Walter's time.

 A tool like this could theoretically be separate from the compiler. That 
 may be the best solution in the end if it's too much work to maintain. I 
 vaguely remember arguments to this end being made months to a year or so 
 before.

 -JJR
Dec 30 2005
next sibling parent "Kris" <fu bar.com> writes:
I forgot to note that DDL does not help here at all. On the contrary, DDL 
firmly places a generously sized spotlight directly upon the issue.

- Kris


"Kris" <fu bar.com> wrote in message news:dp3sve$2hm8$1 digitaldaemon.com...

 Just to clarify a couple of minor things:

 As John noted, implementation hiding is not about encryption ~ it's about 
 supporting closed-source development.

 D /does/ support implementation hiding, and has done for a long time. 
 What's been missing is a nice, simple, automated, standard, and 
 compiler-supported way to generate a bridge between the "hidden" 
 implementation, and client usage thereof. It's always been done manually, 
 and it's easy to get confused over which file is which since there's 
 common module names involved etc.

 One way to approach information hiding is to use Interfaces in conjunction 
 with a Factory pattern. There's been a couple of trivial examples posted 
 in this thread, but the general idea is that Interfaces are used to 
 abstract away all implementation details, and one or more public functions 
 are exposed in a "bridging module" to create instances on a clients 
 behalf.

 Another approach to implementation hiding is to simply strip out existing 
 method bodies and leave the bare declarations, along with any enums and/or 
 typedefs they may use. Many requests have been made for an automated tool 
 to do exactly this: which would also work well for the Interface approach 
 noted above. The end result would be the same in both cases ~ a secondary 
 "declaration only" file would be produced ~ which would be used by a 
 client as a means to get the appropriate code linked. In all cases, the 
 client is expected to know nothing of, nor care about, the implementation 
 under the hood.

 This is where a misunderstanding arose with -H which, as it turns out, has 
 no real value with respect to the above. Thus, D does not have a tool to 
 help produce a client-binding from an implementation ~ you /still/ have to 
 do that manually. That doesn't mean it can't be done, but it can certainly 
 be a chore, and it would be good to get some standardization in the 
 mechanics of it (such as a standard file-extension, for example, such that 
 the conflict over module names can be better managed).

 So ~

 1) D still needs a much more solid *story* for closed-source development. 
 Without it, one could argue it deserves to remain a niche language. Note 
 that this is not a technical hurdle, but it's one that has repeatedly 
 failed to be passed. All it needs is a little TLC and attention.

 2) The -H flag is apparently for something else entirely.


 Hope that helps?

 - Kris



 "John Reimer" <terminal.node gmail.com> wrote

 Kyle Furlong wrote:

 John Reimer wrote:



 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. 
 Obviously we got the wrong idea about the whole thing.

 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).

 Good to know.  Personally, I was less interested in compile performance 
 and more interested in implementation hiding.  The -H flag, therefore, 
 will not be of much use to me.

 -JJR


 Right on John. Walter, what is your answer for closed source 
 implementation hiding?


 I think Walter already answered this question in a previous post in this 
 thread.  If I understand correctly, he basically detailed how 
 implementation hiding is nigh to impossible to do effectively.  I'm not 
 sure if he meant that it's not worth trying to program a solution because 
 data can't be effectively hidden or that the difficultly is not worth the 
 effort.

 I just want to clarify that my idea of implementation-hiding really 
 didn't entail encrypted libraries.  I know somebody could find out 
 implementation details if they really wanted to.  The idea, much like the 
 Modula 2 and 3 interface definition files, is merely to separate the 
 details: (1) so that it isn't /easy/ to see implementation details and 
 (2) so that we have a summary of the features a library provides.  A 
 nifty side effect is that there's probably increase compiler performance.

 In the end though, maybe something like DDL (www.dsource.org) will solve 
 all our problems in some way.  Eliminating the need for the compiler to 
 parse a header altogether may be a big step forward.

 Also, while I feel that the -H option seems to be more of distraction 
 from fixing higher priority items in D, it may do well for us at some 
 point; I'm just hoping, though, that all the bugs it introduces don't 
 take up Walter's time.

 A tool like this could theoretically be separate from the compiler. That 
 may be the best solution in the end if it's too much work to maintain. I 
 vaguely remember arguments to this end being made months to a year or so 
 before.

 -JJR
Dec 30 2005
prev sibling next sibling parent John Reimer <terminal.node gmail.com> writes:
Kris wrote:

 Just to clarify a couple of minor things:
 
 As John noted, implementation hiding is not about encryption ~ it's about 
 supporting closed-source development.
 
 D /does/ support implementation hiding, and has done for a long time. What's 
 been missing is a nice, simple, automated, standard, and compiler-supported 
 way to generate a bridge between the "hidden" implementation, and client 
 usage thereof. It's always been done manually, and it's easy to get confused 
 over which file is which since there's common module names involved etc.
 
 One way to approach information hiding is to use Interfaces in conjunction 
 with a Factory pattern. There's been a couple of trivial examples posted in 
 this thread, but the general idea is that Interfaces are used to abstract 
 away all implementation details, and one or more public functions are 
 exposed in a "bridging module" to create instances on a clients behalf.
 
 Another approach to implementation hiding is to simply strip out existing 
 method bodies and leave the bare declarations, along with any enums and/or 
 typedefs they may use. Many requests have been made for an automated tool to 
 do exactly this: which would also work well for the Interface approach noted 
 above. The end result would be the same in both cases ~ a secondary 
 "declaration only" file would be produced ~ which would be used by a client 
 as a means to get the appropriate code linked. In all cases, the client is 
 expected to know nothing of, nor care about, the implementation under the 
 hood.
 
 This is where a misunderstanding arose with -H which, as it turns out, has 
 no real value with respect to the above. Thus, D does not have a tool to 
 help produce a client-binding from an implementation ~ you /still/ have to 
 do that manually. That doesn't mean it can't be done, but it can certainly 
 be a chore, and it would be good to get some standardization in the 
 mechanics of it (such as a standard file-extension, for example, such that 
 the conflict over module names can be better managed).
 
 So ~
 
 1) D still needs a much more solid *story* for closed-source development. 
 Without it, one could argue it deserves to remain a niche language. Note 
 that this is not a technical hurdle, but it's one that has repeatedly failed 
 to be passed. All it needs is a little TLC and attention.
 
 2) The -H flag is apparently for something else entirely.
 
 
 Hope that helps?
 
 - Kris
 
 


Well said, Kris. Much better than I could put into words.  Thanks!

-JJR
Dec 30 2005
prev sibling next sibling parent reply "Dave" <Dave_member pathlink.com> writes:
"Kris" <fu bar.com> wrote in message news:dp3sve$2hm8$1 digitaldaemon.com...

 Just to clarify a couple of minor things:

 As John noted, implementation hiding is not about encryption ~ it's about
 supporting closed-source development.

 D /does/ support implementation hiding, and has done for a long time.
 What's been missing is a nice, simple, automated, standard, and
 compiler-supported way to generate a bridge between the "hidden"
 implementation, and client usage thereof. It's always been done manually,
 and it's easy to get confused over which file is which since there's
 common module names involved etc.

 One way to approach information hiding is to use Interfaces in conjunction
 with a Factory pattern. There's been a couple of trivial examples posted
 in this thread, but the general idea is that Interfaces are used to
 abstract away all implementation details, and one or more public functions
 are exposed in a "bridging module" to create instances on a clients
 behalf.

 Another approach to implementation hiding is to simply strip out existing
 method bodies and leave the bare declarations, along with any enums and/or
 typedefs they may use. Many requests have been made for an automated tool
 to do exactly this: which would also work well for the Interface approach
 noted above. The end result would be the same in both cases ~ a secondary


This is basically what -H does, except that inlinable function bodies are 
emitted. As pointed out earlier, private members often need to be there
too, with or without function bodies (offsets, const initializers, 
templates), otherwise it gets pretty complicated, fast.

I experimented with most/all the implementation hiding stuff and got most of 
it working, I think, but in the end after testing it would have taken a 
whole heck of a lot more work (and time maintaining it) to make and keep the 
thing bulletproof (see: 
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).

Why don't you give -H a try by using it against phobos, DWT and Mango (once 
the integration issues are taken care of)? Then see if it still really makes 
the implementation details all that apparent (or at least more apparent than 
C/++ library header files do now)?

- Dave


 "declaration only" file would be produced ~ which would be used by a
 client as a means to get the appropriate code linked. In all cases, the
 client is expected to know nothing of, nor care about, the implementation
 under the hood.

 This is where a misunderstanding arose with -H which, as it turns out, has
 no real value with respect to the above. Thus, D does not have a tool to
 help produce a client-binding from an implementation ~ you /still/ have to
 do that manually. That doesn't mean it can't be done, but it can certainly
 be a chore, and it would be good to get some standardization in the
 mechanics of it (such as a standard file-extension, for example, such that
 the conflict over module names can be better managed).

 So ~

 1) D still needs a much more solid *story* for closed-source development.
 Without it, one could argue it deserves to remain a niche language. Note
 that this is not a technical hurdle, but it's one that has repeatedly
 failed to be passed. All it needs is a little TLC and attention.

 2) The -H flag is apparently for something else entirely.


 Hope that helps?

 - Kris



 "John Reimer" <terminal.node gmail.com> wrote

 Kyle Furlong wrote:

 John Reimer wrote:



 Okay, now I see why there is so much confusion going on with the
 addition of this feature.  There are two problems to be solved: one is
 the issue of implementation hiding; the other is the issue of compiler
 performance.  I think most of us mistook the di file generator to be a
 solution for implementation hiding for closed source projects.
 Obviously we got the wrong idea about the whole thing.

 If di files serve the same purpose as "precompiled headers," obviously
 there is no consideration whatsoever for implementation hiding (and
 there never was any such intention).

 Good to know.  Personally, I was less interested in compile performance
 and more interested in implementation hiding.  The -H flag, therefore,
 will not be of much use to me.

 -JJR


 Right on John. Walter, what is your answer for closed source
 implementation hiding?


 I think Walter already answered this question in a previous post in this
 thread.  If I understand correctly, he basically detailed how
 implementation hiding is nigh to impossible to do effectively.  I'm not
 sure if he meant that it's not worth trying to program a solution because
 data can't be effectively hidden or that the difficultly is not worth the
 effort.

 I just want to clarify that my idea of implementation-hiding really
 didn't entail encrypted libraries.  I know somebody could find out
 implementation details if they really wanted to.  The idea, much like the
 Modula 2 and 3 interface definition files, is merely to separate the
 details: (1) so that it isn't /easy/ to see implementation details and
 (2) so that we have a summary of the features a library provides.  A
 nifty side effect is that there's probably increase compiler performance.

 In the end though, maybe something like DDL (www.dsource.org) will solve
 all our problems in some way.  Eliminating the need for the compiler to
 parse a header altogether may be a big step forward.

 Also, while I feel that the -H option seems to be more of distraction
 from fixing higher priority items in D, it may do well for us at some
 point; I'm just hoping, though, that all the bugs it introduces don't
 take up Walter's time.

 A tool like this could theoretically be separate from the compiler. That
 may be the best solution in the end if it's too much work to maintain. I
 vaguely remember arguments to this end being made months to a year or so
 before.

 -JJR
Dec 30 2005
parent reply "Kris" <fu bar.com> writes:
Perhaps there's further misunderstanding here:

"Dave" <Dave_member pathlink.com> wrote in

 "Kris" <fu bar.com> wrote ...

 Another approach to implementation hiding is to simply strip out existing
 method bodies and leave the bare declarations, along with any enums 
 and/or
 typedefs they may use. Many requests have been made for an automated tool
 to do exactly this: which would also work well for the Interface approach
 noted above. The end result would be the same in both cases ~ a secondary


 This is basically what -H does, except that inlinable function bodies are 
 emitted. As pointed out earlier, private members often need to be there
 too, with or without function bodies (offsets, const initializers, 
 templates), otherwise it gets pretty complicated, fast.



And because of that, it really has no value for implementation hiding. I'd 
go as far as to say it simply complicates things instead. Do imports get 
exposed also? I'd try this myself, but the compiler GPFs. Further, exposing 
public attributes in this manner seems to defy logic, and appears to breaks 
every visibility rule in the book. Such declarations were made private 
because they were intended to be, /ahem/, private?



 I experimented with most/all the implementation hiding stuff and got most 
 of it working, I think, but in the end after testing it would have taken a 
 whole heck of a lot more work (and time maintaining it) to make and keep 
 the thing bulletproof (see: 
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).

 Why don't you give -H a try by using it against phobos, DWT and Mango 
 (once the integration issues are taken care of)? Then see if it still 
 really makes the implementation details all that apparent (or at least 
 more apparent than C/++ library header files do now)?



Forgive me Dave, but I just can't see how that's relevant. What you describe 
is not what, for example, any Fortune-500 company would consider usable. 
Walter apparently agrees. In reference to software contract-exposure: ".di 
files aren't necessarilly going to help you here."

Then there's the question of what this feature thinks it addresses. Walter 
seems to feel it's about compiler performance: "di files' main advantage is 
improving build speeds for large projects", and this is borne out in the 
flurry of posts. You also intimated this in a recent post. Yet you're also 
saying it can be used for implementation-hiding? Pardon me, but I have to 
disagree on all points:

<rant>

1) -H has little or no benefit for implementation hiding. It may look as 
though it could suffice, but I'll be happy to bet the current incarnation is 
a questionable toy in this regard. If I could actually get it to work, then 
I'd be happy to prove the point. Perhaps surprisingly, Walter appears to 
agree.

2) -H certainly ought to cut down on import size. So, just how much does 
that affect the compiler performance? It's already blazingly fast. I'll be a 
monkey's uncle if -H makes true /real-world/ differences to more than 5% of 
users. On the other hand, a /binary/ symbolic representation (as discussed 
several times) could make a really notable dent for certain environments, if 
such speedups were to become a necessity. If -H is all about performance, 
then why isn't it based around such a representation?

3) You speculatively mentioned DWT and RAD IDEs. There's a few things to 
consider there: (a) if the API is stable, the headers can be produced by a 
standalone tool ~ it doesn't need to be part of the compiler; (b) it the 
headers in question are being revised, they need to be regenerated on the 
fly; and (c) a RAD tool would likely prefer to have a library made available 
that hides all the details of creating and managing such headers, and would 
likely prefer that happened at the binary-symbolic level, explicitly for 
reasons of performance. Especially in case (a). Partial source-code munging 
does little to help such a tool ~ it's just not going far enough.

I've no intent on causing discomfort to you or anyone here, but I really 
have to say this feature appears to have been given not /nearly/ sufficient 
thought. It fails to support industrial-level implementation-hiding (the one 
place it is needed) and it doesn't push the performance envelope very hard 
at all.

You can call me an ass if you like, Dave, but this is surely feature-creep 
of the worst kind ~ there's apparently no meat. Can you, or anyone, please 
help me grasp where the real value lies? It's not that I'm somehow against 
new functionality. Part of what bugs me about this is the reference compiler 
sets an unwritten "standard" that others must somehow comply with. Whatever 
those standards are, they surely ought to be well considered.

</rant>

I can point to one side-effect that actually does help: the compiler now 
looks for "di" files, which means that hand-coded "implementation bridging" 
files can (at last!) live side-by-side with the real implementation module, 
eliminating some prior grief for a developer. However, this conflicts 
with -H rather badly, and therefore is likely to be an unintended and 
somewhat precarious benefit. Was that aspect considered?

Respectfully;

- Kris
Dec 30 2005
next sibling parent John Reimer <terminal.node gmail.com> writes:
Kris wrote:

 Perhaps there's further misunderstanding here:



 I can point to one side-effect that actually does help: the compiler now 
 looks for "di" files, which means that hand-coded "implementation bridging" 
 files can (at last!) live side-by-side with the real implementation module, 
 eliminating some prior grief for a developer. However, this conflicts 
 with -H rather badly, and therefore is likely to be an unintended and 
 somewhat precarious benefit. Was that aspect considered?
 



Interesting!  I didn't really think of that aspect.  That could be 
useful, indeed. Though, as you said, it probably is precarious.

-JJR
Dec 30 2005
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
Kris wrote:

 
 And because of that, it really has no value for implementation hiding. I'd 
 go as far as to say it simply complicates things instead. Do imports get 
 exposed also? I'd try this myself, but the compiler GPFs. Further, exposing 
 public attributes in this manner seems to defy logic, and appears to breaks 
 every visibility rule in the book. Such declarations were made private 
 because they were intended to be, /ahem/, private?


But, as Walter stated, private data might affect alignment and such so 
it must be exposed.  If implementation hiding is an issue and if 
interfaces are not appropriate, the pimpl idiom from C++ might be worth 
considering.


 Forgive me Dave, but I just can't see how that's relevant. What you describe 
 is not what, for example, any Fortune-500 company would consider usable. 
 Walter apparently agrees. In reference to software contract-exposure: ".di 
 files aren't necessarilly going to help you here."
 
 Then there's the question of what this feature thinks it addresses. Walter 
 seems to feel it's about compiler performance: "di files' main advantage is 
 improving build speeds for large projects", and this is borne out in the 
 flurry of posts. You also intimated this in a recent post. Yet you're also 
 saying it can be used for implementation-hiding?


I think the most relevant comparison is with C/C++ header files, as D 
headers are nearly identical in what they provide.  And as Walter said, 
the primary use of headers is to speed compilation.  That implementation 
can be concealed in some cases is secondary.  Another concern is 
template files.  C++ has 'export' in an attempt to address this, but 
it's a tremendously complex feature that provides questionable benefit. 
  I think Walter is right for not attempting anything like this for D 
quite yet.


 Pardon me, but I have to disagree on all points:
 
 <rant>
 
 1) -H has little or no benefit for implementation hiding. It may look as 
 though it could suffice, but I'll be happy to bet the current incarnation is 
 a questionable toy in this regard. If I could actually get it to work, then 
 I'd be happy to prove the point. Perhaps surprisingly, Walter appears to 
 agree.


See my comment about the pimpl (pointer to implementation) idea above. 
This is commonly used both to obscure implementation details and to 
reduce coupling.

 
 2) -H certainly ought to cut down on import size. So, just how much does 
 that affect the compiler performance? It's already blazingly fast. I'll be a 
 monkey's uncle if -H makes true /real-world/ differences to more than 5% of 
 users. On the other hand, a /binary/ symbolic representation (as discussed 
 several times) could make a really notable dent for certain environments, if 
 such speedups were to become a necessity. If -H is all about performance, 
 then why isn't it based around such a representation?


Another issue is that header files are touched far less frequently than 
implementation files.  Thus, implementation changes can be made and so 
long as the header remains unchanged the compiler doesn't have to 
rebuild dependent modules.  In large projects, this can be of tremendous 
value.  To this end, it might be nice if DMD only touched header files 
for nonzero deltas (so timestamps only changed if the file contents 
changed).


 3) You speculatively mentioned DWT and RAD IDEs. There's a few things to 
 consider there: (a) if the API is stable, the headers can be produced by a 
 standalone tool ~ it doesn't need to be part of the compiler; (b) it the 
 headers in question are being revised, they need to be regenerated on the 
 fly; and (c) a RAD tool would likely prefer to have a library made available 
 that hides all the details of creating and managing such headers, and would 
 likely prefer that happened at the binary-symbolic level, explicitly for 
 reasons of performance. Especially in case (a). Partial source-code munging 
 does little to help such a tool ~ it's just not going far enough.


True enough.  But since a front-end is really required to do the job, 
why not build the feature into the compiler?  Particularly since each 
compiler may have different metrics for inlining and such.


 I've no intent on causing discomfort to you or anyone here, but I really 
 have to say this feature appears to have been given not /nearly/ sufficient 
 thought. It fails to support industrial-level implementation-hiding (the one 
 place it is needed) and it doesn't push the performance envelope very hard 
 at all.


I see implementation hiding as a separate (though related) QOI issue. 
Typically, I want to see readable header files and let things get 
mangled in post-production before the CD is sent out to clients.


 You can call me an ass if you like, Dave, but this is surely feature-creep 
 of the worst kind ~ there's apparently no meat. Can you, or anyone, please 
 help me grasp where the real value lies? It's not that I'm somehow against 
 new functionality. Part of what bugs me about this is the reference compiler 
 sets an unwritten "standard" that others must somehow comply with. Whatever 
 those standards are, they surely ought to be well considered.


I think it's important to separate QOI features from "standard" 
features.  Will it be necessary for all D compilers to have code 
coverage or profiling tools?  Of course not.  Header generation is in 
the same category.


 I can point to one side-effect that actually does help: the compiler now 
 looks for "di" files, which means that hand-coded "implementation bridging" 
 files can (at last!) live side-by-side with the real implementation module, 
 eliminating some prior grief for a developer. However, this conflicts 
 with -H rather badly, and therefore is likely to be an unintended and 
 somewhat precarious benefit. Was that aspect considered?


I haven't installed this new version of DMD yet so I don't know much 
about how -H works yet.  What I said above is based on what I've read 
here today :-)


Sean
Dec 30 2005
next sibling parent reply "Kris" <fu bar.com> writes:
First, I must point out that this branch is about "implementation hiding" ~ 
it say so in the topic :)
Second, if there's a point in this reply, it's at the end.


"Sean Kelly" <sean f4.ca> wrote

 But, as Walter stated, private data might affect alignment and such so it 
 must be exposed.  If implementation hiding is an issue and if interfaces 
 are not appropriate, the pimpl idiom from C++ might be worth considering.


Interfaces are in the same boat, Sean. D still needs something nice and easy 
to generate the "implementation bridge" module that folk have been asking 
after for a long time. You can always build it by hand, but there's always 
been problems of one kind or another. Thankfully, a side-effect of -H will 
help. I'm not familiar with pimpl.




 I think the most relevant comparison is with C/C++ header files, as D 
 headers are nearly identical in what they provide.  And as Walter said, 
 the primary use of headers is to speed compilation.  That implementation 
 can be concealed in some cases is secondary.  Another concern is template 
 files.  C++ has 'export' in an attempt to address this, but it's a 
 tremendously complex feature that provides questionable benefit. I think 
 Walter is right for not attempting anything like this for D quite yet.



Then let's call the darned thing a "performance enhancement" and stop all 
this continued pretense :)




 Another issue is that header files are touched far less frequently than 
 implementation files.  Thus, implementation changes can be made and so 
 long as the header remains unchanged the compiler doesn't have to rebuild 
 dependent modules.  In large projects, this can be of tremendous value. 
 To this end, it might be nice if DMD only touched header files for nonzero 
 deltas (so timestamps only changed if the file contents changed).



In C, yes. D currently does not have any such issues.



 I see implementation hiding as a separate (though related) QOI issue. 
 Typically, I want to see readable header files and let things get mangled 
 in post-production before the CD is sent out to clients.


Ack ~ this is not about mangled headers!



 You can call me an ass if you like, Dave, but this is surely 
 feature-creep of the worst kind ~ there's apparently no meat. Can you, or 
 anyone, please help me grasp where the real value lies? It's not that I'm 
 somehow against new functionality. Part of what bugs me about this is the 
 reference compiler sets an unwritten "standard" that others must somehow 
 comply with. Whatever those standards are, they surely ought to be well 
 considered.


 I think it's important to separate QOI features from "standard" features. 
 Will it be necessary for all D compilers to have code coverage or 
 profiling tools?  Of course not.  Header generation is in the same 
 category.



Is it? If, just if, people start using -H in an attempt at 
implementation-hiding, then other compilers will have to follow suit. After 
all, code will be written to take advantage of those headers (the whole 
point of implementation-hiding), and thus will be dependent on the way the 
information is exposed. Thus, once we start down that road others will have 
to comply.
Dec 30 2005
parent reply Sean Kelly <sean f4.ca> writes:
Kris wrote:

 First, I must point out that this branch is about "implementation hiding" ~ 
 it say so in the topic :)


Sorry :-)  I suppose I read too much into some of the text I omitted.


 Another issue is that header files are touched far less frequently than 
 implementation files.  Thus, implementation changes can be made and so 
 long as the header remains unchanged the compiler doesn't have to rebuild 
 dependent modules.  In large projects, this can be of tremendous value. 
 To this end, it might be nice if DMD only touched header files for nonzero 
 deltas (so timestamps only changed if the file contents changed).


 
 In C, yes. D currently does not have any such issues.


How so?  I'll admit I haven't looked into how the need for recompilation 
is detected.


 I see implementation hiding as a separate (though related) QOI issue. 
 Typically, I want to see readable header files and let things get mangled 
 in post-production before the CD is sent out to clients.


 
 Ack ~ this is not about mangled headers!


I had a few things going through my head and unfortunately only one of 
them made it out :-)  I was thinking of implementation hiding in .NET 
where mangling is pretty much necessary.  So what would be fitting then? 
  The ability to generate pure headers with no inlined function calls 
and no private data?


 I think it's important to separate QOI features from "standard" features. 
 Will it be necessary for all D compilers to have code coverage or 
 profiling tools?  Of course not.  Header generation is in the same 
 category.


 
 Is it? If, just if, people start using -H in an attempt at 
 implementation-hiding, then other compilers will have to follow suit. After 
 all, code will be written to take advantage of those headers (the whole 
 point of implementation-hiding), and thus will be dependent on the way the 
 information is exposed. Thus, once we start down that road others will have 
 to comply.


I'm not sure I understand.  The -H option must necessarily generate 
headers that contain much of the same symbolic information as the 
original module or the header will not be usable.  Hrm... perhaps 
instead of containing private data for alignment purposes, a pragma 
could be used to indicate data size?  ie.

class C {
private:
     int x;
     int y;
}

// translates to

class C {
pragma( pad, 8 );
}
Dec 30 2005
parent "Kris" <fu bar.com> writes:
"Sean Kelly" <sean f4.ca> wrote...

 Sorry :-)  I suppose I read too much into some of the text I omitted.


Mea cupla :)



 Another issue is that header files are touched far less frequently than 
 implementation files.  Thus, implementation changes can be made and so 
 long as the header remains unchanged the compiler doesn't have to 
 rebuild dependent modules.  In large projects, this can be of tremendous 
 value. To this end, it might be nice if DMD only touched header files 
 for nonzero deltas (so timestamps only changed if the file contents 
 changed).


 In C, yes. D currently does not have any such issues.


 How so?  I'll admit I haven't looked into how the need for recompilation 
 is detected.



I'm sorry. I meant that D doesn't have those performance issues. Introducing 
D header files does, of course, introduce the synchronization problem you 
note. But this performance aspect is a different one from implementation 
hiding.



 Ack ~ this is not about mangled headers!


 I had a few things going through my head and unfortunately only one of 
 them made it out :-)  I was thinking of implementation hiding in .NET 
 where mangling is pretty much necessary.  So what would be fitting then? 
 The ability to generate pure headers with no inlined function calls and no 
 private data?



That's effectively what so many requests have been about. Imported modules 
must be considered also. In a closed-source, proprietary environments, one 
shouldn't be exposing various bits of implementation or private attributes; 
certainly when there's no legitimate reason to do so.



 I think it's important to separate QOI features from "standard" 
 features. Will it be necessary for all D compilers to have code coverage 
 or profiling tools?  Of course not.  Header generation is in the same 
 category.


 Is it? If, just if, people start using -H in an attempt at 
 implementation-hiding, then other compilers will have to follow suit. 
 After all, code will be written to take advantage of those headers (the 
 whole point of implementation-hiding), and thus will be dependent on the 
 way the information is exposed. Thus, once we start down that road others 
 will have to comply.


 I'm not sure I understand.  The -H option must necessarily generate 
 headers that contain much of the same symbolic information as the original 
 module or the header will not be usable.  Hrm... perhaps instead of 
 containing private data for alignment purposes, a pragma could be used to 
 indicate data size?  ie.

 class C {
 private:
     int x;
     int y;
 }

 // translates to

 class C {
 pragma( pad, 8 );
 }



Pardon me if I copy a bit from another post:

"I have to say, though, that's there's a fundamental flaw exposed by trying 
to please two masters: you can't do implementation-hiding correctly whilst 
exposing partial-implementation and private attributes. And you can't 
provide for faster-compilation without doing so."

That's a generalization, and refers to inline-code concerns, templates, and 
so on.

To go one further: If D cannot, technically, expose a module or a class for 
external usage without exposing various proprietary private attributes and 
proprietary snippets of code (which in turn may reference proprietary yet 
private imports), then there's something very far wrong with the entire 
concept of D. This would cause DDL a blow, and put into question the 
legitimacy of using D for commercial purpose. I have, and continue to assume 
this is not the case.

We should refer back to Derek's example, but I'll paste it here instead 
(hope Derek doesn't mind):

=========================

This source file ...

//--------------------------------
import std.stdio;

private int qwerty;
class A
{
     private int x;
     void ths()
     {
       x = 1;
       }
}

void xpub()
{
     A a = new A;
     qwerty = 1;
}

private void xpriv()
{
     A b = new A;
     qwerty = 2;
}
//--------------------------------


was turned into this 'header' ...

// D import file generated from 'test.d'
import std.stdio;
private
{
     int qwerty;
}
class A
{
     private
{
     int x;
}
     void ths()
{
x = 1;
}
}
void xpub()
{
A a = new A;
qwerty = 1;
}
private
{
     void xpriv()
{
A b = new A;
qwerty = 2;
}
}
//-------------------------

but I was expecting something more like this ...

// D import file generated by hand
import std.stdio;

class A
{
     void ths(){}
}
void xpub(){}

===================


The latter is what we've been asking for (for a long time). Note that Derek 
also included a (non-private) import in the hand-built version? Private 
imports should not be exposed.
Dec 30 2005
prev sibling parent "Walter Bright" <newshound digitalmars.com> writes:
"Sean Kelly" <sean f4.ca> wrote in message 
news:dp4r60$8e5$1 digitaldaemon.com...

 I think it's important to separate QOI features from "standard" features. 
 Will it be necessary for all D compilers to have code coverage or 
 profiling tools?  Of course not.  Header generation is in the same 
 category.


That's right. I tried to make that distiction explicit in the doc. -H is not 
a D language feature, it's a DMD implementation feature (and one GDC will 
likely pick up as well).
Dec 30 2005
prev sibling next sibling parent reply JT <JT_member pathlink.com> writes:
In article <dp4oq5$6qt$1 digitaldaemon.com>, Kris says...

Perhaps there's further misunderstanding here:

"Dave" <Dave_member pathlink.com> wrote in

 "Kris" <fu bar.com> wrote ...

 Another approach to implementation hiding is to simply strip out existing
 method bodies and leave the bare declarations, along with any enums 
 and/or
 typedefs they may use. Many requests have been made for an automated tool
 to do exactly this: which would also work well for the Interface approach
 noted above. The end result would be the same in both cases ~ a secondary


 This is basically what -H does, except that inlinable function bodies are 
 emitted. As pointed out earlier, private members often need to be there
 too, with or without function bodies (offsets, const initializers, 
 templates), otherwise it gets pretty complicated, fast.



And because of that, it really has no value for implementation hiding. I'd 
go as far as to say it simply complicates things instead. Do imports get 
exposed also? I'd try this myself, but the compiler GPFs. Further, exposing 
public attributes in this manner seems to defy logic, and appears to breaks 
every visibility rule in the book. Such declarations were made private 
because they were intended to be, /ahem/, private?



 I experimented with most/all the implementation hiding stuff and got most 
 of it working, I think, but in the end after testing it would have taken a 
 whole heck of a lot more work (and time maintaining it) to make and keep 
 the thing bulletproof (see: 
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).

 Why don't you give -H a try by using it against phobos, DWT and Mango 
 (once the integration issues are taken care of)? Then see if it still 
 really makes the implementation details all that apparent (or at least 
 more apparent than C/++ library header files do now)?



Forgive me Dave, but I just can't see how that's relevant. What you describe 
is not what, for example, any Fortune-500 company would consider usable. 
Walter apparently agrees. In reference to software contract-exposure: ".di 
files aren't necessarilly going to help you here."

Then there's the question of what this feature thinks it addresses. Walter 
seems to feel it's about compiler performance: "di files' main advantage is 
improving build speeds for large projects", and this is borne out in the 
flurry of posts. You also intimated this in a recent post. Yet you're also 
saying it can be used for implementation-hiding? Pardon me, but I have to 
disagree on all points:

<rant>

1) -H has little or no benefit for implementation hiding. It may look as 
though it could suffice, but I'll be happy to bet the current incarnation is 
a questionable toy in this regard. If I could actually get it to work, then 
I'd be happy to prove the point. Perhaps surprisingly, Walter appears to 
agree.

2) -H certainly ought to cut down on import size. So, just how much does 
that affect the compiler performance? It's already blazingly fast. I'll be a 
monkey's uncle if -H makes true /real-world/ differences to more than 5% of 
users. On the other hand, a /binary/ symbolic representation (as discussed 
several times) could make a really notable dent for certain environments, if 
such speedups were to become a necessity. If -H is all about performance, 
then why isn't it based around such a representation?

3) You speculatively mentioned DWT and RAD IDEs. There's a few things to 
consider there: (a) if the API is stable, the headers can be produced by a 
standalone tool ~ it doesn't need to be part of the compiler; (b) it the 
headers in question are being revised, they need to be regenerated on the 
fly; and (c) a RAD tool would likely prefer to have a library made available 
that hides all the details of creating and managing such headers, and would 
likely prefer that happened at the binary-symbolic level, explicitly for 
reasons of performance. Especially in case (a). Partial source-code munging 
does little to help such a tool ~ it's just not going far enough.

I've no intent on causing discomfort to you or anyone here, but I really 
have to say this feature appears to have been given not /nearly/ sufficient 
thought. It fails to support industrial-level implementation-hiding (the one 
place it is needed) and it doesn't push the performance envelope very hard 
at all.

You can call me an ass if you like, Dave, but this is surely feature-creep 
of the worst kind ~ there's apparently no meat. Can you, or anyone, please 
help me grasp where the real value lies? It's not that I'm somehow against 
new functionality. Part of what bugs me about this is the reference compiler 
sets an unwritten "standard" that others must somehow comply with. Whatever 
those standards are, they surely ought to be well considered.

</rant>

I can point to one side-effect that actually does help: the compiler now 
looks for "di" files, which means that hand-coded "implementation bridging" 
files can (at last!) live side-by-side with the real implementation module, 
eliminating some prior grief for a developer. However, this conflicts 
with -H rather badly, and therefore is likely to be an unintended and 
somewhat precarious benefit. Was that aspect considered?

Respectfully;

- Kris 


I think part of the confusion is lack of knowledge of how the DMD compiler
works. When one compiles a DMD module - it parses and performs full analysis on
all imported modules - and all modules they import etc. Theres a section of
comments refering to importing only pre-compiled symbols for imported objects
instead of just parsing the complete source. I believe this is some sort if
middle ground between importing symbols and parsing the source. And I can very
much see how it could be usefull in huge projects but I personally havent
experienced compile times that are not lighting fast. Also lets not forget what
walter mentioned about this feature fleshing out the code generation output of
the compiler which will be usefull down the road. I just dont understand why
people react with such negativity and disrespect towards walter if he implements
a feature that they dont need. Its really immature. Some of you act like you are
paying Walter or something. What ever.
Dec 30 2005
parent reply "Kris" <fu bar.com> writes:
"JT" <JT_member pathlink.com> wrote

 I think part of the confusion is lack of knowledge of how the DMD compiler
 works. When one compiles a DMD module - it parses and performs full 
 analysis on
 all imported modules - and all modules they import etc.


I think most folks understand this. Is it an issue?


 Theres a section of
 comments refering to importing only pre-compiled symbols for imported 
 objects
 instead of just parsing the complete source. I believe this is some sort 
 if
 middle ground between importing symbols and parsing the source.


The "middle ground" you speak of is exactly that. It's not needed at this 
point, so why burden the state of play with adding such a thing? It's called 
feature creep.



 And I can very
 much see how it could be usefull in huge projects but I personally havent
 experienced compile times that are not lighting fast.


There's no question that a binary-symbolic implementation would be great. 
But, as you point out, there's no pressing need for it right now. Each time 
I compile, it's with 200+ files on a mediocre machine. I just timed it, and 
it takes just over 2 seconds. That includes the time Build takes. The 
compiler is very, very, fast. I think that's indicitive of a good design and 
a lot of valuable experience. The same cannot be said for -H, since it 
doesn't even seem to be clear about what it addresses.


Also lets not forget what

 walter mentioned about this feature fleshing out the code generation 
 output of
 the compiler which will be usefull down the road.


Sure. But that's not what's being addressed. What it being addressed is the 
latest claim that this is somehow for implementation-hiding. It is simply 
not. Performance wise? Different subject matter.



 I just dont understand why
 people react with such negativity and disrespect towards walter if he 
 implements
 a feature that they dont need. Its really immature. Some of you act like 
 you are
 paying Walter or something. What ever.


First, I don't see where there was /any/ negativity or disrespect towards 
Walter in that post. It was adressed entirely towards Dave. But then, so 
what if it were ardently disagreeing with Walter directly? Walter is a tough 
nut to crack ... there's no value in mincing words with him.

You are, of course, welcome to your opinion. As am I.

My opinion is this: The community has a rather large stake in the future of 
D. This is why Walter has this NG ~ it's a way to create the evangelizers, 
and get the message out. D is a grass-root effort. There are a few handfuls 
of people who have contributed an /extraordinary/ amount of their /own/ 
time, in many different ways. Whether it is through ideas and suggestions, 
through encouraging other to try D, through hosting the NG, through hosting 
dsource.org, writing libraries or building tools. Do you think the community 
has no claim whatsoever to the D language? Do you feel we should not be able 
to shape its direction? Or criticize it? Chastise it?

Thus, we're not paying Walter to write the compiler. Nor is he paying those 
who contribute so much. It goes both ways, and it's pretty immature of you 
to state otherwise. I'm sure you'd agree.
Dec 30 2005
parent reply Sean Kelly <sean f4.ca> writes:
Kris wrote:

 "JT" <JT_member pathlink.com> wrote

 I think part of the confusion is lack of knowledge of how the DMD compiler
 works. When one compiles a DMD module - it parses and performs full 
 analysis on
 all imported modules - and all modules they import etc.


 
 I think most folks understand this. Is it an issue?


Ah, and this is what you meant about timestamping not being used in D. 
Works for me.


Sean
Dec 30 2005
parent reply Sean Kelly <sean f4.ca> writes:
Sean Kelly wrote:

 Kris wrote:

 "JT" <JT_member pathlink.com> wrote

 I think part of the confusion is lack of knowledge of how the DMD 
 compiler
 works. When one compiles a DMD module - it parses and performs full 
 analysis on
 all imported modules - and all modules they import etc.


 I think most folks understand this. Is it an issue?


 
 Ah, and this is what you meant about timestamping not being used in D. 
 Works for me.


I posted without thinking about this :-p.  While this may be how the 
compiler works (and approximately how any compiler works, so far as I'm 
aware), this is separate from the timestamping issue.


Sean
Dec 30 2005
parent reply JT <JT_member pathlink.com> writes:
In article <dp53dk$e44$1 digitaldaemon.com>, Sean Kelly says...

Sean Kelly wrote:

 Kris wrote:

 "JT" <JT_member pathlink.com> wrote

 I think part of the confusion is lack of knowledge of how the DMD 
 compiler
 works. When one compiles a DMD module - it parses and performs full 
 analysis on
 all imported modules - and all modules they import etc.


 I think most folks understand this. Is it an issue?


 
 Ah, and this is what you meant about timestamping not being used in D. 
 Works for me.


I posted without thinking about this :-p.  While this may be how the 
compiler works (and approximately how any compiler works, so far as I'm 
aware), this is separate from the timestamping issue.


Sean


well in C++ you only parse headers.... thats all this -H is doing. its providing
a way to compile large projects without having to parse every module for every
module etc. I still cant understand why people are upset because a feature
exists they dont need. Its downright comedy.
Dec 30 2005
next sibling parent reply "Kris" <fu bar.com> writes:
"JT" <JT_member pathlink.com> wrote

  I still cant understand why people are upset because a feature
 exists they dont need. Its downright comedy.


It would be, if it had no ramifications. It's clear that you're thinking 
about the claimed "performance enhancing" aspects alone, and not the real 
topic at all. If you really think that I personally fit into your little 
frame above, then you're sorely mistaken. Sure is nice that you're being 
entertained, though :)
Dec 30 2005
parent reply JT <JT_member pathlink.com> writes:
In article <dp574e$gie$1 digitaldaemon.com>, Kris says...

"JT" <JT_member pathlink.com> wrote

  I still cant understand why people are upset because a feature
 exists they dont need. Its downright comedy.


It would be, if it had no ramifications. It's clear that you're thinking 
about the claimed "performance enhancing" aspects alone, and not the real 
topic at all. If you really think that I personally fit into your little 
frame above, then you're sorely mistaken. Sure is nice that you're being 
entertained, though :) 


Well I agree with a lot of what you are saying but I think you clearly want a
*DIFFERENT* solution for implementation hiding than -H. Its like people are
saying "Walter, this apple tastes like an orange!". maybe thats because it is an
orange. :D  Maybe you need to use the front end and come up with your own
solution? just a suggestion....
Dec 30 2005
parent "Kris" <fu bar.com> writes:
Indeed. We should try to be clear that -H performs some kind of source-code 
munging for purposes other than this topic. Several folk (including myself) 
have tried to say, in a variety of different ways, "we should desist from 
associating it with implementation-hiding, since it is not currently 
targeted for that task".

As you say, it is currently "an orange" :)




"JT" <JT_member pathlink.com> wrote in message 
news:dp57m1$gs6$1 digitaldaemon.com...

 In article <dp574e$gie$1 digitaldaemon.com>, Kris says...

"JT" <JT_member pathlink.com> wrote

  I still cant understand why people are upset because a feature
 exists they dont need. Its downright comedy.


It would be, if it had no ramifications. It's clear that you're thinking
about the claimed "performance enhancing" aspects alone, and not the real
topic at all. If you really think that I personally fit into your little
frame above, then you're sorely mistaken. Sure is nice that you're being
entertained, though :)


 Well I agree with a lot of what you are saying but I think you clearly 
 want a
 *DIFFERENT* solution for implementation hiding than -H. Its like people 
 are
 saying "Walter, this apple tastes like an orange!". maybe thats because it 
 is an
 orange. :D  Maybe you need to use the front end and come up with your own
 solution? just a suggestion....
Dec 30 2005
prev sibling parent "Derek Parnell" <derek psych.ward> writes:
On Sat, 31 Dec 2005 16:44:18 +1100, JT <JT_member pathlink.com> wrote:


 I still cant understand why people are upset because a feature
 exists they dont need. Its downright comedy.


It's because of a misunderstanding. Myself and some others assumed its  
purpose was to support closed-source development, especially by the way it  
was presented by Walter, but it turns out that it's purpose is to speed up  
compilation when libraries are being used. I'm afraid its really just  
because we misunderstood Walter's (and Dave's) intent for the feature.  
Personally, I have no need for it as compilations speeds are so fast that  
even a 50% reduction would have zero effect on productivity.

-- 
Derek Parnell
Melbourne, Australia
Dec 31 2005
prev sibling next sibling parent reply Dave <Dave_member pathlink.com> writes:
In article <dp4oq5$6qt$1 digitaldaemon.com>, Kris says...

Perhaps there's further misunderstanding here:

"Dave" <Dave_member pathlink.com> wrote in

 "Kris" <fu bar.com> wrote ...

 Another approach to implementation hiding is to simply strip out existing
 method bodies and leave the bare declarations, along with any enums 
 and/or
 typedefs they may use. Many requests have been made for an automated tool
 to do exactly this: which would also work well for the Interface approach
 noted above. The end result would be the same in both cases ~ a secondary




I've no intent on causing discomfort to you or anyone here, but I really 
have to say this feature appears to have been given not /nearly/ sufficient 
thought. It fails to support industrial-level implementation-hiding (the one 
place it is needed) and it doesn't push the performance envelope very hard 
at all.


No discomfort at all - Everyone has their opinions. This obviously doesn't have
to be the be-all and end-all to the 'header', symbol file, what-have-you
functionality, and I don't think it was ever intended as such, certainly not by
me.

At the very least it brought out the discussion -- included in my earlier post
was a link to an older request for functionality and no-one replied, so I
implemented what seemed to make the most sense and what I had time for, at least
as a *first step*.

I would applaud anyone (Kris?, Derek?) who would take what's there and provide a
better solution -- plain-and-simple fact is I've spent all the time I can on it.

- Dave
Dec 30 2005
next sibling parent "Kris" <fu bar.com> writes:
"Dave" <Dave_member pathlink.com> wrote

 No discomfort at all - Everyone has their opinions.


I'm very glad you feel that way.



 This obviously doesn't have
 to be the be-all and end-all to the 'header', symbol file, what-have-you
 functionality, and I don't think it was ever intended as such, certainly 
 not by
 me.


Hey; it's tough to please anyone these days :)



 At the very least it brought out the discussion -- included in my earlier 
 post
 was a link to an older request for functionality and no-one replied, so I
 implemented what seemed to make the most sense and what I had time for, at 
 least
 as a *first step*.

 I would applaud anyone (Kris?, Derek?) who would take what's there and 
 provide a
 better solution -- plain-and-simple fact is I've spent all the time I can 
 on it.



It's certainly refreshing to hear such down-to-earth statements. 
Unfortunately, it's proven nigh impossible to get fixes into the darned 
library (phobos). It'll be harder still to get fixes/changes into the 
front-end. Perhaps you'll share your secret :)

I have to say, though, that's there's a fundamental flaw exposed by trying 
to please two masters: you can't do implementation-hiding correctly whilst 
exposing partial-implementation and private attributes. And you can't 
provide for faster-compilation without doing so. While I personally admire 
you tremendously for taking the initiative (no mean feat), I have to take 
serious issue with the imp-hiding aspect, since it has long-term 
ramifications.

Regards;
Dec 30 2005
prev sibling parent reply J C Calvarese <technocrat7 gmail.com> writes:
In article <dp50iu$bna$1 digitaldaemon.com>, Dave says...

In article <dp4oq5$6qt$1 digitaldaemon.com>, Kris says...

Perhaps there's further misunderstanding here:

"Dave" <Dave_member pathlink.com> wrote in






..


I've no intent on causing discomfort to you or anyone here, but I really 
have to say this feature appears to have been given not /nearly/ sufficient 
thought. It fails to support industrial-level implementation-hiding (the one 
place it is needed) and it doesn't push the performance envelope very hard 
at all.


No discomfort at all - Everyone has their opinions. This obviously doesn't have
to be the be-all and end-all to the 'header', symbol file, what-have-you
functionality, and I don't think it was ever intended as such, certainly not by
me.

At the very least it brought out the discussion -- included in my earlier post
was a link to an older request for functionality and no-one replied, so I
implemented what seemed to make the most sense and what I had time for, at
>least as a *first step*.

I would applaud anyone (Kris?, Derek?) who would take what's there and provide
>a better solution -- plain-and-simple fact is I've spent all the time I can on
>it.

- Dave


Dave, thanks for taking all of the negative criticism so well. (I'm sure I
wouldn't be able to maintain my calm in the face of so many surprisingly severe
posts.)

I think the main issue might be that one or more people are jealous that you not
only implemented something that you find useful, but also convinced Walter to
add it to the compiler. I once posted a bug report (including the simple fix)
that went unacknowleged for months. ;)

I don't see the real harm in the "-H" switch. Since I guess .di files are read
first, I can see that people will need to make sure that their .di files stay in
synch with their .d files, but that could be solved with a batch file that
negates any possible speed-up benefits are the "-H" switch. It's the same issue
that happens when I install lib's into \dmd\lib and need to remember to copy new
libs to reap the benefits from the changes I've tried.

On the other hand, this new switch will give us a new way to analyze D code
with. We could see what is a candidate for inlining and what isn't. That seems
like it could be pretty useful to me.

I just hope that if people feel so strongly about the need to have
implementation-hiding in D that they'll spend a little more time trying to hack
up the front-end to reach their goals (and a little less time ranting about it
in the newsgroup).

jcc7



 Jan 01 2006



  parent reply Dave <Dave_member pathlink.com>  writes:


In article <dp9bci$jcn$1 digitaldaemon.com>, J C Calvarese says...

I think the main issue might be that one or more people are jealous that you not
only implemented something that you find useful, but also convinced Walter to
add it to the compiler. I once posted a bug report (including the simple fix)
that went unacknowleged for months. ;)


I don't think I convinced Walter of anything here <g> More of "coinciding
opinion" probably. I think he saw a need for something offered as a contribution
already implemented and tested, so he could spend the time on other more
important stuff.

Hey, it was enjoyable and instructive for me <g>


On the other hand, this new switch will give us a new way to analyze D code
with. We could see what is a candidate for inlining and what isn't. That seems
like it could be pretty useful to me.


Hopefully it will eventually be seen as more useful than just inline feedback
(although that part is kind of cool too) <g>

As mentioned elsewhere, it cuts the code for the compiler to process by 80% for
a library like DWT. I agree with Walter that for large projects and such the
value of this should become more apparent as D is used in those environments
more.

Other people have mentioned binary representation, symbol files, etc... IMHO,
the problem there is that you'll need a tool to read (and debug) those and/or
the 'symbol' representation will be implemented differently between compilers.
Unless, of course, you first specify an "intermediate language" which really
wouldn't hide anything anyway. It's taken 5+ years to specify D already - how
long for the IL ;)

At least this is legal and readable D code, is automatically generated, and
really should not give away any more implementation details than comparable C/++
header files do now.


I just hope that if people feel so strongly about the need to have
implementation-hiding in D that they'll spend a little more time trying to hack
up the front-end to reach their goals (and a little less time ranting about it
in the newsgroup).


I'm not saying this to discourage anyone from the effort, but AFAIK most/all of
the things mentioned in this thread, have been tried and abandoned for various
reasons (see: http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).
It was a circuitous route. <g>

It'd be nice if someone out there can come up with something better since
obviously it is very important to some people and probably somewhat important to
many others.


jcc7





 Jan 01 2006



 next sibling parent  J C Calvarese <technocrat7 gmail.com>  writes:


In article <dpa8f8$195s$1 digitaldaemon.com>, Dave says...

.snip...


Hopefully it will eventually be seen as more useful than just inline feedback
(although that part is kind of cool too) <g>


I wasn't try to put it down. I get that it's cool, but I think most of its
benefits are somewhat over my head. ;)

..snip...


I just hope that if people feel so strongly about the need to have
implementation-hiding in D that they'll spend a little more time trying to
>>hack up the front-end to reach their goals (and a little less time ranting
>>about it in the newsgroup).


I'm not saying this to discourage anyone from the effort, but AFAIK most/all of
the things mentioned in this thread, have been tried and abandoned for various
reasons (see: http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).
It was a circuitous route. <g>


I've read that post a few times now, but I don't know if it's sunk in yet. It's
not that don't believe you. The thing is I don't really need implementation
hiding for my own purposes, so I'm not trying to get in the middle of this
argument. 

From reading the posts on this topics, I think that more implementation can be
hidden. It's more a question of "Do the people that want it want it enough to
get it done?" No, not everything should or could be hidden. Templates would have
to be visible. If someone doesn't care about inlining, couldn't he drop a few
function bodies that would be inlineable otherwise? And hiding the
implementation wouldn't prevent reverse engineering (though reverse engineering
might be more difficult). Or maybe I'm wrong about this, and anything else is
infeasible. Sometimes, I just have to convince myself that something is
impossible before I can admit what's obvious to someone else.


It'd be nice if someone out there can come up with something better since
obviously it is very important to some people and probably somewhat important
>to many others.


Sure.

jcc7



 Jan 01 2006



prev sibling  parent  "Kris" <fu bar.com>  writes:


Dave ~ a couple of comments:

"Dave" <Dave_member pathlink.com> wrote

 Other people have mentioned binary representation, symbol files, etc... 
 IMHO,
 the problem there is that you'll need a tool to read (and debug) those 
 and/or
 the 'symbol' representation will be implemented differently between 
 compilers.
 Unless, of course, you first specify an "intermediate language" which 
 really
 wouldn't hide anything anyway. It's taken 5+ years to specify D already - 
 how
 long for the IL ;)


I fully agree with your assertion on the time-scale of things. While I feel 
that this aspect of the compiler is premature-optimization, that's just my 
opinion ... not that it makes a blind bit of difference anyway. As noted 
before, and fwiw, I truly commend you for taking the initiative and actually 
/doing something/!

Just for the record, I recall it was Walter who said he would be doing a 
binary-symbolic representation.



 At least this is legal and readable D code, is automatically generated, 
 and
 really should not give away any more implementation details than 
 comparable C/++
 header files do now.


Walter has now noted that he feels it's "impractical" to mechanically 
perform the kind of implementation-hiding we'd been asking for, and thus we 
have to still do that manually. That's cool, but sometimes it's like 
pulling-teeth just trying to get some clarity around here. I'd like to refer 
back to the original posting on this subject-line (branch), since I think 
the perspective is relevant there? This branch is not about the 
"optimization" aspect, although it apparantly got mixed in.



 I'm not saying this to discourage anyone from the effort, but AFAIK 
 most/all of
 the things mentioned in this thread, have been tried and abandoned for 
 various
 reasons (see: 
 http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).
 It was a circuitous route. <g>


I wish I'd seen that post on Nov 12th ~ would have immediately noted 
(regarding the last paragraph of the main thrust) that emitting source code 
for 'inlineable' would mean potentially exposing private import modules. 
That's not a "good thing"tm for the kind of implementation-hiding we'd been 
asking for (for a long time). It also seems from your post that you had more 
in mind than what's currently exposed.



 obviously it is very important to some people and probably somewhat 
 important to
 many others.


It can be acutely important from a commercial standpoint. Because most 
people are simply tinkering with the language, you won't see much in the way 
of opinion there. Nor much from the open-source community.

We should all understand that there's a lot of inertia against change in IT. 
To get a foot into the commercial segment, D needs as much help as it can 
possibly get in order to avoid being marginalized ~ even just keeping the 
darned lawyers out of the way. Implementation-hiding is one such aspect that 
helps. I can tell you that D is /not/ being considered where I work. Hence, 
I have a beef with certain weak attributes of the language and tools.

I hope this clarifies certain aspects further; and thank you for being 
open-minded on the subject.



 Jan 02 2006



prev sibling next sibling parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"Kris" <fu bar.com> wrote in message news:dp4oq5$6qt$1 digitaldaemon.com...

 Forgive me Dave, but I just can't see how that's relevant. What you 
 describe is not what, for example, any Fortune-500 company would consider 
 usable.


I disagree. Take a look at C++ header files: full class 'private' members, 
inlined function bodies, full template bodies, etc., all must be there. It's 
the same issue (which is what led to the demand for exported templates, 
which failed). Furthermore, Java is well accepted despite no practical 
hiding at all.

COM/interface/factory methodology in C++ does enable full hiding, however, 
and if you do that in D, you get full hiding as well.

Essentially, to get *practical* implementation hiding, you have to use some 
form of generic indirection. Successful implementation hiding schemes have 
all used this in one form or another: hooking interrupts in DOS, handles in 
the early Mac API, double indirection needed to access DLL methods, and COM 
programming. 



 Dec 30 2005



 next sibling parent  "Kris" <fu bar.com>  writes:


Absolutely agreed; but -H does not do this for the "indirection" case 
either. Nobody is asking for miracles, but the fact remains that Derek's 
example shows that -H is hopelessly lame at hiding the implementation. If I 
had a compiler that worked, I could show you exactly what I mean.

-H is no good for implementation hiding. We should stop trying to describe 
it as such when you yourself have said as much in prior posts.

Respectfully;

- Kris



"Walter Bright" <newshound digitalmars.com> wrote in message 
news:dp52vk$dqu$1 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp4oq5$6qt$1 digitaldaemon.com...

 Forgive me Dave, but I just can't see how that's relevant. What you 
 describe is not what, for example, any Fortune-500 company would consider 
 usable.


 I disagree. Take a look at C++ header files: full class 'private' members, 
 inlined function bodies, full template bodies, etc., all must be there. 
 It's the same issue (which is what led to the demand for exported 
 templates, which failed). Furthermore, Java is well accepted despite no 
 practical hiding at all.

 COM/interface/factory methodology in C++ does enable full hiding, however, 
 and if you do that in D, you get full hiding as well.

 Essentially, to get *practical* implementation hiding, you have to use 
 some form of generic indirection. Successful implementation hiding schemes 
 have all used this in one form or another: hooking interrupts in DOS, 
 handles in the early Mac API, double indirection needed to access DLL 
 methods, and COM programming.
 





 Dec 30 2005



prev sibling  parent reply "Kris" <fu bar.com>  writes:


"Walter Bright" <newshound digitalmars.com> wrote

 COM/interface/factory methodology in C++ does enable full hiding, however, 
 and if you do that in D, you get full hiding as well.



With respect to indirection: let's go back to the Factory example I posted 
earlier, which the compiler failed on:

~~~~~~~~~~~~~~~~~~~
SearchInterfaces.d
~~~~~~~~~~~~~~~~~~~

module SearchInterfaces;

interface ISearchEngine
{
        ISearchItem[] lookup (char[] terms);
}

interface ISearchItem
{
        int relevancy();
        char[] retrievalUrl();
        char[] displaySnippet();
}



~~~~~~~~~~~~~~~~~~~
SearchFactory.d
~~~~~~~~~~~~~~~~~~~

module SearchFactory;


public import SearchInterfaces;

// private import a.bunch.of.proprietary.stuff;


// a factory for hooking up a client to the hidden implementation
ISearchEngine createSearchEngine (char[] someAttributes)
{
        // potentially reference a.bunch.of.proprietary.stuff;
        return new XYX(someAttributes).
}


// do some proprietary setup
private static this()
{
        // potentially do a.bunch.of.proprietary.stuff;
}

~~~~~~~~~~~~~~~~~~~~~


That's the Factory implementation, which has a bunch of proprietary stuff in 
it along with references to proprietary imports. To correctly decouple a 
client from this implementation, one should provide another file (I'm 
covering Walter's statements here, from a long time ago) along the following 
lines:

~~~~~~~~~~~~~~~~~~~
SearchFactory.di
~~~~~~~~~~~~~~~~~~~

module SearchFactory;   // I believe this has to be the same name?

public import SearchInterfaces;

ISearchEngine createSearchEngine (char[] someAttributes);

~~~~~~~~~~~~~~~~~~~


This setup does the right thing from an implementation-hiding aspect. It's 
what's been asked for many times in the past. Note that only public 
decalarations have been exposed. There's no class in there at all, just to 
make it as simple as possible. Unfortunately, I can't show the -H version. 
However, it's not much of a stretch to map the above onto Derek's original 
example, and assert that (for example) the implementation for both the 
factory method, and the static ctor() would be exposed (along with any other 
method considered to be inline-able) That would be no good, Walter. If it 
were the case, we'd have to ship all those proprietary imports too, along 
with everything they referenced ~ ad infinitum.

I do wish I had a working compiler so this could be more concrete. 



 Dec 30 2005



 next sibling parent  Sean Kelly <sean f4.ca>  writes:


Kris wrote:

 
 This setup does the right thing from an implementation-hiding aspect. It's 
 what's been asked for many times in the past. Note that only public 
 decalarations have been exposed. There's no class in there at all, just to 
 make it as simple as possible. Unfortunately, I can't show the -H version. 
 However, it's not much of a stretch to map the above onto Derek's original 
 example, and assert that (for example) the implementation for both the 
 factory method, and the static ctor() would be exposed (along with any other 
 method considered to be inline-able) That would be no good, Walter. If it 
 were the case, we'd have to ship all those proprietary imports too, along 
 with everything they referenced ~ ad infinitum.


So why not provide a method to turn off inlined function generation? 
Since inlining isn't always desired, this seems a reasonable request.


Sean



 Dec 30 2005



prev sibling  parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"Kris" <fu bar.com> wrote in message news:dp56kj$gc8$1 digitaldaemon.com...

 This setup does the right thing from an implementation-hiding aspect. It's 
 what's been asked for many times in the past. Note that only public 
 decalarations have been exposed. There's no class in there at all, just to 
 make it as simple as possible. Unfortunately, I can't show the -H version. 
 However, it's not much of a stretch to map the above onto Derek's original 
 example, and assert that (for example) the implementation for both the 
 factory method, and the static ctor() would be exposed (along with any 
 other method considered to be inline-able) That would be no good, Walter. 
 If it were the case, we'd have to ship all those proprietary imports too, 
 along with everything they referenced ~ ad infinitum.

 I do wish I had a working compiler so this could be more concrete.


I know what you're talking about. But it just isn't practical. It isn't just 
turning off inlining - the types of the function parameters can reference 
private types or types in imported modules. So can parameter default values. 
So can const declarations. Templates too.

There are two solutions you can use. Both involve putting your interfaces in 
a separate module. One means making a module just for your factory stub:
    ISearchEngine createSearchEngine (char[] someAttributes);
Another is to use the COM approach of having a magic cookie (GUIDs serve as 
the magic cookie for COM) for each class, and pass that cookie to a generic 
factory function that is itself an interface member.



 Dec 31 2005



 next sibling parent  Sean Kelly <sean f4.ca>  writes:


Walter Bright wrote:

 "Kris" <fu bar.com> wrote in message news:dp56kj$gc8$1 digitaldaemon.com...

 This setup does the right thing from an implementation-hiding aspect. It's 
 what's been asked for many times in the past. Note that only public 
 decalarations have been exposed. There's no class in there at all, just to 
 make it as simple as possible. Unfortunately, I can't show the -H version. 
 However, it's not much of a stretch to map the above onto Derek's original 
 example, and assert that (for example) the implementation for both the 
 factory method, and the static ctor() would be exposed (along with any 
 other method considered to be inline-able) That would be no good, Walter. 
 If it were the case, we'd have to ship all those proprietary imports too, 
 along with everything they referenced ~ ad infinitum.

 I do wish I had a working compiler so this could be more concrete.


 
 I know what you're talking about. But it just isn't practical. It isn't just 
 turning off inlining - the types of the function parameters can reference 
 private types or types in imported modules. So can parameter default values. 
 So can const declarations. Templates too.


Ack!  I forgot about this.

By the way Kris, I found a good description of the pimpl idiom here:

http://www.gotw.ca/gotw/024.htm
http://www.gotw.ca/gotw/028.htm


 There are two solutions you can use. Both involve putting your interfaces in 
 a separate module. One means making a module just for your factory stub:
     ISearchEngine createSearchEngine (char[] someAttributes);
 Another is to use the COM approach of having a magic cookie (GUIDs serve as 
 the magic cookie for COM) for each class, and pass that cookie to a generic 
 factory function that is itself an interface member.


Aye.  See above :-)


Sean



 Dec 31 2005



prev sibling  parent reply "Kris" <fu bar.com>  writes:


Okay. I think there's an opportunity to clear this up via a few short 
questions. These refer back to the SearchEngine example posted previously 
(repeated below):

a) The example provided, where the SearchFactory.di file was hand-crafted? 
Will usage of that "di" module operate correctly?

b) You say below " it just isn't practical". Do you mean it is not 
technically feasible to mechanically extract the example SearchFactory.di 
from the given SearchFactory.d?


You'll hopefully note that example factory-module was explicitly written to 
avoid import references within all exposed types, has no templates, etc. 
Exactly the kind of thing a developer has to take responsibility over when 
decoupling usage from implementation (unless they explicitly wish to expose 
those also).

I have two more questions to follow up with, if you don't mind clearing this 
up?

- Kris


p.s. I'm reposting the example for the sake of clarity:

~~~~~~~~~~~~~~~~~~~
SearchInterfaces.d
~~~~~~~~~~~~~~~~~~~

module SearchInterfaces;

interface ISearchEngine
{
        ISearchItem[] lookup (char[] terms);
}

interface ISearchItem
{
        int relevancy();
        char[] retrievalUrl();
        char[] displaySnippet();
}


~~~~~~~~~~~~~~~~~~~
SearchFactory.d
~~~~~~~~~~~~~~~~~~~

module SearchFactory;


public import SearchInterfaces;

// private import a.bunch.of.proprietary.stuff;


// a factory for hooking up a client to the hidden implementation
public ISearchEngine createSearchEngine (char[] someAttributes)
{
        // potentially reference a.bunch.of.proprietary.stuff;
        return new XYX(someAttributes).
}


// do some proprietary setup
private static this()
{
        // potentially do a.bunch.of.proprietary.stuff;
}

~~~~~~~~~~~~~~~~~~~~~


That's the Factory implementation, which has a bunch of proprietary stuff in
it along with references to proprietary imports. To correctly decouple a
client from this implementation, one should provide another file (I'm
covering Walter's statements here, from a long time ago) along the following
lines:

~~~~~~~~~~~~~~~~~~~
SearchFactory.di
~~~~~~~~~~~~~~~~~~~

module SearchFactory;   // I believe this has to be the same name?

public import SearchInterfaces;

public ISearchEngine createSearchEngine (char[] someAttributes);

~~~~~~~~~~~~~~~~~~~




"Walter Bright" <newshound digitalmars.com> wrote in message 
news:dp6hsf$1hdi$1 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp56kj$gc8$1 digitaldaemon.com...

 This setup does the right thing from an implementation-hiding aspect. 
 It's what's been asked for many times in the past. Note that only public 
 decalarations have been exposed. There's no class in there at all, just 
 to make it as simple as possible. Unfortunately, I can't show the -H 
 version. However, it's not much of a stretch to map the above onto 
 Derek's original example, and assert that (for example) the 
 implementation for both the factory method, and the static ctor() would 
 be exposed (along with any other method considered to be inline-able) 
 That would be no good, Walter. If it were the case, we'd have to ship all 
 those proprietary imports too, along with everything they referenced ~ ad 
 infinitum.

 I do wish I had a working compiler so this could be more concrete.


 I know what you're talking about. But it just isn't practical. It isn't 
 just turning off inlining - the types of the function parameters can 
 reference private types or types in imported modules. So can parameter 
 default values. So can const declarations. Templates too.

 There are two solutions you can use. Both involve putting your interfaces 
 in a separate module. One means making a module just for your factory 
 stub:
    ISearchEngine createSearchEngine (char[] someAttributes);
 Another is to use the COM approach of having a magic cookie (GUIDs serve 
 as the magic cookie for COM) for each class, and pass that cookie to a 
 generic factory function that is itself an interface member.

 





 Dec 31 2005



  parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"Kris" <fu bar.com> wrote in message news:dp6onr$1m6u$1 digitaldaemon.com...

 Okay. I think there's an opportunity to clear this up via a few short 
 questions. These refer back to the SearchEngine example posted previously 
 (repeated below):

 a) The example provided, where the SearchFactory.di file was hand-crafted? 
 Will usage of that "di" module operate correctly?

 b) You say below " it just isn't practical". Do you mean it is not 
 technically feasible to mechanically extract the example SearchFactory.di 
 from the given SearchFactory.d?


It isn't practical to determine that a declaration has *no* dependencies on 
any private class members or anything imported from an embedded import 
statement. It *is* practical to elide the bodies of functions that don't 
appear in templates, because there's no way for another declaration to look 
inside it, so -H does that. static this's are also elided, as nothing else 
can refer to them anyway.

Think of it in these terms, and I think it'll become clear. -H can elide 
something *only* if it can guarantee there is nothing that can look inside 
it.

Also, .di files have to be independent of changes in files the .di file 
imports. Thus, the import declarations have to remain, even if they are 
private imports.

It is technically feasible to analyse the source code to the point where one 
can prove it depends on no private data, no private imports, etc., but that 
is a huge project, and would likely not yield much as few source files are 
written this way. Hence, I said it was impractical. 



 Dec 31 2005



  parent reply "Kris" <fu bar.com>  writes:


"Walter Bright" <newshound digitalmars.com> wrote in message 
news:dp7459$1stn$1 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp6onr$1m6u$1 digitaldaemon.com...

 Okay. I think there's an opportunity to clear this up via a few short 
 questions. These refer back to the SearchEngine example posted previously 
 (repeated below):

 a) The example provided, where the SearchFactory.di file was 
 hand-crafted? Will usage of that "di" module operate correctly?

 b) You say below " it just isn't practical". Do you mean it is not 
 technically feasible to mechanically extract the example SearchFactory.di 
 from the given SearchFactory.d?


 It isn't practical to determine that a declaration has *no* dependencies 
 on any private class members or anything imported from an embedded import 
 statement. It *is* practical to elide the bodies of functions that don't 
 appear in templates, because there's no way for another declaration to 
 look inside it, so -H does that. static this's are also elided, as nothing 
 else can refer to them anyway.



Okay. And what about question (a) above, please? 



 Dec 31 2005



  parent  "Walter Bright" <newshound digitalmars.com>  writes:


"Kris" <fu bar.com> wrote in message news:dp76j5$1u9p$1 digitaldaemon.com...

 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:dp7459$1stn$1 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp6onr$1m6u$1 digitaldaemon.com...

 Okay. I think there's an opportunity to clear this up via a few short 
 questions. These refer back to the SearchEngine example posted 
 previously (repeated below):

 a) The example provided, where the SearchFactory.di file was 
 hand-crafted? Will usage of that "di" module operate correctly?




 Okay. And what about question (a) above, please?


It should work fine. 



 Jan 03 2006



prev sibling  parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"Kris" <fu bar.com> wrote in message news:dp4oq5$6qt$1 digitaldaemon.com...

 I can point to one side-effect that actually does help: the compiler now 
 looks for "di" files, which means that hand-coded "implementation 
 bridging" files can (at last!) live side-by-side with the real 
 implementation module, eliminating some prior grief for a developer. 
 However, this conflicts with -H rather badly, and therefore is likely to 
 be an unintended and somewhat precarious benefit. Was that aspect 
 considered?


Actually, it's intended. You can automatically build them, hand build them, 
or hand edit the automatically built one. 



 Dec 30 2005



  parent reply "Kris" <fu bar.com>  writes:


"Walter Bright" <newshound digitalmars.com> wrote in message 
news:dp52vk$dqu$2 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp4oq5$6qt$1 digitaldaemon.com...

 I can point to one side-effect that actually does help: the compiler now 
 looks for "di" files, which means that hand-coded "implementation 
 bridging" files can (at last!) live side-by-side with the real 
 implementation module, eliminating some prior grief for a developer. 
 However, this conflicts with -H rather badly, and therefore is likely to 
 be an unintended and somewhat precarious benefit. Was that aspect 
 considered?


 Actually, it's intended. You can automatically build them, hand build 
 them, or hand edit the automatically built one.


You can imagine the poor sod who carefully crafts his "di" file by hand 
(through necessity), only to find the "performance enhancing" aspects of the 
compiler trash his file at some later point.

This ain't good. 



 Dec 30 2005



  parent reply Sean Kelly <sean f4.ca>  writes:


Kris wrote:

 "Walter Bright" <newshound digitalmars.com> wrote in message 
 news:dp52vk$dqu$2 digitaldaemon.com...

 "Kris" <fu bar.com> wrote in message 
 news:dp4oq5$6qt$1 digitaldaemon.com...

 I can point to one side-effect that actually does help: the compiler now 
 looks for "di" files, which means that hand-coded "implementation 
 bridging" files can (at last!) live side-by-side with the real 
 implementation module, eliminating some prior grief for a developer. 
 However, this conflicts with -H rather badly, and therefore is likely to 
 be an unintended and somewhat precarious benefit. Was that aspect 
 considered?


 Actually, it's intended. You can automatically build them, hand build 
 them, or hand edit the automatically built one.


 
 You can imagine the poor sod who carefully crafts his "di" file by hand 
 (through necessity), only to find the "performance enhancing" aspects of the 
 compiler trash his file at some later point.


But don't you have to supply a -H command-line parameter to indicate 
that the header should be generated?  I'm not sure it's fair to 
criticize DMD for what seems like user error.


Sean



 Dec 30 2005



  parent  "Kris" <fu bar.com>  writes:


"Sean Kelly" <sean f4.ca> wrote...

 But don't you have to supply a -H command-line parameter to indicate that 
 the header should be generated?  I'm not sure it's fair to criticize DMD 
 for what seems like user error.


It's a precarious approach, which I noted in the OP. You've used Build 
before ... it's very easy to take a mis-step here, in all manner of ways.



 Dec 30 2005



prev sibling  parent reply James Dunne <james.jdunne gmail.com>  writes:


For implementation hiding, just move the private members in classes to 
the end of the vtable, after all public and protected members (define 
this in the ABI and modify the reference compiler to guarantee this). 
Problem solved.

No more private members need to be included in the generated header 
files because their vtable offsets are always greater than vtable 
offsets of public/protected members and will have no push/pull affect on 
them if left out.  Simply ignoring them in the _header_ class creates no 
binary compatibility issues.

Const initializers; simply include them as well in the generated header 
classes for public/protected members to preserve correct offsets.

Templates... well that's another story.



 Dec 30 2005



 next sibling parent  John Reimer <terminal.node gmail.com>  writes:


James Dunne wrote:

 For implementation hiding, just move the private members in classes to 
 the end of the vtable, after all public and protected members (define 
 this in the ABI and modify the reference compiler to guarantee this). 
 Problem solved.
 
 No more private members need to be included in the generated header 
 files because their vtable offsets are always greater than vtable 
 offsets of public/protected members and will have no push/pull affect on 
 them if left out.  Simply ignoring them in the _header_ class creates no 
 binary compatibility issues.
 
 Const initializers; simply include them as well in the generated header 
 classes for public/protected members to preserve correct offsets.
 


I was wondering about this!


 Templates... well that's another story.


I guess it is another story.  Maybe templates should just stay in there 
own D file and be processed the old fashioned way.

Oh boy, look at us.  We're all still trying to solve the 
implementation-hiding issue, insatiable creatures we are. :)

-JJR



 Dec 30 2005



prev sibling next sibling parent reply "Derek Parnell" <derek psych.ward>  writes:


On Sat, 31 Dec 2005 18:03:46 +1100, James Dunne <james.jdunne gmail.com>  
wrote:


 For implementation hiding, just move the private members in classes to  
 the end of the vtable, after all public and protected members (define  
 this in the ABI and modify the reference compiler to guarantee this).  
 Problem solved.

 No more private members need to be included in the generated header  
 files because their vtable offsets are always greater than vtable  
 offsets of public/protected members and will have no push/pull affect on  
 them if left out.  Simply ignoring them in the _header_ class creates no  
 binary compatibility issues.

 Const initializers; simply include them as well in the generated header  
 classes for public/protected members to preserve correct offsets.

 Templates... well that's another story.


I'm sure this is all true, but wouldn't it be nice if we had some  
automated way of achieving this end, rather than relying on hand crafted  
code.

-- 
Derek Parnell
Melbourne, Australia



 Dec 31 2005



 next sibling parent  John Reimer <terminal.node gmail.com>  writes:


Derek Parnell wrote:

 On Sat, 31 Dec 2005 18:03:46 +1100, James Dunne <james.jdunne gmail.com> 
 wrote:
 

 For implementation hiding, just move the private members in classes to 
 the end of the vtable, after all public and protected members (define 
 this in the ABI and modify the reference compiler to guarantee this). 
 Problem solved.

 No more private members need to be included in the generated header 
 files because their vtable offsets are always greater than vtable 
 offsets of public/protected members and will have no push/pull affect 
 on them if left out.  Simply ignoring them in the _header_ class 
 creates no binary compatibility issues.

 Const initializers; simply include them as well in the generated 
 header classes for public/protected members to preserve correct offsets.

 Templates... well that's another story.


 
 I'm sure this is all true, but wouldn't it be nice if we had some 
 automated way of achieving this end, rather than relying on hand crafted 
 code.
 
 --Derek Parnell
 Melbourne, Australia


I think that was the idea he was trying to get across: it's not 
impossible to do.



 Dec 31 2005



prev sibling  parent  James Dunne <james.jdunne gmail.com>  writes:


Derek Parnell wrote:

 On Sat, 31 Dec 2005 18:03:46 +1100, James Dunne 
 <james.jdunne gmail.com>  wrote:
 

 For implementation hiding, just move the private members in classes 
 to  the end of the vtable, after all public and protected members 
 (define  this in the ABI and modify the reference compiler to 
 guarantee this).  Problem solved.

 No more private members need to be included in the generated header  
 files because their vtable offsets are always greater than vtable  
 offsets of public/protected members and will have no push/pull affect 
 on  them if left out.  Simply ignoring them in the _header_ class 
 creates no  binary compatibility issues.

 Const initializers; simply include them as well in the generated 
 header  classes for public/protected members to preserve correct offsets.

 Templates... well that's another story.


 
 
 I'm sure this is all true, but wouldn't it be nice if we had some  
 automated way of achieving this end, rather than relying on hand 
 crafted  code.
 


That's what I was saying... a compiler change.  It would be silly to 
require all D programmers to define their private members at the end of 
their classes.



 Dec 31 2005



prev sibling  parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"James Dunne" <james.jdunne gmail.com> wrote in message 
news:dp5agf$ioj$1 digitaldaemon.com...

 For implementation hiding, just move the private members in classes to the 
 end of the vtable, after all public and protected members (define this in 
 the ABI and modify the reference compiler to guarantee this). Problem 
 solved.


That works until you try to derive from it. <g>


 Const initializers; simply include them as well in the generated header 
 classes for public/protected members to preserve correct offsets.


? Const initializers may refer to private declarations.


 Templates... well that's another story.


OOP is implementation hiding, templates are the antithesis of implementation 
hiding. In C++, exported templates were created to solve this problem, but 
it's pretty obvious to me that it does no such thing. I haven't been too 
popular in some circles for pointing this out <g>. 



 Dec 31 2005



 next sibling parent reply Sean Kelly <sean f4.ca>  writes:


Walter Bright wrote:

 "James Dunne" <james.jdunne gmail.com> wrote in message 
 

 Templates... well that's another story.


 
 OOP is implementation hiding, templates are the antithesis of implementation 
 hiding. In C++, exported templates were created to solve this problem, but 
 it's pretty obvious to me that it does no such thing. I haven't been too 
 popular in some circles for pointing this out <g>. 


There's been a lot of confusion about the real purpose of "export" in 
C++, much like the -H option here ;-)  All the feature really provides 
is a way to speed compilation--implementation hiding is not really 
possible (as you've pointed out).  Whether potential benefit of "export" 
is worth the development cost is an issue of much contention.  The folks 
at EDG argued against the feature before it was made standard, but now 
they're arguing for it... possibly because they've spent so much time 
implementing it.  I don't think Microsoft has any plans of implementing 
"export," though I haven't heard anything from other vendors.


Sean



 Dec 31 2005



  parent  "Walter Bright" <newshound digitalmars.com>  writes:


"Sean Kelly" <sean f4.ca> wrote in message 
news:dp6j8o$1ibk$1 digitaldaemon.com...

 There's been a lot of confusion about the real purpose of "export" in C++, 
 much like the -H option here ;-)  All the feature really provides is a way 
 to speed compilation--implementation hiding is not really possible (as 
 you've pointed out).


I don't believe it speeds up compilation either. None of the EDG compilers 
have been willing to take up my challenge on DMC++ with precompiled headers 
vs EDG and exported templates <g>.


 Whether potential benefit of "export" is worth the development cost is an 
 issue of much contention.


The people who argue it is worth it are either the ones for which it is a 
sunk cost or are not the ones who'll have to pay for it. Although I've 
pointed out that the ones who think they don't have to pay for it will 
actually have to pay for it, in terms that they won't get other 
features/support they probably need much more.


 The folks at EDG argued against the feature before it was made standard, 
 but now they're arguing for it... possibly because they've spent so much 
 time implementing it.


For EDG, it's a sunk cost. Their cost for the feature going forward is 
minimal, so once it is a sunk cost, the cost/benefit ratio suddenly becomes 
very favorable. It's in EDG's economic best interest at this point to 
strongly support export.


 I don't think Microsoft has any plans of implementing "export," though I 
 haven't heard anything from other vendors.


I haven't heard anyone else planning to implement it, either. 



 Dec 31 2005



prev sibling  parent reply James Dunne <james.jdunne gmail.com>  writes:


Walter Bright wrote:

 "James Dunne" <james.jdunne gmail.com> wrote in message 
 news:dp5agf$ioj$1 digitaldaemon.com...
 

For implementation hiding, just move the private members in classes to the 
end of the vtable, after all public and protected members (define this in 
the ABI and modify the reference compiler to guarantee this). Problem 
solved.


 
 That works until you try to derive from it. <g>
 


*thinking...*

Ow.

Well, who says the binary layout of derived classes must be equivalent? 
  Isn't this just an implementation detail that most compilers assume is 
true in order to gain speed and ease of implementation for 
virtual/overridden methods?

I'll be thinking this one out for a while ...


Const initializers; simply include them as well in the generated header 
classes for public/protected members to preserve correct offsets.


 
 
 ? Const initializers may refer to private declarations.
 


So there's the catcher... I didn't know that, thanks!

Well, why not const-fold out that private declaration into the initializer?

Oh right... again with the deriving... I always forget about the deriving...


 

Templates... well that's another story.


 
 
 OOP is implementation hiding, templates are the antithesis of implementation 
 hiding. In C++, exported templates were created to solve this problem, but 
 it's pretty obvious to me that it does no such thing. I haven't been too 
 popular in some circles for pointing this out <g>. 
 


Anyone wishing to hide their implementation of templates should be 
reconsidering their worth to society. =)



 Dec 31 2005



  parent  "Kris" <fu bar.com>  writes:


"James Dunne" <james.jdunne gmail.com> wrote in


 Anyone wishing to hide their implementation of templates should be 
 reconsidering their worth to society. =)


Amen. 



 Dec 31 2005



prev sibling  parent reply Carlos Santander <csantander619 gmail.com>  writes:


John Reimer escribió:

 
 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. Obviously 
 we got the wrong idea about the whole thing.
 
 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).
 
 Good to know.  Personally, I was less interested in compile performance 
 and more interested in implementation hiding.  The -H flag, therefore, 
 will not be of much use to me.
 
 -JJR


Same here. I mean, I also agree with all of you guys that -H should do
something 
different.

BTW, IIRC, digc did something like that. Does anybody know if it still
compiles? 
Did it really do that?

-- 
Carlos Santander Bernal



 Dec 30 2005



  parent  John Reimer <terminal.node gmail.com>  writes:


Carlos Santander wrote:

 John Reimer escribió:

 Okay, now I see why there is so much confusion going on with the 
 addition of this feature.  There are two problems to be solved: one is 
 the issue of implementation hiding; the other is the issue of compiler 
 performance.  I think most of us mistook the di file generator to be a 
 solution for implementation hiding for closed source projects. 
 Obviously we got the wrong idea about the whole thing.

 If di files serve the same purpose as "precompiled headers," obviously 
 there is no consideration whatsoever for implementation hiding (and 
 there never was any such intention).

 Good to know.  Personally, I was less interested in compile 
 performance and more interested in implementation hiding.  The -H 
 flag, therefore, will not be of much use to me.

 -JJR


 
 Same here. I mean, I also agree with all of you guys that -H should do 
 something different.
 
 BTW, IIRC, digc did something like that. Does anybody know if it still 
 compiles? Did it really do that?
 


I think digc was updated several months ago by Burton to do a whole lot 
more than it used to do.  It might be worth checking out.  I distinctly 
recall that it received very little attention when he announced the 
update, which was a surprise to me.  I haven't looked into it much 
myself, though.

-JJR



 Dec 30 2005



prev sibling next sibling parent reply bobef <bobef lessequal.com>  writes:


Walter Bright wrote:

 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!
 
 http://www.digitalmars.com/d/changelog.html
 
 



I play OpenTTD all night and in the morning see what happens :)
Seriously. I won't participate in the dispute because I am not 100% sure 
I fully understand it (I am weak with all these programming terms :), 
but I wish to support Walter in some way. As much as I understand the 
new feature helps speeding up the compiler but it isn't hiding all the 
code wich comes to be a problem for people wanting close source 
projects. From the posts I almost thought nobody feels the slow 
compilition a problem and it is the "people vs Walter". I want to say 
that this was exactly what I needed, because the compilation speed is a 
big issue to me. Once I started a thread about this. (Now I am shooting 
in the dark sorry if I am wrong) It seems that many people are using the 
build tool (which I've never used but I guess it works this way) or even 
if they are not they pass all the source files to the compiler. In this 
case the speed is optimal but if you compile each file separately it 
becomes like 15 times slower (for my project). And I do it because I 
have more control over the process this way.

So I just wanted to support Walter and say that this is a very useful 
feature for me. Thank you, Walter (with much help from Dave Fladebo ;).

P.S. I didn't undestand how you control if the compiler is importing .d 
or .di . I looked in the docs but I didn't find any info on that. I mean 
if they have the same name?



 Dec 30 2005



 next sibling parent reply "Derek Parnell" <derek psych.ward>  writes:


On Fri, 30 Dec 2005 21:10:52 +1100, bobef <bobef lessequal.com> wrote:

 P.S. I didn't undestand how you control if the compiler is importing .d  
 or .di . I looked in the docs but I didn't find any info on that. I mean  
 if they have the same name?


If you code "import xyz" the compiler looks first for xyz.d, and if it  
can't find that it tries to find xyz.di and if that's not found it errors.

-- 
Derek Parnell
Melbourne, Australia



 Dec 30 2005



  parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"Derek Parnell" <derek psych.ward> wrote in message 
news:op.s2le7jga6b8z09 ginger.vic.bigpond.net.au...

 On Fri, 30 Dec 2005 21:10:52 +1100, bobef <bobef lessequal.com> wrote:

 P.S. I didn't undestand how you control if the compiler is importing .d 
 or .di . I looked in the docs but I didn't find any info on that. I mean 
 if they have the same name?


 If you code "import xyz" the compiler looks first for xyz.d, and if it 
 can't find that it tries to find xyz.di and if that's not found it errors.


No, first it looks for .di, then .d. 



 Dec 30 2005



  parent  "Derek Parnell" <derek psych.ward>  writes:


On Sat, 31 Dec 2005 10:28:10 +1100, Walter Bright  
<newshound digitalmars.com> wrote:


 "Derek Parnell" <derek psych.ward> wrote in message
 news:op.s2le7jga6b8z09 ginger.vic.bigpond.net.au...

 On Fri, 30 Dec 2005 21:10:52 +1100, bobef <bobef lessequal.com> wrote:

 P.S. I didn't undestand how you control if the compiler is importing .d
 or .di . I looked in the docs but I didn't find any info on that. I  
 mean
 if they have the same name?


 If you code "import xyz" the compiler looks first for xyz.d, and if it
 can't find that it tries to find xyz.di and if that's not found it  
 errors.


 No, first it looks for .di, then .d.


Brillilant! I think I can use that behaviour in the macro preprocessor  
coming soon to Build.



-- 
Derek Parnell
Melbourne, Australia



 Dec 31 2005



prev sibling next sibling parent reply Ivan Senji <ivan.senji_REMOVE_ _THIS__gmail.com>  writes:


bobef wrote:

 Walter Bright wrote:
 

 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!

 http://www.digitalmars.com/d/changelog.html


 
 
 I play OpenTTD all night and in the morning see what happens :)


Wow! It is one of the *best* games I ever played. It is great to see 
there are more fans.



 Dec 30 2005



 next sibling parent  John Reimer <terminal.node gmail.com>  writes:


Ivan Senji wrote:

 bobef wrote:

 Walter Bright wrote:


 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!

 http://www.digitalmars.com/d/changelog.html



 I play OpenTTD all night and in the morning see what happens :)


 
 Wow! It is one of the *best* games I ever played. It is great to see 
 there are more fans.



Hmmm... I've never tried it before.  I'll have to take a look.

-JJR



 Dec 30 2005



prev sibling  parent reply John Reimer <terminal.node gmail.com>  writes:


Ivan Senji wrote:

 bobef wrote:

 Walter Bright wrote:


 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!

 http://www.digitalmars.com/d/changelog.html



 I play OpenTTD all night and in the morning see what happens :)


 
 Wow! It is one of the *best* games I ever played. It is great to see 
 there are more fans.


Woah!  I didn't realize that I needed the original game graphics in 
order to make this work.  So much for that.

-JJR



 Dec 30 2005



  parent  Ivan Senji <ivan.senji_REMOVE_ _THIS__gmail.com>  writes:


John Reimer wrote:

 Ivan Senji wrote:
 

 bobef wrote:


 Walter Bright wrote:


 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!

 http://www.digitalmars.com/d/changelog.html



 I play OpenTTD all night and in the morning see what happens :)



 Wow! It is one of the *best* games I ever played. It is great to see 
 there are more fans.


 
 
 Woah!  I didn't realize that I needed the original game graphics in 
 order to make this work.  So much for that.
 


You can find patched original version of the game, it is practically the 
same as OpenTTD and has original graphics. But you have to be warned 
that some people really don't like this type of games. I spent countless 
hours playing it, but some people just don't get it. :)



 Dec 30 2005



prev sibling next sibling parent reply J C Calvarese <technocrat7 gmail.com>  writes:


In article <dp3132$1r9i$1 digitaldaemon.com>, bobef says...

I play OpenTTD all night and in the morning see what happens :)
Seriously. I won't participate in the dispute because I am not 100% sure 
I fully understand it (I am weak with all these programming terms :), 


Really, I think it's been more of a misunderstanding than anything else. When
Walter announced that DMD 0.142 could generate header, a few people started
salivating at the thought of having the ability to hide the implementation of
files. It's been request quite a few times, and I'd rather be able to tell them
about "the H switch" than have to teach someone how to do it manually. 

But unfortunately, when these eager D citizens tried out the new compiler
function they realized something was terribly wrong. The compiler wouldn't hide
things that one would expect it to hide. "Hey, Walter, I think it's broken,"
they say.

"No," says Walter. "That's how it's supposed to work. Unless it crashes -- I'll
fix the things that make it crash."

And I'm not saying that what it does isn't useful. Obviously, it must useful to
Dave or he wouldn't have bothered implemented it, and I can see how it could
speed compilation of large projects. I think the ability to hide the
implementation (as much as possible) would be of a greater benefit to the D
community, but I don't see a reason why we couldn't eventually have both. 

So it's more of a case of openning up a present that you think is going to be a
cool DVD, and it ends up being a book. It might be a very good book, but you
were hoping for a movie. (And I do like to read, too; I'm not illiterate or
anything.) ;)


but I wish to support Walter in some way. As much as I understand the 
new feature helps speeding up the compiler but it isn't hiding all the 
code wich comes to be a problem for people wanting close source 
projects. From the posts I almost thought nobody feels the slow 
compilition a problem and it is the "people vs Walter". I want to say 
that this was exactly what I needed, because the compilation speed is a 
big issue to me. Once I started a thread about this. (Now I am shooting 
in the dark sorry if I am wrong) It seems that many people are using the 
build tool (which I've never used but I guess it works this way) or even 
if they are not they pass all the source files to the compiler. In this 
case the speed is optimal but if you compile each file separately it 
becomes like 15 times slower (for my project). And I do it because I 
have more control over the process this way.

So I just wanted to support Walter and say that this is a very useful 
feature for me. Thank you, Walter (with much help from Dave Fladebo ;).


I'm not at all against Walter adding this feature to the compiler (and I don't
think anyone else really is against it either). I just think he should have
called it something else in the announcement. Like "header generator for faster
compiling". (Of course, everyone is much better of naming stuff than I am, so
that's not a very good suggestion.) Oh, well. What's done is done.

So thanks Walter and Dave. Good job. It's not what I expected, but maybe it's
just what D needs. ;)

jcc7



 Dec 30 2005



  parent reply "Walter Bright" <newshound digitalmars.com>  writes:


"J C Calvarese" <technocrat7 gmail.com> wrote in message 
news:dp3vqv$2jkt$1 digitaldaemon.com...

 But unfortunately, when these eager D citizens tried out the new compiler
 function they realized something was terribly wrong. The compiler wouldn't 
 hide
 things that one would expect it to hide. "Hey, Walter, I think it's 
 broken,"
 they say.

 "No," says Walter. "That's how it's supposed to work. Unless it crashes --  
 I'll
 fix the things that make it crash."


It winds up putting the same things in the 'header' that you have to put in 
in C++, for the same reasons. 



 Dec 30 2005



  parent  J C Calvarese <technocrat7 gmail.com>  writes:


In article <dp4hdq$31jl$3 digitaldaemon.com>, Walter Bright says...

"J C Calvarese" <technocrat7 gmail.com> wrote in message 
news:dp3vqv$2jkt$1 digitaldaemon.com...

 But unfortunately, when these eager D citizens tried out the new compiler
 function they realized something was terribly wrong. The compiler wouldn't 
 hide
 things that one would expect it to hide. "Hey, Walter, I think it's 
 broken,"
 they say.

 "No," says Walter. "That's how it's supposed to work. Unless it crashes --  
 I'll
 fix the things that make it crash."


It winds up putting the same things in the 'header' that you have to put in 
in C++, for the same reasons. 


After reading some of the most recent posts, I think I understand better than I
did before what some of these issues boil down to. 

Apparently, implementation-hiding is _a lot_ more complicated than I realized.
Part of why I thought that it wouldn't be that complicated is because dig's
strip.d is a pretty short program. I didn't really test it, but I thought the
dig's strip.d worked pretty good. Now, I'm pretty sure it could work only
because it wasn't thoroughly tested with complicated enough code to make it
choke.

jcc7



 Jan 01 2006



prev sibling  parent  "Walter Bright" <newshound digitalmars.com>  writes:


"bobef" <bobef lessequal.com> wrote in message 
news:dp3132$1r9i$1 digitaldaemon.com...

 So I just wanted to support Walter and say that this is a very useful 
 feature for me. Thank you, Walter (with much help from Dave Fladebo ;).


You're welcome!


 P.S. I didn't undestand how you control if the compiler is importing .d or 
 .di . I looked in the docs but I didn't find any info on that. I mean if 
 they have the same name?


It looks at each directory in the import path, first for a .di file, then a 
.d file. 



 Dec 30 2005



prev sibling  parent  Bruno Medeiros <I.AM FAIAL.com>  writes:


Walter Bright wrote:

 This incorporates a new 'header' generator capability, written by Dave
 Fladebo, now working!
 
 http://www.digitalmars.com/d/changelog.html
 
 


Jeez, what a mess of a thread.

Before of all, as for this pre-parse header feature, since it is 
(primarily?) a performance feature, is it planned that in the future 
(after debugging) this option will be enabled by default?

Now, what is the appropriate way to create and use in D an external 
library, in terms of module importation?
Is it to use the whole library source and import it's modules (but not 
compile them of course)?, Or to hand-craft header modules?
But what if I want to use different versions of said library with the 
same aplication *executable* (say, like updating a buggy DLL)? This 
makes the first option (use full source) inviable, right? (because the 
the bugfix may be in a inlined function?)

Thus, if hand-crafting is the only solution, wouldn't it be useful to 
have a tool that would automatically generate this interface(the set of 
headers)?

However, what is the interface of a module and what is it's 
implemention? I think that it is in this question that many of the 
confusion of this thread originated.
For Walter, this interface is everything that might be need by the 
importing module, even on a binary level, so this includes privates and 
inlinable function bodies.
This, is the standard C++ point of view and pratice, as Walter pointed 
out and exemplified.

However, since in many cases the conceptual interface may be smaller 
than this standard interface, some people "sugested" that the generated 
interface(header) should be closer to this conceptual interface, 
particularly: it should not contain privates and function bodies.

However, for this to be done(implemented), it has to be *correct* and 
*clear* on how it would work. People said that privates shouldn't appear 
on the generated headers, but "private" and protection attributes in 
general are (again I say this) not made for code hiding, but just for 
protection. And so, when using them for information hiding (like 
removing private code from generated headers), it doesn't work, as 
Walter pointed out in many exemples (used internally, problems with the 
binary interface, etc.).
So if one wants to complain about not having this feature implemented, 
then it should first of all be well tought out and defined how the 
feature works. (and unfortunately that didn't happen)
That means extending and clarifying the semantics of private for such 
purpose. For instance when a public function has a private parameter. (I 

PC, so I can't try it out)



 Jan 01 2006