www.digitalmars.com         C & C++   DMDScript  

D - Distributing Libraries/Modules without Source

reply "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> writes:
How would you distribute a D module in a library without giving away the
source code?  Can you just stub out the function calls in the module source
that you distribute along with the library?  Even with this approach I can't
see how you wouldn't be giving away the store because you would need to
include the declarations for all the modules in the application.

What am I missing?
Aug 12 2002
next sibling parent reply "anderson" <anderson firestar.com.au> writes:
D compliers auto-generates the header files.

"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
news:aja2id$4dj$1 digitaldaemon.com...
 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module

 that you distribute along with the library?  Even with this approach I

 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.

 What am I missing?

Aug 13 2002
parent reply "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> writes:
 D compliers auto-generates the header files.

declarations is not the same as "auto-generating" header files. I see nothing that says header files are "generated," such as you would need to do for external linking with a library that you don't have source for. I could picture a world where you could specifiy a .LIB file on the DMD command line and have the module definition sucked out of that, but that is not implemented (AFAIK) and certainly wouldn't be implemented with the GCC port.
Aug 13 2002
parent "anderson" <anderson firestar.com.au> writes:
Your right, I think it's in the plan.

"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
news:ajbr21$1uck$1 digitaldaemon.com...
 D compliers auto-generates the header files.

declarations is not the same as "auto-generating" header files. I see nothing that says header files are "generated," such as you would need to

 for external linking with a library that you don't have source for.  I

 picture a world where you could specifiy a .LIB file on the DMD command

 and have the module definition sucked out of that, but that is not
 implemented (AFAIK) and certainly wouldn't be implemented with the GCC


Aug 13 2002
prev sibling next sibling parent Juarez Rudsatz <juarez nowhere.com> writes:
"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in
news:aja2id$4dj$1 digitaldaemon.com: 

 How would you distribute a D module in a library without giving away
 the source code? 
 
 What am I missing?

You can use a source code generated doc. See the coddoc thread above.
Aug 13 2002
prev sibling next sibling parent Pavel Minayev <evilone omen.ru> writes:
On Mon, 12 Aug 2002 21:44:39 -0700 "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> 
wrote:

 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module source
 that you distribute along with the library?  Even with this approach I can't
 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.
 
 What am I missing?

Currently, there is no way to hide the source. In the future, it will most likely be much like Free Pascal - when module is compiled, it also generates a binary file which contains interface for that module.
Aug 12 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
news:aja2id$4dj$1 digitaldaemon.com...
 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module

 that you distribute along with the library?  Even with this approach I

 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.
 What am I missing?

Good question. There are a couple of answers. The first is you can strip out all the function bodies, and leave the function declarations, like a C++ .h file. This will work in D - it will link successfully to a library compiled with the full source. The file phobos/gc.d is an example. The second answer is that the compiler can generate a symbol file for each compiled module, and then just read in that symbol file to satisfy the import statements. For reasons of sloth and laziness, the current D compiler does not do that.
Aug 15 2002
next sibling parent "anderson" <anderson firestar.com.au> writes:
I would be good if this could link in with a D doc gen (when one is created)
to alow for more readable output.

"Walter" <walter digitalmars.com> wrote in message
news:ajhtlr$a8o$1 digitaldaemon.com...
 "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
 news:aja2id$4dj$1 digitaldaemon.com...
 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module

 that you distribute along with the library?  Even with this approach I

 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.
 What am I missing?

Good question. There are a couple of answers. The first is you can strip

 all the function bodies, and leave the function declarations, like a C++

 file. This will work in D - it will link successfully to a library

 with the full source. The file phobos/gc.d is an example.

 The second answer is that the compiler can generate a symbol file for each
 compiled module, and then just read in that symbol file to satisfy the
 import statements. For reasons of sloth and laziness, the current D

 does not do that.

Aug 15 2002
prev sibling parent reply Joe Battelle <Joe_member pathlink.com> writes:
file. This will work in D - it will link successfully to a library compiled
with the full source. The file phobos/gc.d is an example.

Playing around with this I noticed two things about the current linker, both pertaining to linking in a module twice. 1) You _CANNOT_ link program P.d with module A.obj and library L.lib containing module A.obj if _ANY_ of the defitions in the two module A's differ. 2) You _CAN_ link program P with module A.obj and library L.lib containing module A.obj if the two versions of module A are disjoint. What this means: (1) disallows a user of an opaque module from over-riding it's behavior; (2) allows the designer of the opaque library to leave some behavior up to the user of the library. These seem very reasonable to me, however, I would like case 1 relaxed with a link-time switch so that the designer of an opaque library could allow the user to overide default behavior. I'm wondering if this is intentional and can be relied on in the future, or if this is just an artifact of the current implementation? --- source code for those who are lost --- This example implements one function (d) in the module and leaves another (a) up to the user of the library. //file A.d module A; version (special) { int a() { return 0xA; } int d() ; } else { int a() ; int d() { return 0xD; } } //file test.d import c.stdio; import A; int main (char[][] args) { printf("d()=%X a()=%X\n", d(), a()); return 0; } dmd -c A lib L.lib /c +A.obj; dmd -c -version=special A dmd test A.obj L.lib test
d()=D a()=A

Aug 15 2002
parent reply "Walter" <walter digitalmars.com> writes:
Yes, it is intentional behavior of the linker, which is a standard linker.
To override part of the functionality of a library, the right way to do it
is to replace the entire module or derive from a class.

"Joe Battelle" <Joe_member pathlink.com> wrote in message
news:aji7m4$kn2$1 digitaldaemon.com...
file. This will work in D - it will link successfully to a library


with the full source. The file phobos/gc.d is an example.

Playing around with this I noticed two things about the current linker,

 pertaining to linking in a module twice.

 1) You _CANNOT_ link program P.d with module A.obj and library L.lib

 module A.obj if _ANY_ of the defitions in the two module A's differ.
 2) You _CAN_ link program P with module A.obj and library L.lib containing
 module A.obj if the two versions of module A are disjoint.

 What this means: (1) disallows a user of an opaque module from over-riding

 behavior; (2) allows the designer of the opaque library to leave some

 up to the user of the library.  These seem very reasonable to me, however,

 would like case 1 relaxed with a link-time switch so that the designer of

 opaque library could allow the user to overide default behavior.

 I'm wondering if this is intentional and can be relied on in the future,

 this is just an artifact of the current implementation?

 --- source code for those who are lost ---
 This example implements one function (d) in the module and leaves another

 to the user of the library.

 //file A.d
 module A;
 version (special) {
 int a() { return 0xA; }
 int d() ;
 } else {
 int a() ;
 int d() { return 0xD; }
 }

 //file test.d
 import c.stdio;
 import A;

 int main (char[][] args)
 {
 printf("d()=%X a()=%X\n", d(), a());
 return 0;
 }

 dmd -c A
 lib L.lib /c +A.obj;
 dmd -c -version=special A
 dmd test A.obj L.lib
 test
d()=D a()=A


Aug 16 2002
parent reply Joe Battelle <Joe_member pathlink.com> writes:
Yes, it is intentional behavior of the linker, which is a standard linker.
To override part of the functionality of a library, the right way to do it
is to replace the entire module or derive from a class.

the same file, doing away with includes, gives the impression that all the code for a D module needs to be in one place. Thankfully with your present implementation, D modules can be defined piecewise, and a D module can map to several .obj modules. And as you point out, as long as you're overloading at the .obj module granularity it's fine. I think it is important that this be specified in the D spec so that future implementors don't try and outsmart us by "simplifying" things to D module = .obj module. Here's a concrete example that fleshes this out: In embedded systems, it is common to have a framework with a few stub functions that are specific to your custom hardware. These low level functions like timers and basic serial I/O are usually provided by the end user. In old school C, these functions were each put in their own .obj module, and the end user could put his own putchar/getchar or whathaveyou on the link line and customize the framework behavior. In this situation, inheritence is out because you probably don't want to require that the garbage collector be running before you can output debug messages. You could stick with the extern "C" functions each in their own object module, but I would like to see them implemented in D, having the scope of the D module, including access to variables/functions without requiring a friend relationship. This is possible the way D is currently implemented--and I'm thankful for it. This is almost a separate thread, but implicit in the above paragraph was the idea that there is a subset of D that doesn't require a garbage collector. This is absolutely essential* for doing embedded systems work. I am making the naive assumption that as long as you don't dynamically allocate anything, the collector is not needed. Is this true? Can this be spelled out in the spec as well? (*) This is especially true when porting a framework over to an untested piece of hardware; you want as little as possible getting in the way of your first effort.
Aug 16 2002
next sibling parent reply Joe Battelle <Joe_member pathlink.com> writes:
One last thing on this, I realize that most of this goes away if you just have
two separate D modules, both importing each other.  And that may even be a
cleaner design decision for some applications.  I wasn't arguing that piecewise
module definition was better, or right, just that it is useful and should be
kept.  And if it is kept--acknowledged in the spec.
Aug 16 2002
parent "Walter" <walter digitalmars.com> writes:
"Joe Battelle" <Joe_member pathlink.com> wrote in message
news:ajjqp0$2bst$1 digitaldaemon.com...
 One last thing on this, I realize that most of this goes away if you just

 two separate D modules, both importing each other.  And that may even be a
 cleaner design decision for some applications.  I wasn't arguing that

 module definition was better, or right, just that it is useful and should

 kept.  And if it is kept--acknowledged in the spec.

I think you're right.
Aug 17 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Joe Battelle" <Joe_member pathlink.com> wrote in message
news:ajjpkv$2ami$1 digitaldaemon.com...
 This is almost a separate thread, but implicit in the above paragraph was

 idea that there is a subset of D that doesn't require a garbage collector.

 is absolutely essential* for doing embedded systems work.  I am making the

 assumption that as long as you don't dynamically allocate anything, the
 collector is not needed.  Is this true?  Can this be spelled out in the

 well?

Yes, it is true. In fact, you can malloc/free objects (this is how the gc bootstraps itself!).
Aug 17 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Sat, 17 Aug 2002 09:35:04 -0700 "Walter" <walter digitalmars.com> wrote:

 Yes, it is true. In fact, you can malloc/free objects (this is how the gc
 bootstraps itself!).

But don't dynamic arrays use GC to allocate memory?
Aug 17 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN37485980272581 news.digitalmars.com...
 On Sat, 17 Aug 2002 09:35:04 -0700 "Walter" <walter digitalmars.com>

 Yes, it is true. In fact, you can malloc/free objects (this is how the


 bootstraps itself!).


Yes, but you can malloc a dynamic array: char *p = (char *)malloc(size); char[] a = p[0 .. size];
Aug 17 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Sat, 17 Aug 2002 21:34:54 -0700 "Walter" <walter digitalmars.com> wrote:

 Yes, but you can malloc a dynamic array:
 
     char *p = (char *)malloc(size);
     char[] a = p[0 .. size];

Yes, you can. But you can't use almost any char[] functions - toString(), regexp module... many others which allocate temporary char[] arrays inside which you don't know about. You either have to look the code for every function you use, to see if it uses dynamic arrays, or to use only API calls.
Aug 18 2002
parent "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374865090227199 news.digitalmars.com...
 On Sat, 17 Aug 2002 21:34:54 -0700 "Walter" <walter digitalmars.com>

 Yes, but you can malloc a dynamic array:
     char *p = (char *)malloc(size);
     char[] a = p[0 .. size];

toString(), regexp module... many others which allocate temporary char[] arrays inside which you don't know about. You either have to look the code for every function you use, to see if it uses dynamic arrays, or to use only API calls.

That's right, you'll likely need to use your own library code.
Aug 20 2002