www.digitalmars.com         C & C++   DMDScript  

D - Distributing Libraries/Modules without Source

reply "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> writes:
How would you distribute a D module in a library without giving away the
source code?  Can you just stub out the function calls in the module source
that you distribute along with the library?  Even with this approach I can't
see how you wouldn't be giving away the store because you would need to
include the declarations for all the modules in the application.

What am I missing?
Aug 12 2002
next sibling parent reply "anderson" <anderson firestar.com.au> writes:
D compliers auto-generates the header files.

"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
news:aja2id$4dj$1 digitaldaemon.com...

 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module


source

 that you distribute along with the library?  Even with this approach I


can't

 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.

 What am I missing?
Aug 13 2002
parent reply "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> writes:
 D compliers auto-generates the header files.


Not using header files and, thereby,  comingling definitions and
declarations is not the same as "auto-generating" header files.  I see
nothing that says header files are "generated," such as you would need to do
for external linking with a library that you don't have source for.  I could
picture a world where you could specifiy a .LIB file on the DMD command line
and have the module definition sucked out of that, but that is not
implemented (AFAIK) and certainly wouldn't be implemented with the GCC port.
Aug 13 2002
parent "anderson" <anderson firestar.com.au> writes:
Your right, I think it's in the plan.

"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
news:ajbr21$1uck$1 digitaldaemon.com...

 D compliers auto-generates the header files.


 Not using header files and, thereby,  comingling definitions and
 declarations is not the same as "auto-generating" header files.  I see
 nothing that says header files are "generated," such as you would need to


do

 for external linking with a library that you don't have source for.  I


could

 picture a world where you could specifiy a .LIB file on the DMD command


line

 and have the module definition sucked out of that, but that is not
 implemented (AFAIK) and certainly wouldn't be implemented with the GCC


port.





 Aug 13 2002



prev sibling next sibling parent  Juarez Rudsatz <juarez nowhere.com>  writes:


"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in
news:aja2id$4dj$1 digitaldaemon.com: 


 How would you distribute a D module in a library without giving away
 the source code? 
 
 What am I missing?


 
You can use a source code generated doc.
See the coddoc thread above.



 Aug 13 2002



prev sibling next sibling parent  Pavel Minayev <evilone omen.ru>  writes:


On Mon, 12 Aug 2002 21:44:39 -0700 "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> 
wrote:


 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module source
 that you distribute along with the library?  Even with this approach I can't
 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.
 
 What am I missing?


Currently, there is no way to hide the source. In the future, it will most
likely be much like Free Pascal - when module is compiled, it also generates
a binary file which contains interface for that module.



 Aug 12 2002



prev sibling  parent reply "Walter" <walter digitalmars.com>  writes:


"Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
news:aja2id$4dj$1 digitaldaemon.com...

 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module


source

 that you distribute along with the library?  Even with this approach I


can't

 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.
 What am I missing?


Good question. There are a couple of answers. The first is you can strip out
all the function bodies, and leave the function declarations, like a C++ .h
file. This will work in D - it will link successfully to a library compiled
with the full source. The file phobos/gc.d is an example.

The second answer is that the compiler can generate a symbol file for each
compiled module, and then just read in that symbol file to satisfy the
import statements. For reasons of sloth and laziness, the current D compiler
does not do that.



 Aug 15 2002



 next sibling parent  "anderson" <anderson firestar.com.au>  writes:


I would be good if this could link in with a D doc gen (when one is created)
to alow for more readable output.

"Walter" <walter digitalmars.com> wrote in message
news:ajhtlr$a8o$1 digitaldaemon.com...

 "Joe Battelle" <b-a-t-t-e-l-l-e mail.com> wrote in message
 news:aja2id$4dj$1 digitaldaemon.com...

 How would you distribute a D module in a library without giving away the
 source code?  Can you just stub out the function calls in the module


 source

 that you distribute along with the library?  Even with this approach I


 can't

 see how you wouldn't be giving away the store because you would need to
 include the declarations for all the modules in the application.
 What am I missing?


 Good question. There are a couple of answers. The first is you can strip


out

 all the function bodies, and leave the function declarations, like a C++


.h

 file. This will work in D - it will link successfully to a library


compiled

 with the full source. The file phobos/gc.d is an example.

 The second answer is that the compiler can generate a symbol file for each
 compiled module, and then just read in that symbol file to satisfy the
 import statements. For reasons of sloth and laziness, the current D


compiler

 does not do that.





 Aug 15 2002



prev sibling  parent reply Joe Battelle <Joe_member pathlink.com>  writes:


file. This will work in D - it will link successfully to a library compiled
with the full source. The file phobos/gc.d is an example.


Playing around with this I noticed two things about the current linker, both
pertaining to linking in a module twice.

1) You _CANNOT_ link program P.d with module A.obj and library L.lib containing
module A.obj if _ANY_ of the defitions in the two module A's differ.
2) You _CAN_ link program P with module A.obj and library L.lib containing
module A.obj if the two versions of module A are disjoint.

What this means: (1) disallows a user of an opaque module from over-riding it's
behavior; (2) allows the designer of the opaque library to leave some behavior
up to the user of the library.  These seem very reasonable to me, however, I
would like case 1 relaxed with a link-time switch so that the designer of an
opaque library could allow the user to overide default behavior.

I'm wondering if this is intentional and can be relied on in the future, or if
this is just an artifact of the current implementation?

--- source code for those who are lost ---
This example implements one function (d) in the module and leaves another (a) up
to the user of the library.

//file A.d
module A;
version (special) {
int a() { return 0xA; }
int d() ;
} else {
int a() ;
int d() { return 0xD; }
}

//file test.d
import c.stdio;
import A;

int main (char[][] args)
{
printf("d()=%X a()=%X\n", d(), a());	
return 0;
}

dmd -c A
lib L.lib /c +A.obj;
dmd -c -version=special A
dmd test A.obj L.lib
test

d()=D a()=A





 Aug 15 2002



  parent reply "Walter" <walter digitalmars.com>  writes:


Yes, it is intentional behavior of the linker, which is a standard linker.
To override part of the functionality of a library, the right way to do it
is to replace the entire module or derive from a class.

"Joe Battelle" <Joe_member pathlink.com> wrote in message
news:aji7m4$kn2$1 digitaldaemon.com...

file. This will work in D - it will link successfully to a library




compiled

with the full source. The file phobos/gc.d is an example.


 Playing around with this I noticed two things about the current linker,


both

 pertaining to linking in a module twice.

 1) You _CANNOT_ link program P.d with module A.obj and library L.lib


containing

 module A.obj if _ANY_ of the defitions in the two module A's differ.
 2) You _CAN_ link program P with module A.obj and library L.lib containing
 module A.obj if the two versions of module A are disjoint.

 What this means: (1) disallows a user of an opaque module from over-riding


it's

 behavior; (2) allows the designer of the opaque library to leave some


behavior

 up to the user of the library.  These seem very reasonable to me, however,


I

 would like case 1 relaxed with a link-time switch so that the designer of


an

 opaque library could allow the user to overide default behavior.

 I'm wondering if this is intentional and can be relied on in the future,


or if

 this is just an artifact of the current implementation?

 --- source code for those who are lost ---
 This example implements one function (d) in the module and leaves another


(a) up

 to the user of the library.

 //file A.d
 module A;
 version (special) {
 int a() { return 0xA; }
 int d() ;
 } else {
 int a() ;
 int d() { return 0xD; }
 }

 //file test.d
 import c.stdio;
 import A;

 int main (char[][] args)
 {
 printf("d()=%X a()=%X\n", d(), a());
 return 0;
 }

 dmd -c A
 lib L.lib /c +A.obj;
 dmd -c -version=special A
 dmd test A.obj L.lib
 test

d()=D a()=A







 Aug 16 2002



  parent reply Joe Battelle <Joe_member pathlink.com>  writes:


Yes, it is intentional behavior of the linker, which is a standard linker.
To override part of the functionality of a library, the right way to do it
is to replace the entire module or derive from a class.


Right.  I brought this point up though because mixing source and declarations in
the same file, doing away with includes, gives the impression that all the code
for a D module needs to be in one place.  Thankfully with your present
implementation, D modules can be defined piecewise, and a D module can map to
several .obj modules.  And as you point out, as long as you're overloading at
the .obj module granularity it's fine.  I think it is important that this be
specified in the D spec so that future implementors don't try and outsmart us by
"simplifying" things to D module = .obj module.

Here's a concrete example that fleshes this out:
In embedded systems, it is common to have a framework with a few stub functions
that are specific to your custom hardware.  These low level functions like
timers and basic serial I/O are usually provided by the end user.  In old school
C, these functions were each put in their own .obj module, and the end user
could put his own putchar/getchar or whathaveyou on the link line and customize
the framework behavior.  In this situation, inheritence is out because you
probably don't want to require that the garbage collector be running before you
can output debug messages.  You could stick with the extern "C" functions each
in their own object module, but I would like to see them implemented in D,
having the scope of the D module, including access to variables/functions
without requiring a friend relationship.  This is possible the way D is
currently implemented--and I'm thankful for it.

This is almost a separate thread, but implicit in the above paragraph was the
idea that there is a subset of D that doesn't require a garbage collector.  This
is absolutely essential* for doing embedded systems work.  I am making the naive
assumption that as long as you don't dynamically allocate anything, the
collector is not needed.  Is this true?  Can this be spelled out in the spec as
well?

(*) This is especially true when porting a framework over to an untested piece
of hardware; you want as little as possible getting in the way of your first
effort.



 Aug 16 2002



 next sibling parent reply Joe Battelle <Joe_member pathlink.com>  writes:


One last thing on this, I realize that most of this goes away if you just have
two separate D modules, both importing each other.  And that may even be a
cleaner design decision for some applications.  I wasn't arguing that piecewise
module definition was better, or right, just that it is useful and should be
kept.  And if it is kept--acknowledged in the spec.



 Aug 16 2002



  parent  "Walter" <walter digitalmars.com>  writes:


"Joe Battelle" <Joe_member pathlink.com> wrote in message
news:ajjqp0$2bst$1 digitaldaemon.com...

 One last thing on this, I realize that most of this goes away if you just


have

 two separate D modules, both importing each other.  And that may even be a
 cleaner design decision for some applications.  I wasn't arguing that


piecewise

 module definition was better, or right, just that it is useful and should


be

 kept.  And if it is kept--acknowledged in the spec.


I think you're right.



 Aug 17 2002



prev sibling  parent reply "Walter" <walter digitalmars.com>  writes:


"Joe Battelle" <Joe_member pathlink.com> wrote in message
news:ajjpkv$2ami$1 digitaldaemon.com...

 This is almost a separate thread, but implicit in the above paragraph was


the

 idea that there is a subset of D that doesn't require a garbage collector.


This

 is absolutely essential* for doing embedded systems work.  I am making the


naive

 assumption that as long as you don't dynamically allocate anything, the
 collector is not needed.  Is this true?  Can this be spelled out in the


spec as

 well?


Yes, it is true. In fact, you can malloc/free objects (this is how the gc
bootstraps itself!).



 Aug 17 2002



  parent reply Pavel Minayev <evilone omen.ru>  writes:


On Sat, 17 Aug 2002 09:35:04 -0700 "Walter" <walter digitalmars.com> wrote:


 Yes, it is true. In fact, you can malloc/free objects (this is how the gc
 bootstraps itself!).


But don't dynamic arrays use GC to allocate memory?



 Aug 17 2002



  parent reply "Walter" <walter digitalmars.com>  writes:


"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN37485980272581 news.digitalmars.com...

 On Sat, 17 Aug 2002 09:35:04 -0700 "Walter" <walter digitalmars.com>


wrote:

 Yes, it is true. In fact, you can malloc/free objects (this is how the




gc

 bootstraps itself!).


 But don't dynamic arrays use GC to allocate memory?


Yes, but you can malloc a dynamic array:

    char *p = (char *)malloc(size);
    char[] a = p[0 .. size];



 Aug 17 2002



  parent reply Pavel Minayev <evilone omen.ru>  writes:


On Sat, 17 Aug 2002 21:34:54 -0700 "Walter" <walter digitalmars.com> wrote:


 Yes, but you can malloc a dynamic array:
 
     char *p = (char *)malloc(size);
     char[] a = p[0 .. size];


Yes, you can. But you can't use almost any char[] functions -
toString(), regexp module... many others which allocate 
temporary char[] arrays inside which you don't know about.
You either have to look the code for every function you use,
to see if it uses dynamic arrays, or to use only API calls.



 Aug 18 2002



  parent  "Walter" <walter digitalmars.com>  writes:


"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374865090227199 news.digitalmars.com...

 On Sat, 17 Aug 2002 21:34:54 -0700 "Walter" <walter digitalmars.com>


wrote:

 Yes, but you can malloc a dynamic array:
     char *p = (char *)malloc(size);
     char[] a = p[0 .. size];


 Yes, you can. But you can't use almost any char[] functions -
 toString(), regexp module... many others which allocate
 temporary char[] arrays inside which you don't know about.
 You either have to look the code for every function you use,
 to see if it uses dynamic arrays, or to use only API calls.


That's right, you'll likely need to use your own library code.



 Aug 20 2002