www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Distributing libraries made with D

reply Niko Korhonen <niktheblak hotmail.com> writes:
Greetings, I'm a newcomer in D and I've been very happy with the 
language so far. There's one topic that I can't figure out though; how 
am I supposed to distribute libraries for D.

Let's suppose I want to program a /closed-source/ library for D. People 
should be able to use that library but without access to the source code.

When using Java I can provide a .jar package with the necessary .class 
files. Users can can import this package and specify the .jar file as a 
compiler argument.

In .NET I provide the generated .dll file and the .xml documentation 
file. These can be plugged in and used in Visual Studio very easily.

In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header 
files. No source code required so far (although Bytecode and MSIL are 
easy to decompile).

It seems that the D compiler requires the full source code for D 
libraries. Whenever I import a package in D source code as such:

#  import mylib;

The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument 
doesn't do the trick. Am I missing something here? Since I want to 
provide directly usable libraries for D I really wouldn't like to limit 
myself to that export(C) assembler stuff.

So can I build full-featured (not exported) libraries with D, for D, 
without opening the source?
Jul 28 2004
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 29 Jul 2004 02:41:30 +0300, Niko Korhonen wrote:

 Greetings, I'm a newcomer in D and I've been very happy with the 
 language so far. There's one topic that I can't figure out though; how 
 am I supposed to distribute libraries for D.
 
 Let's suppose I want to program a /closed-source/ library for D. People 
 should be able to use that library but without access to the source code.
 
 When using Java I can provide a .jar package with the necessary .class 
 files. Users can can import this package and specify the .jar file as a 
 compiler argument.
 
 In .NET I provide the generated .dll file and the .xml documentation 
 file. These can be plugged in and used in Visual Studio very easily.
 
 In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header 
 files. No source code required so far (although Bytecode and MSIL are 
 easy to decompile).
 
 It seems that the D compiler requires the full source code for D 
 libraries. Whenever I import a package in D source code as such:
 
 #  import mylib;
 
 The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument 
 doesn't do the trick. Am I missing something here? Since I want to 
 provide directly usable libraries for D I really wouldn't like to limit 
 myself to that export(C) assembler stuff.
 
 So can I build full-featured (not exported) libraries with D, for D, 
 without opening the source?

It can be done! I worked it out with the help of a few people here. I was *not* intuitive and involved a bit of /smoke & mirrors/. Unfortunately, I can't remember all the steps involved. I'll try to nut it out again and this time write down the process. ;-) -- Derek Melbourne, Australia 29/Jul/04 10:18:26 AM
Jul 28 2004
parent Derek Parnell <derek psych.ward> writes:
On Thu, 29 Jul 2004 10:21:32 +1000, Derek Parnell wrote:

 On Thu, 29 Jul 2004 02:41:30 +0300, Niko Korhonen wrote:
 
 Greetings, I'm a newcomer in D and I've been very happy with the 
 language so far. There's one topic that I can't figure out though; how 
 am I supposed to distribute libraries for D.
 
 Let's suppose I want to program a /closed-source/ library for D. People 
 should be able to use that library but without access to the source code.
 
 When using Java I can provide a .jar package with the necessary .class 
 files. Users can can import this package and specify the .jar file as a 
 compiler argument.
 
 In .NET I provide the generated .dll file and the .xml documentation 
 file. These can be plugged in and used in Visual Studio very easily.
 
 In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header 
 files. No source code required so far (although Bytecode and MSIL are 
 easy to decompile).
 
 It seems that the D compiler requires the full source code for D 
 libraries. Whenever I import a package in D source code as such:
 
 #  import mylib;
 
 The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument 
 doesn't do the trick. Am I missing something here? Since I want to 
 provide directly usable libraries for D I really wouldn't like to limit 
 myself to that export(C) assembler stuff.
 
 So can I build full-featured (not exported) libraries with D, for D, 
 without opening the source?

It can be done! I worked it out with the help of a few people here. I was *not* intuitive and involved a bit of /smoke & mirrors/. Unfortunately, I can't remember all the steps involved. I'll try to nut it out again and this time write down the process. ;-)

Okay. This is what I did... ** Created a folder called 'IMP' ** Stored my source code in here. These were implementation code, and not 'headers'. example: --- file: mfoo.d --- module mfoo; import std.stdio; class Foo { void foo(int x){ writef("FOO\n");} } --- file: mbar.d --- module mbar; import std.stdio; import mfoo; class Bar: Foo { alias Foo.foo foo; void foo(char[] x){writef("BAR\n");}; } ** Compile these with the -c switch. ** Move the resulting *.obj files into a library file (eg. via the DigitalMars lib program) ** Copy the source files. eg. mfoo.d --> mfoo_exp.d mbar.d --> mbar_exp.d ** Edit the _exp.d files and remove all the implementation stuff. example: --- file: mfoo_exp.d --- module mfoo; import std.stdio; class Foo { void foo(int x); } --- file: mbar.d --- module mbar; import std.stdio; import mfoo; class Bar: Foo { alias Foo.foo foo; void foo(char[] x); } ** Create another folder. eg. EXPORT ** Copy the library from IMP to EXPORT ** Copy the _exp.d files from IMP to EXPORT ** Rename the copied _exp.d files in EXPORT to their .d names (remove the _exp) part. ** The EXPORT folder is now your distribution package. To use the distribution package. Create your application code as normal and compile it using the .d files and .lib file from the distribution. example: --- file: test.d --- import mfoo; import mbar; void main() { Foo f = new Foo; Bar b = new Bar; int i; char[] s = "test"; b.foo(i); b.foo(s); } -------------------------- and compiled with ... dmd test mylib.lib So the net result is that the user of your library only gets the .LIB file and the stripped down source code files, containing definitions but no implementation details. -- Derek Melbourne, Australia 29/Jul/04 11:13:49 AM
Jul 28 2004
prev sibling next sibling parent "Vathix" <vathixSpamFix dprogramming.com> writes:
"Niko Korhonen" <niktheblak hotmail.com> wrote in message
news:ce9dgi$phm$1 digitaldaemon.com...
 Greetings, I'm a newcomer in D and I've been very happy with the
 language so far. There's one topic that I can't figure out though; how
 am I supposed to distribute libraries for D.

 Let's suppose I want to program a /closed-source/ library for D. People
 should be able to use that library but without access to the source code.

 When using Java I can provide a .jar package with the necessary .class
 files. Users can can import this package and specify the .jar file as a
 compiler argument.

 In .NET I provide the generated .dll file and the .xml documentation
 file. These can be plugged in and used in Visual Studio very easily.

 In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header
 files. No source code required so far (although Bytecode and MSIL are
 easy to decompile).

 It seems that the D compiler requires the full source code for D
 libraries. Whenever I import a package in D source code as such:

 #  import mylib;

 The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument
 doesn't do the trick. Am I missing something here? Since I want to
 provide directly usable libraries for D I really wouldn't like to limit
 myself to that export(C) assembler stuff.

 So can I build full-featured (not exported) libraries with D, for D,
 without opening the source?

Yes, make a stripped D file just for importing. Given the actual source file is foo.d, make foo_imp.d: module foo; // Not foo_imp. class Foo { int publicfield; void stuff(); } int bar(); // etc ,,..,
Jul 28 2004
prev sibling next sibling parent Andy Friesen <andy ikagames.com> writes:
Niko Korhonen wrote:
 Greetings, I'm a newcomer in D and I've been very happy with the 
 language so far. There's one topic that I can't figure out though; how 
 am I supposed to distribute libraries for D.
 
 Let's suppose I want to program a /closed-source/ library for D. People 
 should be able to use that library but without access to the source code.
 
 So can I build full-featured (not exported) libraries with D, for D, 
 without opening the source?

The D compiler needs declarations, but doesn't care about definitions. Except for templates, you can just leave out the definition: class Foo : Bar { this(); void thing(); int x; etc(); } int nonmember_function(); The linker will sort it all out. -- andy
Jul 28 2004
prev sibling next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Niko Korhonen" <niktheblak hotmail.com> wrote in message
news:ce9dgi$phm$1 digitaldaemon.com...
 So can I build full-featured (not exported) libraries with D, for D,
 without opening the source?

Sure! Look at phobos/object.d and phobos/internal/object.d for an example. The trick is to remove the function bodies from the shipped version of the source, and leave the bodies in for the compiled library that you ship.
Jul 28 2004
parent reply David Tiktin <dtiktin nospam.totally-bogus.com> writes:
On 28 Jul 2004, "Walter" <newshound digitalmars.com> wrote:

 "Niko Korhonen" <niktheblak hotmail.com> wrote in message
 news:ce9dgi$phm$1 digitaldaemon.com...
 So can I build full-featured (not exported) libraries with D, for
 D, without opening the source?

Sure! Look at phobos/object.d and phobos/internal/object.d for an example. The trick is to remove the function bodies from the shipped version of the source, and leave the bodies in for the compiled library that you ship.

I think this is an important issue and not just for keeping source code closed. One feature I have not yet found in D is a clean solution for separating interface from implementation in the way C prototypes and forward declarations do. So far, this method of stripping out the implementation seems to be the *only* offered solution, and it seems kludgy. Would it be possible to do something like this? A module could (optionally) use 2 files, say x.d and x.i. x.d would be the "stripped" interface file. x.i would be the "full" implementation file. x.d would have a statement like: module x; implementation x; When building the object x.obj, the compiler would "bring in" the implementation and compile the "combined" file. When compiling any other object using "import x" it would not. It would just see the .d interface file. You'd just ship the .lib containing the .obj and .d files. Just a thought. BTW, do you have a tool or script of some kind that strips the function bodies? It looks to me like that might be hard to do reliably without actually parsing the code to distinguish function bodies from other types of blocks. Dave -- D.a.v.i.d T.i.k.t.i.n t.i.k.t.i.n [at] a.d.v.a.n.c.e.d.r.e.l.a.y [dot] c.o.m
Jul 29 2004
next sibling parent h3r3tic <h3r3tic_member pathlink.com> writes:
I think this is an important issue and not just for keeping source 
code closed.  One feature I have not yet found in D is a clean 
solution for separating interface from implementation in the way C 
prototypes and forward declarations do.  So far, this method of 
stripping out the implementation seems to be the *only* offered 
solution, and it seems kludgy.

Exactly the thing I've been thinking about recently.
  x.i would be the "full" implementation 
file.  x.d would have a statement like: 

module x;
implementation x;

That I haven't thought about ;) It might get better: module implementation x; and I think it wouldn't be too painful to parse if that's concerned
Jul 29 2004
prev sibling next sibling parent reply Ilya Minkov <minkov cs.tum.edu> writes:
David Tiktin schrieb:

 I think this is an important issue and not just for keeping source 
 code closed.  One feature I have not yet found in D is a clean 
 solution for separating interface from implementation in the way C 
 prototypes and forward declarations do.  So far, this method of 
 stripping out the implementation seems to be the *only* offered 
 solution, and it seems kludgy.

It's really nice for development to have it together. Like, there are different opinions, and i do program in C++ primarily, but i still feel much more comfortable with both implementation and declarations in one file. Nice for people coming from Delphi (like myself) or Java originally. What else can probably be done, make 2 sections within one file, the upper for (public) declarations, the lower for implementation. I don't know whether it always works, but i think it should, so it's worth a try. And where it doesn't, report a bug.
 Would it be possible to do something like this?  A module could 
 (optionally) use 2 files, say x.d and x.i.  x.d would be the 
 "stripped" interface file.  x.i would be the "full" implementation 
 file.  x.d would have a statement like: 

Yikes! It was such a relief that D doesn't force one to do that. but... you really want it back? Walters opinion to this is, is that interface mixed with implementation is OK, but one needs some documentation tool. Nice HTMLs are far easier to read than stripped source or headers.
 module x;
 implementation x;
 
 When building the object x.obj, the compiler would "bring in" the 
 implementation and compile the "combined" file.  When compiling any 
 other object using "import x" it would not.  It would just see the .d 
 interface file.  You'd just ship the .lib containing the .obj and .d 
 files.  Just a thought.

If you can use separate interface from implementation as above, one could expand on it. For example, by using a version statement and a simple tool to strip out the contents of it, or by chaining the preprocessor into the build process.
 BTW, do you have a tool or script of some kind that strips the 
 function bodies?  It looks to me like that might be hard to do 
 reliably without actually parsing the code to distinguish function 
 bodies from other types of blocks.

I recall dig compiler driver (look for undig at dsource) would usually be able to do the trick, but someone was working on a better utility. -eye
Jul 29 2004
next sibling parent reply David Tiktin <dtiktin nospam.totally-bogus.com> writes:
On 29 Jul 2004, Ilya Minkov <minkov cs.tum.edu> wrote:

 David Tiktin schrieb:
 
 I think this is an important issue and not just for keeping
 source code closed.  One feature I have not yet found in D is a
 clean solution for separating interface from implementation in
 the way C prototypes and forward declarations do.  So far, this
 method of stripping out the implementation seems to be the *only*
 offered solution, and it seems kludgy.

It's really nice for development to have it together. Like, there are different opinions, and i do program in C++ primarily, but i still feel much more comfortable with both implementation and declarations in one file. Nice for people coming from Delphi (like myself) or Java originally. What else can probably be done, make 2 sections within one file, the upper for (public) declarations, the lower for implementation. I don't know whether it always works, but i think it should, so it's worth a try. And where it doesn't, report a bug.

You'd still have to physically remove the lower part when shipping a library if you wanted to keep the source code closed.
 Would it be possible to do something like this?  A module could 
 (optionally) use 2 files, say x.d and x.i.  x.d would be the 
 "stripped" interface file.  x.i would be the "full"
 implementation file.  x.d would have a statement like: 


 Yikes! It was such a relief that D doesn't force one to do that.
 but... you really want it back?

I did say "optionally" :-). But yes, I guess I do want something *like* this and not just for code privacy. (Yes, Java and C# do the same thing, and have the same problem. They have spawned a code obfuscation industry so people can ship classes that can't be decompiled. Yuck!) I'm not suggesting that D require this, just that it support it. I think conceptual separation of interface and implmenatation is the first principle of good software engineering. So I think there should be language constructs which support good abstraction barriers. But I agree that forcing people to be good isn't nice :-).
 Walters opinion to this is, is that interface mixed with
 implementation is OK, but one needs some documentation tool. Nice
 HTMLs are far easier to read than stripped source or headers.

I agree with the last part, but that also introduces lots of redundancy in the code. You have the code *and* HTML comments that explain it. And we know how easy it is for the 2 to get out of synch. I think this is a really *hard* problem. No one wants the same thing in 2 places. Having the same thing in 2 places is the first principle of bad software engineering!
 module x;
 implementation x;
 
 When building the object x.obj, the compiler would "bring in" the
 implementation and compile the "combined" file.  When compiling
 any other object using "import x" it would not.  It would just
 see the .d interface file.  You'd just ship the .lib containing
 the .obj and .d files.  Just a thought.

If you can use separate interface from implementation as above, one could expand on it. For example, by using a version statement and a simple tool to strip out the contents of it, or by chaining the preprocessor into the build process.

Yes, there could be a solution along these lines. But does D allow you to "reopen" a class definition? [pseudo code] class X { this(int); } version (implementation) { class X { this(int x) { x_ = x; } private int x_; } } [/pseudo code] If it did, I suppose you could just do: [pseudo code] version (implementation) { import x.impl; } [/pseudo code] Or something like that.
 BTW, do you have a tool or script of some kind that strips the 
 function bodies?  It looks to me like that might be hard to do 
 reliably without actually parsing the code to distinguish
 function bodies from other types of blocks.

I recall dig compiler driver (look for undig at dsource) would usually be able to do the trick, but someone was working on a better utility.

Thanks, I'll look there. Dave -- D.a.v.i.d T.i.k.t.i.n t.i.k.t.i.n [at] a.d.v.a.n.c.e.d.r.e.l.a.y [dot] c.o.m
Jul 29 2004
next sibling parent Niko Korhonen <niktheblak hotmail.com> writes:
David Tiktin wrote:

 (Yes, Java and C# do the same thing, and have the same problem.
 They have spawned a code obfuscation industry so people can ship
 classes that can't be decompiled.  Yuck!)

OT: The obfuscation industry is just exploiting people's paranoia. Decompiling Bytecode/MSIL IMO isn't such a big threat. A company can just put a "reverse engineering prohibited" discalimer in the EULA and sue the hell out of anyone who might be doing so. Individuals who can't afford lawyers develop open source anyway :) Remember that you can disassemble any executable. But figuring out what the code does is harder. *Much* harder. But it's still /possible/ :)
Jul 29 2004
prev sibling parent Ilya Minkov <minkov cs.tum.edu> writes:
David Tiktin schrieb:
 Yes, there could be a solution along these lines.  But does D allow 
 you to "reopen" a class definition?

No. I think it should rather done like in C++: you make a complete class declaration in the "header" part, excluding code, and then fill in the methods below in the version(implementation) part, e.g. X.this(int x) { x_ = x; } x_ has to be defined in the public part anyway, because else the class definition would be incomplete, size unknown, and thus it would all break when you create a class derived from X.
 If it did, I suppose you could just do:

   import x.impl;

This line cannot work properly, because qualified names for classes would be different. Perhaps one could make alias fix that, but i think then it would be too seducing to place data members here, and thus break the definition. You should also be aware, that D automatically evaluates functions for inlining, if their source is available during compilation. If not, then... bad luck. D is geared towards whole-program optimizations, which might be the next big thing which might cause Java and CLI/.NET to grow stronger, and C++ to fall back a bit. This question of source protection has actually popped up a couple of times in the long history of D newsgroups, back when it wasn't that crowded and i read all the couple of thousands messages per year. It was agreed upon that when commercial compilers appear, they could foresee a format to replace lib, containing part compiled code, part declarations, part parsed representation of inlinable functions. For current development, it would be putting a priority on a wrong spot - there is way too much primary things to iron out. But an external tool would be thinkable, which would be based on a stripper anyway, plus an archive and compilation manager. Have a mercy with Walter: he cannot clone himself, people hardly help him, and he doesn't have a time machine. -eye
Jul 30 2004
prev sibling parent "C. Sauls" <ibisbasenji yahoo.com> writes:
Ilya Minkov wrote:
 I recall dig compiler driver (look for undig at dsource) would usually 
 be able to do the trick, but someone was working on a better utility.

I was at one time, is that the one you mean? I basically scrapped most of it pending some changes in both D and Mango... I guess I ought to get that thing written now. Just hadn't needed it myself, so there was no hurry. -Chris S. -Invironz
Jul 29 2004
prev sibling parent reply Andy Friesen <andy ikagames.com> writes:
David Tiktin wrote:

 Would it be possible to do something like this?  A module could 
 (optionally) use 2 files, say x.d and x.i.  x.d would be the 
 "stripped" interface file.  x.i would be the "full" implementation 
 file.  x.d would have a statement like: 
 
 module x;
 implementation x;
 
 When building the object x.obj, the compiler would "bring in" the 
 implementation and compile the "combined" file.  When compiling any 
 other object using "import x" it would not.  It would just see the .d 
 interface file.  You'd just ship the .lib containing the .obj and .d 
 files.  Just a thought.

A simpler way would be to allow a class to be redeclared in a different source file within the same compilation phase iff there are no conflicting definitions. Then we could do foo.d module foo; class Foo : FooBase { this(); void do_things(); } foo.impl.d (or whatever you want to name it) module foo; // foo, not foo_impl! Currently not legal. class Foo : FooBase { this() { ... } // implemented this time void do_things() { ... } } All the compiler really needs to do is make sure that any conflicting definitions are identical, save for omitted function bodies. The linker can already catch any duplicated or missing definitions. With respect to the filename not matching the module name, there are two solutions that come to mind: silently ignore it, or make the ".impl.d" extension (or something like it) explicitly part of the language. Silently ignoring it is probably much easier: code declared in foo.impl.d is compiled as though it was part of module foo. (name mangling and so forth) Modules importing foo will get the stub instead, as per existing rules, which should be okay in most situations. Or we could just write a good stripping tool and stay free from manual duplication that way. :) -- andy
Jul 29 2004
parent David Tiktin <dtiktin nospam.totally-bogus.com> writes:
On 29 Jul 2004, Andy Friesen <andy ikagames.com> wrote:

 Or we could just write a good stripping tool and stay free from
 manual duplication that way. :)

That could work as long as the stripping tool is a standard part of the D toolchain. After all, you really want to use it only when the code is done (the interface is stable) and you're building the library for distribution. It could be part of the release build process to *optionally* generate the "stripped" source files. Dave -- D.a.v.i.d T.i.k.t.i.n t.i.k.t.i.n [at] a.d.v.a.n.c.e.d.r.e.l.a.y [dot] c.o.m
Jul 29 2004
prev sibling parent reply Niko Korhonen <niktheblak hotmail.com> writes:
Thanks for the responses everyone! I already suspected that I had to 
provide some sort of stripped-down (let's not call them headers) 
versions of the source code.

Ok, this works well enough. Technically I'm against having /two/ sets of 
source files since they are bound to drift apart as time goes by.

How nightmarish C/C++ headers may be, they still work well enough since 
the compiler processes them too, hopefully emitting a warning if the 
interface and implementation differ too much. But if I'm keeping two 
sets of D source, I develop actively the other set and pretty much 
ignore the "header" (Ok, now I called it a header) set. Since the 
compiler doesn't check these "headers" while compiling the library, 
there's probably going to be some nasty surprises sooner or later.

The Java/.NET way of using only one set of source files is IMO 
definitely the way to go. But I guess what is done with Java/.NET 
(distribute only .jar or .dll files, no headers or source code) is 
impossible in D, since Bytecode/MSIL contains all the information the 
source code does. The binary code that C/C++/D compiler emits doesn't.
Jul 29 2004
next sibling parent reply h3r3tic <h3r3tic dev.null> writes:
Though about it for a longer moment and the best option now really seems 
to have a good tool that would generate the headers. That tool should, 
being open-source and crossplatform would be used in every IDE so that 
one would only click on an icon 'generate headers' or sth like that and 
immediately have them with no pain at all. I think this wouldnt hurt anybody
Jul 29 2004
parent reply parabolis <parabolis softhome.net> writes:
h3r3tic wrote:

 Though about it for a longer moment and the best option now really seems 
 to have a good tool that would generate the headers. That tool should, 
 being open-source and crossplatform would be used in every IDE so that 
 one would only click on an icon 'generate headers' or sth like that and 
 immediately have them with no pain at all. I think this wouldnt hurt 
 anybody

It would be more universal (and ergo more helpful) if it were actually a compiler option...
Jul 29 2004
next sibling parent reply stonecobra <scott stonecobra.com> writes:
parabolis wrote:

 h3r3tic wrote:
 
 Though about it for a longer moment and the best option now really 
 seems to have a good tool that would generate the headers. That tool 
 should, being open-source and crossplatform would be used in every IDE 
 so that one would only click on an icon 'generate headers' or sth like 
 that and immediately have them with no pain at all. I think this 
 wouldnt hurt anybody

It would be more universal (and ergo more helpful) if it were actually a compiler option...

the front-end source is available, no? Perhaps you could code it up and submit a patch to Walter. Then it would be a compiler option Scott
Jul 29 2004
parent parabolis <parabolis softhome.net> writes:
stonecobra wrote:
 parabolis wrote:
 
 h3r3tic wrote:

 Though about it for a longer moment and the best option now really 
 seems to have a good tool that would generate the headers. That tool 
 should, being open-source and crossplatform would be used in every 
 IDE so that one would only click on an icon 'generate headers' or sth 
 like that and immediately have them with no pain at all. I think this 
 wouldnt hurt anybody

It would be more universal (and ergo more helpful) if it were actually a compiler option...

the front-end source is available, no? Perhaps you could code it up and submit a patch to Walter. Then it would be a compiler option

That is a good point... Although I was just thinking over Martin M. Pedersen's post in which he suggested shoving it all into an .obj file which would happen in the optlink...
Jul 29 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ceccnk$21uc$1 digitaldaemon.com>, parabolis says...

It would be more universal (and ergo more helpful) if it were 
actually a compiler option...

<rant> It would be "helpful" only for those people wishing to hide their source code from the rest of us. Make the closed source brigade do their own dirty work, I say. If they're going to be charging us money for their closed source software they can damn well pay for their own stripping tool. </rant> Jill
Jul 30 2004
next sibling parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cectuh$2aa6$1 digitaldaemon.com...
 In article <ceccnk$21uc$1 digitaldaemon.com>, parabolis says...

It would be more universal (and ergo more helpful) if it were
actually a compiler option...

<rant> It would be "helpful" only for those people wishing to hide their source code from the rest of us. Make the closed source brigade do their own dirty work, I say. If they're going to be charging us money for their closed source software they can damn well pay for their own stripping tool. </rant>

I trust that was wind-up? It most certainly should be an option in the compiler. Absence of such is not going to persuade people to release their IP, it will merely dissuade them from using D.
Jul 30 2004
prev sibling parent reply Niko Korhonen <niktheblak hotmail.com> writes:
Arcane Jill wrote:
 It would be "helpful" only for those people wishing to hide their source code
 from the rest of us.

I'm not asking "how can I close my sources" just out of spite. In fact, I /am/ a fan of open source, and all of my personal projects are BSD-licensed anyway. But the fact is that not all software development is done by open source -oriented individuals. Only a small minority of it is. It's a common business practice not to divulge your intellectual property. I asked the question strictly with business considerations of the D language in mind.
Jul 30 2004
next sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Niko Korhonen" <niktheblak hotmail.com> wrote in message
news:cedfu1$2ifg$1 digitaldaemon.com...
 Arcane Jill wrote:
 It would be "helpful" only for those people wishing to hide their source code
 from the rest of us.

I'm not asking "how can I close my sources" just out of spite. In fact, I /am/ a fan of open source, and all of my personal projects are BSD-licensed anyway. But the fact is that not all software development is done by open source -oriented individuals. Only a small minority of it is. It's a common business practice not to divulge your intellectual property. I asked the question strictly with business considerations of the D language in mind.

Quite right. The idea that all software must be open-source is indicative of a peurile notion amongst many NG-hangers, i.e. that everything in the IT world should be free. I've never had a paying client for whom an open-source model could ever conceivably be appropriate. To not recognise this, or to not care, is simply stupid, and demonstrates an attitude that is not commensurate with the long term aims of D (and of Walter). I completely agree that the DMD distribution should include tools to assist with closed-source development, and urge Walter (or anyone with a bit of spare time) to consider how we can get this with 1.0.
Jul 30 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
In article <cedhr2$2jjk$2 digitaldaemon.com>, Matthew says...
Quite right. The idea that all software must be open-source is indicative of a
peurile notion amongst many NG-hangers,
i.e. that everything in the IT world should be free. I've never had a paying
client for whom an open-source model could
ever conceivably be appropriate. To not recognise this, or to not care, is
simply stupid, and demonstrates an attitude
that is not commensurate with the long term aims of D (and of Walter).

I completely agree that the DMD distribution should include tools to assist
with closed-source development, and urge
Walter (or anyone with a bit of spare time) to consider how we can get this
with 1.0.

Agreed. But to the open source folks I will say that I'm quite willing to pay extra for libraries that ship with full source code, so long as the license doesn't require me to give credit in my docs or any other such nonsense. Sean
Jul 30 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cedhr2$2jjk$2 digitaldaemon.com>, Matthew says...
The idea that all software must be open-source is indicative of a peurile
notion amongst many NG-hangers,
i.e. that everything in the IT world should be free. 

Quite right. Why? Did someone on this newsgroup claim that all software must be open source? Or that everything in the IT world should be free? I must have missed that post. I don't think it unreasonable though to suggest that if someone is likely to gain commercially from the use of a tool, that they shouldn't necessarily expect that tool to be handed to them on a plate free of charge. But that's just an opinion, and it does seems a shame that one can't express such things on this forum without incurring overreaction. Jill PS. Come on, lighten up, Matthew. Nothing enclosed within <rant> and </rant> should ever be taken too seriously. Just imagine like I'm complaining about the price of fish or the state of the bus service or something, and take it on that level. I thought Niko's response to my post was perfectly reasonable. Why spoil it? Let's keep this place fun.
Jul 30 2004
parent "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cedvfk$2rfq$1 digitaldaemon.com...
 In article <cedhr2$2jjk$2 digitaldaemon.com>, Matthew says...
The idea that all software must be open-source is indicative of a peurile
notion amongst many NG-hangers,
i.e. that everything in the IT world should be free.

Quite right. Why? Did someone on this newsgroup claim that all software must be open source? Or that everything in the IT world should be free? I must have missed that post. I don't think it unreasonable though to suggest that if someone is likely to gain commercially from the use of a tool, that they shouldn't necessarily expect that tool to be handed to them on a plate free of charge. But that's just an opinion, and it does seems a shame that one can't express such things on this forum without incurring overreaction. Jill PS. Come on, lighten up, Matthew. Nothing enclosed within <rant> and </rant> should ever be taken too seriously. Just imagine like I'm complaining about the price of fish or the state of the bus service or something, and take it on that level. I thought Niko's response to my post was perfectly reasonable. Why spoil it? Let's keep this place fun.

I am light. I just like carping on from time to time. (Which I know you know by now.)
Jul 30 2004
prev sibling parent Helmut Leitner <helmut.leitner wikiservice.at> writes:
Niko Korhonen wrote:
 
 Arcane Jill wrote:
 It would be "helpful" only for those people wishing to hide their source code
 from the rest of us.

I'm not asking "how can I close my sources" just out of spite. In fact, I /am/ a fan of open source, and all of my personal projects are BSD-licensed anyway. But the fact is that not all software development is done by open source -oriented individuals. Only a small minority of it is. It's a common business practice not to divulge your intellectual property. I asked the question strictly with business considerations of the D language in mind.

I remember that Burton Radons built a solution for his libraries and distributed them this way. -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 31 2004
prev sibling next sibling parent reply Charlie <Charlie_member pathlink.com> writes:
Ive gotten the DMD front-end to compile ( super cool design i might add!) , and
am working out the details of the symbol table ( i dont have any experience with
this sort of thing ) , but it actually shouldnt be too hard to write a stripper
tool given the front end.  In fact its doable right now, I plan to work on this
when I get back from vacation ( pop quiz, what state is the self proclaimed
Sportsmans Paradise  :) ).

Charlie

In article <cebo1r$1p37$1 digitaldaemon.com>, Niko Korhonen says...
Thanks for the responses everyone! I already suspected that I had to 
provide some sort of stripped-down (let's not call them headers) 
versions of the source code.

Ok, this works well enough. Technically I'm against having /two/ sets of 
source files since they are bound to drift apart as time goes by.

How nightmarish C/C++ headers may be, they still work well enough since 
the compiler processes them too, hopefully emitting a warning if the 
interface and implementation differ too much. But if I'm keeping two 
sets of D source, I develop actively the other set and pretty much 
ignore the "header" (Ok, now I called it a header) set. Since the 
compiler doesn't check these "headers" while compiling the library, 
there's probably going to be some nasty surprises sooner or later.

The Java/.NET way of using only one set of source files is IMO 
definitely the way to go. But I guess what is done with Java/.NET 
(distribute only .jar or .dll files, no headers or source code) is 
impossible in D, since Bytecode/MSIL contains all the information the 
source code does. The binary code that C/C++/D compiler emits doesn't.

Jul 29 2004
parent J C Calvarese <jcc7 cox.net> writes:
Charlie wrote:
 Ive gotten the DMD front-end to compile ( super cool design i might add!) , and

Great!
 am working out the details of the symbol table ( i dont have any experience
with
 this sort of thing ) , but it actually shouldnt be too hard to write a stripper
 tool given the front end.  In fact its doable right now, I plan to work on this

For what it's worth, dig's strip.d isn't particularly long, but it's filled with "pointer magic" that hurts my brain to look at it. It's probably easy to understand for someone experienced in C++, but it doesn't seem like a very D way of doing it.
 when I get back from vacation ( pop quiz, what state is the self proclaimed
 Sportsmans Paradise  :) ).

Have fun!
 
 Charlie
 
 In article <cebo1r$1p37$1 digitaldaemon.com>, Niko Korhonen says...
 
Thanks for the responses everyone! I already suspected that I had to 
provide some sort of stripped-down (let's not call them headers) 
versions of the source code.

Ok, this works well enough. Technically I'm against having /two/ sets of 
source files since they are bound to drift apart as time goes by.

How nightmarish C/C++ headers may be, they still work well enough since 
the compiler processes them too, hopefully emitting a warning if the 
interface and implementation differ too much. But if I'm keeping two 
sets of D source, I develop actively the other set and pretty much 
ignore the "header" (Ok, now I called it a header) set. Since the 
compiler doesn't check these "headers" while compiling the library, 
there's probably going to be some nasty surprises sooner or later.

The Java/.NET way of using only one set of source files is IMO 
definitely the way to go. But I guess what is done with Java/.NET 
(distribute only .jar or .dll files, no headers or source code) is 
impossible in D, since Bytecode/MSIL contains all the information the 
source code does. The binary code that C/C++/D compiler emits doesn't.


-- Justin (a/k/a jcc7) http://jcc_7.tripod.com/d/
Jul 29 2004
prev sibling next sibling parent David L. Davis <SpottedTiger yahoo.com> writes:
In article <cebo1r$1p37$1 digitaldaemon.com>, Niko Korhonen says...
...

The Java/.NET way of using only one set of source files is IMO 
definitely the way to go. But I guess what is done with Java/.NET 
(distribute only .jar or .dll files, no headers or source code) is 
impossible in D, since Bytecode/MSIL contains all the information the 
source code does. The binary code that C/C++/D compiler emits doesn't.

Niko Korhonen : If you're wanting to place Free Functions with default parameters (I haven't figured out the exporting Classes to a DLL and then using them with a D app yet) into a DLL without the need of exposing your D code. I have a project called the "Financial Functions Package" on my site, that creates a Windows' DLL in D that's then explicitly loaded into a D Application. Please download and check out the source code in financial_dll.d and loadfindll.d (the loader / D App) from the project at the link below. It's always good to have more options. :)) http://spottedtiger.tripod.com/D_Language/D_Financial_Package_XP.html Also I've written an article (1st part - the 2nd part will be about the loader) of how I created the D DLL and some of the pitfalls here: http://spottedtiger.tripod.com/D_Language/D_Demystifying_D_DLLs1_XP.html Hope this helps a little. David L. ------------------------------------------------------------------- "Dare to reach for the Stars...Dare to Dream, Build, and Achieve!"
Jul 29 2004
prev sibling next sibling parent reply "Martin M. Pedersen" <martin moeller-pedersen.dk> writes:
"Niko Korhonen" <niktheblak hotmail.com> skrev i en meddelelse
news:cebo1r$1p37$1 digitaldaemon.com...
 The Java/.NET way of using only one set of source files is IMO
 definitely the way to go.

I agree, wholeheartedly.
 But I guess what is done with Java/.NET
 (distribute only .jar or .dll files, no headers or source code) is
 impossible in D, since Bytecode/MSIL contains all the information the
 source code does. The binary code that C/C++/D compiler emits doesn't.

Because the binary code that the D compiler emits now, does not include all that information, it does not mean that it could not be made to do it. I cannot imagine that there is not a way to put that information into object files and/or libraries. It could be as simple a producing the stripped down version of the input source file and put it into the object files as some predefined symbol or stuff it into a comment field. No new file formats needs to be invented - only a built-in "stripper" in the compiler and the ability to parse a source embedded in an object file. Regards, Martin M. Pedersen
Jul 29 2004
parent reply parabolis <parabolis softhome.net> writes:
Martin M. Pedersen wrote:

 Because the binary code that the D compiler emits now, does not include all
 that information, it does not mean that it could not be made to do it. I
 cannot imagine that there is not a way to put that information into object
 files and/or libraries. It could be as simple a producing the stripped down
 version of the input source file and put it into the object files as some
 predefined symbol or stuff it into a comment field. No new file formats
 needs to be invented - only a built-in "stripper" in the compiler and the
 ability to parse a source embedded in an object file.

This is actually an intriguing idea. (Besides the fact that humans would not be able to read the stripped down files.) The OMF format (which is what dm uses) does have a comment field that might be repurposed to do just that. (COMMENT type 0xDA should work)
Jul 29 2004
parent reply "Martin M. Pedersen" <martin moeller-pedersen.dk> writes:
"parabolis" <parabolis softhome.net> skrev i en meddelelse
news:cecegg$22k5$1 digitaldaemon.com...
 Martin M. Pedersen wrote:
 This is actually an intriguing idea. (Besides the fact that
 humans would not be able to read the stripped down files.)

It would require only a very simple tool to extract the interfaces. I think it is desirable to keep the interface with compiled code. If they are separate as in C, it opens up for inconsistencies - something that I'm sure every C/C++ programmer here has learned to hate. Many years ago I was working with Turbo Pascal, and it's TPU-files was the combination of the compiled code and the interface. It worked very well, and also allowed the compiler to do lazy recompilation minimizing the need for tools such as "make". I don't know how Delphi works today. I think this approach would be very similar to Turbo Pascal's TPU-files, except that they are compatible with a C development environment, and we have the complexities of libraries to consider too.
 OMF format (which is what dm uses) does have a comment field
 that might be repurposed to do just that. (COMMENT type 0xDA
 should work)

I thought so :-) Regards, Martin M. Pedersen
Jul 30 2004
parent parabolis <parabolis softhome.net> writes:
Martin M. Pedersen wrote:

 "parabolis" <parabolis softhome.net> skrev i en meddelelse
 news:cecegg$22k5$1 digitaldaemon.com...
 
Martin M. Pedersen wrote:
This is actually an intriguing idea. (Besides the fact that
humans would not be able to read the stripped down files.)

It would require only a very simple tool to extract the interfaces. I think it is desirable to keep the interface with compiled code. If they are separate as in C, it opens up for inconsistencies - something that I'm sure every C/C++ programmer here has learned to hate. Many years ago I was working with Turbo Pascal, and it's TPU-files was the combination of the compiled code and the interface. It worked very well, and also allowed the compiler to do lazy recompilation minimizing the need for tools such as "make". I don't know how Delphi works today. I think this approach would be very similar to Turbo Pascal's TPU-files, except that they are compatible with a C development environment, and we have the complexities of libraries to consider too.

Actually I saw someone suggest .dar files which sound like they could do what you want them to. In Java .jar files are .zip files which (usually compression level of 0). The internals are treated in a manner similar to /lib/... So the suggestion allows for: /lib/... obj files here /inc/... auto created stripped down/include files here /src/... full source file here if you want /kitchen-sink/... self explanatory Provided the compiler can read/write directly to .dar files then it can also automatically save the stripped down .d file in the appropriate /inc/ directory. Then D would also need a seperate .dar management tool but given zlib capabilities already exist the whole idea is really probably quite simply realizable. You will thus be able to distribute your whole package with full or stripped down source along with any necc. .obj files and with decent compression.
Jul 30 2004
prev sibling parent reply J C Calvarese <jcc7 cox.net> writes:
Niko Korhonen wrote:
 Thanks for the responses everyone! I already suspected that I had to 
 provide some sort of stripped-down (let's not call them headers) 
 versions of the source code.
 
 Ok, this works well enough. Technically I'm against having /two/ sets of 
 source files since they are bound to drift apart as time goes by.
 
 How nightmarish C/C++ headers may be, they still work well enough since 
 the compiler processes them too, hopefully emitting a warning if the 
 interface and implementation differ too much. But if I'm keeping two 
 sets of D source, I develop actively the other set and pretty much 
 ignore the "header" (Ok, now I called it a header) set. Since the 
 compiler doesn't check these "headers" while compiling the library, 
 there's probably going to be some nasty surprises sooner or later.
 
 The Java/.NET way of using only one set of source files is IMO 
 definitely the way to go. But I guess what is done with Java/.NET 
 (distribute only .jar or .dll files, no headers or source code) is 
 impossible in D, since Bytecode/MSIL contains all the information the 
 source code does. The binary code that C/C++/D compiler emits doesn't.

If this is really important to you, I don't know why we (and by we and mean Walter) couldn't implement a .dar file format. It could be .zip-format file containing stripped headers and a corresponding .lib. This is definitely post-1.0 talk, but I don't think it'd have to be too hard. We'd want to get the compiler to write .dar files (as well as read them). I suppose if someone was really industrious that could create a tool to do all of this now as a build system that calls dmd.exe and lib.exe. (And, no, I'm not that person. I'm busy doing other stuff...) -- Justin (a/k/a jcc7) http://jcc_7.tripod.com/d/
Jul 29 2004
parent reply Niko Korhonen <niktheblak hotmail.com> writes:
J C Calvarese wrote:
 If this is really important to you, I don't know why we (and by we and 
 mean Walter) couldn't implement a .dar file format. It could be 
 .zip-format file containing stripped headers and a corresponding .lib. 

While I can't say that doesn't sound tempting, I'll say no. It's not really /that/ important. It doesn't apply to me or my personal developing methodologies very much, what I asked was more of a theoretical question for (future) business aspects of D. I asked whether distributing closed-source libraries for D is possible, and I got my answer. I'm satisfied. It's currently about as easy as in C, I just have to write additional "header" files. If I'm targeting a widely-usable C library, I going to have to use extern(C) and write the C headers anyway. What you suggested or what Martin M. Pedersen said about putting the necessary information into .obj/.lib files /would/ be a relief though, and it would push D a notch into the Java/.NET realm. Whether this is desirable or not is probably a subject of debate. But no, as of yet this isn't necessary. If a required number of people start asking about this, *then* you could possibly take it into consideration :)
Jul 30 2004
parent J C Calvarese <jcc7 cox.net> writes:
In article <cedhpe$2jja$1 digitaldaemon.com>, Niko Korhonen says...
J C Calvarese wrote:
 If this is really important to you, I don't know why we (and by we and 
 mean Walter) couldn't implement a .dar file format. It could be 
 .zip-format file containing stripped headers and a corresponding .lib. 

While I can't say that doesn't sound tempting, I'll say no. It's not really /that/ important. It doesn't apply to me or my personal developing methodologies very much, what I asked was more of a theoretical question for (future) business aspects of D.

Fair enough. I wasn't trying to guilt anyone into doing anything. I was just hinting that since D is pretty easy to parse, the DMD front end source is public, and open source archiving code is out there (such as std.zip), this is probably something that someone could develop -- even without Walter's help. (And Walter has shown that he'll accept donated code.) This particular topic has come up before (not recently, but several times). I guess I've just read too many "Oh, D really needs this" messages in the past. D is designed to allow both open and closed source projects, so that everyone can be happy. And allowing stripped headers is a part of that idea. jcc7
Jul 30 2004