www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - dub: how to reference a compiled package

reply mahdi <xemailpro yahoo.co.uk> writes:
Hi,

Suppose I have a package `mypack` in `~/mypack`. I run `dub` 
command on this package and have the compiled `mypack` file (OS 
is Linux).

Now I am working on my project. I know how to use the source-code 
of `mypack` package in the project but what if I only have the 
compiled binary? How can I reference and use the modules of the 
compiled `mypack`?

(I looked into the DUB homepage and it's Getting Started page but 
could not find anything).
Feb 25 2016
next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 25 Feb 2016 12:15:42 +0000, mahdi wrote:

 Hi,
 
 Suppose I have a package `mypack` in `~/mypack`. I run `dub` command on
 this package and have the compiled `mypack` file (OS is Linux).
 
 Now I am working on my project. I know how to use the source-code of
 `mypack` package in the project but what if I only have the compiled
 binary? How can I reference and use the modules of the compiled
 `mypack`?
 
 (I looked into the DUB homepage and it's Getting Started page but could
 not find anything).
First thing, you need D interface files. See: https://dlang.org/dmd-linux.html#interface-files Note that D interface files still require you to include the full body of anything in a template, so you might have to modify your code somewhat for improved secrecy. Copy the *.di files and binaries into another dub project and add a postBuildCommand to copy that binary to the output location. I think that will work. dub might complain that it doesn't have any source files, in which case you'll have to provide at least one D file for it to compile, but that can be empty.
Feb 25 2016
parent reply mahdi <xemailpro yahoo.co.uk> writes:
On Thursday, 25 February 2016 at 16:45:46 UTC, Chris Wright wrote:
 On Thu, 25 Feb 2016 12:15:42 +0000, mahdi wrote:

 Hi,
 
 Suppose I have a package `mypack` in `~/mypack`. I run `dub` 
 command on this package and have the compiled `mypack` file 
 (OS is Linux).
 
 Now I am working on my project. I know how to use the 
 source-code of `mypack` package in the project but what if I 
 only have the compiled binary? How can I reference and use the 
 modules of the compiled `mypack`?
 
 (I looked into the DUB homepage and it's Getting Started page 
 but could
 not find anything).
First thing, you need D interface files. See: https://dlang.org/dmd-linux.html#interface-files Note that D interface files still require you to include the full body of anything in a template, so you might have to modify your code somewhat for improved secrecy. Copy the *.di files and binaries into another dub project and add a postBuildCommand to copy that binary to the output location. I think that will work. dub might complain that it doesn't have any source files, in which case you'll have to provide at least one D file for it to compile, but that can be empty.
Thanks. Is there a way to use a D library without having access to it's source code? I tried `dmd -lib abcd.d` which creates a static library. But still I need to specify path to library's source files using -I option when compiling the code that uses that library. So if we have just access to the library file, is it possible to use it in the code?
Feb 25 2016
parent reply Mike Parker <aldacron gmail.com> writes:
On Thursday, 25 February 2016 at 21:06:59 UTC, mahdi wrote:
 On Thursday, 25 February 2016 at 16:45:46 UTC, Chris Wright
 Thanks. Is there a way to use a D library without having access 
 to it's source code? I tried `dmd -lib abcd.d` which creates a 
 static library. But still I need to specify path to library's 
 source files using -I option when compiling the code that uses 
 that library.

 So if we have just access to the library file, is it possible 
 to use it in the code?
The compiler needs to know what symbols are available from any imports you use in your source. .di files exist to allow closed source projects to be distributed as binary. They are analagous to C or C++ header files. You could create them by hand like so: // foo.d struct S { int x, y; } void addTwo(S s) { s.x += 2; s.y += 2; } // foo.di struct S { int x, y; } void addTwo(S s); The compiler needs to know about S and its types, and it needs to know the signature of addTwo. The .di file allows you to provide that while keeping the implementation of addTwo closed. When foo is imported in client code, the compiler will find foo.di and use that instead of foo.d. However, the compiler must have the source for templates, as they are instantiated when they are used, not when the library is compiled. The same is true for any functions you want inlined. In the example above, addTwo can only be inlined when foo.d is used, since the compiler will not have the implementation with foo.di.
Feb 25 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 26 February 2016 at 02:49:20 UTC, Mike Parker wrote:

 The compiler needs to know about S and its types, and it needs
S and its *members*
Feb 25 2016
prev sibling parent reply mahdi <xemailpro yahoo.co.uk> writes:
On Friday, 26 February 2016 at 02:49:20 UTC, Mike Parker wrote:
 On Thursday, 25 February 2016 at 21:06:59 UTC, mahdi wrote:
 On Thursday, 25 February 2016 at 16:45:46 UTC, Chris Wright
 Thanks. Is there a way to use a D library without having 
 access to it's source code? I tried `dmd -lib abcd.d` which 
 creates a static library. But still I need to specify path to 
 library's source files using -I option when compiling the code 
 that uses that library.

 So if we have just access to the library file, is it possible 
 to use it in the code?
The compiler needs to know what symbols are available from any imports you use in your source. .di files exist to allow closed source projects to be distributed as binary. They are analagous to C or C++ header files. You could create them by hand like so: // foo.d struct S { int x, y; } void addTwo(S s) { s.x += 2; s.y += 2; } // foo.di struct S { int x, y; } void addTwo(S s); The compiler needs to know about S and its types, and it needs to know the signature of addTwo. The .di file allows you to provide that while keeping the implementation of addTwo closed. When foo is imported in client code, the compiler will find foo.di and use that instead of foo.d. However, the compiler must have the source for templates, as they are instantiated when they are used, not when the library is compiled. The same is true for any functions you want inlined. In the example above, addTwo can only be inlined when foo.d is used, since the compiler will not have the implementation with foo.di.
Great! Thanks. I was looking for a feature like `jar` files in Java or `assemblies` in C# where all compiled code and metadata/symbols are stored together inside a single binary file. I think same can be implemented for D language and it won't break any code because it is not touching the language itself, but the compiler. Anyway, thanks.
Feb 25 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 26 February 2016 at 03:19:26 UTC, mahdi wrote:

 Great! Thanks.

 I was looking for a feature like `jar` files in Java or 
 `assemblies` in C# where all compiled code and metadata/symbols 
 are stored together inside a single binary file.

 I think same can be implemented for D language and it won't 
 break any code because it is not touching the language itself, 
 but the compiler.
The magic in Java isn't from Jar files, but from the fact that class binaries are distributed as byte code. All of the information from the source is still there. The compiler can learn everything it needs to know about imported classes simply by reading the byte code instead of the source. In the model that D uses, I don't know enough about object file formats to know how much information would still be available if they were used at compile time, but I'm fairly certain it wouldn't allow the use of templates. There was a project several years ago that allowed loading D classes dynamically from object files, but I suspect if it were practical to use object files at compile time in the same way Java class files are used, someone would have implemented such a thing already (in C and C++ as well). That said, it might be an interesting enhancement for the compiler to support reading source or interface files from zip archives (which is I know is what you're getting at), allowing the required imports and the binary to be located in one place so that only a single path need be passed on the command line to find everything.
Feb 25 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 26 Feb 2016 03:19:26 +0000, mahdi wrote:

 Great! Thanks.
 
 I was looking for a feature like `jar` files in Java or `assemblies` in
 C# where all compiled code and metadata/symbols are stored together
 inside a single binary file.
C# and Java provide their own linkers and specify their own object formats. One aspect of that was including enough metadata to reconstruct the source code from the binary in a convenient and performant manner. DMD could store an extra symbol in each file it output including the autogenerated .di file for each source file. However, that would require the frontend to be able to parse arbitrary library and object formats. It's much easier to solve this in an external tool like Dub.
Feb 26 2016
prev sibling parent reply BBasile <b2.temp gmx.com> writes:
On Thursday, 25 February 2016 at 12:15:42 UTC, mahdi wrote:
 Hi,

 Suppose I have a package `mypack` in `~/mypack`. I run `dub` 
 command on this package and have the compiled `mypack` file (OS 
 is Linux).

 Now I am working on my project. I know how to use the 
 source-code of `mypack` package in the project but what if I 
 only have the compiled binary? How can I reference and use the 
 modules of the compiled `mypack`?

 (I looked into the DUB homepage and it's Getting Started page 
 but could not find anything).
As you've been told previous you need a D interface file. But additionally: The D interface file must be specified to DUB using "sourceFiles" : ["folder/interface.di"], either in a config or in the globals. The binary, so either a .lib | .a or .obj | .o must be specified to DUB using "DFlags" : ["folder/binary.a"], Here again also accepted in a config or the globals. This is because DUB doesn't consider such binary as source file but DMD or LDMD2 will accept them as source in the command line. e.g the DMD equivalent for the two previous example is DMD "sourceThis.d" "folder/interface.di" "folder/binary.a" -ofbin/thesoft
Feb 25 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Friday, 26 February 2016 at 04:03:15 UTC, BBasile wrote:

 The D interface file must be specified to DUB using

 "sourceFiles" : ["folder/interface.di"],

 either in a config or in the globals.

 The binary, so either a .lib | .a or .obj | .o must be 
 specified to DUB using

 "DFlags" : ["folder/binary.a"],

 Here again also accepted in a config or the globals.

 This is because DUB doesn't consider such binary as source file 
 but DMD or LDMD2 will accept them as source in the command line.

 e.g the DMD equivalent for the two previous example is

 DMD "sourceThis.d" "folder/interface.di" "folder/binary.a" 
 -ofbin/thesoft
Why? As long as the interface file is on the import path, it shouldn't need to be passed along to DMD. When you import foo.bar, the compiler will look for both foo/bar.di and foo/bar.d.
Feb 25 2016
prev sibling parent asdf <a b.c> writes:
On Friday, 26 February 2016 at 04:03:15 UTC, BBasile wrote:
 e.g the DMD equivalent for the two previous example is

 DMD "sourceThis.d" "folder/interface.di" "folder/binary.a" 
 -ofbin/thesoft
You can mix unlinked binaries and text-editor source files on commandline? Didn't know that when I tried this. http://imgur.com/AIgZGmW I ought to try these interface *.di files. Just tried `ldc2 -H <dlang-file>` but it wouldn't do a C file also.
Feb 25 2016