www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Modules ... "import" vs. "compilation" ... what is the real process

reply james.p.leblanc <james.p.leblanc gmail.com> writes:
Dear D-ers,

I have trouble understanding "module imports" vs. "module 
compilations".

For example, in the dlang.org/tour, we have:

**"The import statement makes all public functions and types from 
the given module available."**

And from the dlang.org/spec we have:

**"Modules are always compiled at global scope and are unaffected 
by surrounding attributes or other modifiers."**

Finally, there have been discussions about allowing new ways of 
"compiling a module" by including its name on the command line.  
For example this from 2017:

https://forum.dlang.org/post/tcrdpvqvwxffnewzohuj forum.dlang.org

The more I look into this, the more that I realize that I do not 
understand this as well as I had hoped.

So, when we specify the module name on the command line, then it 
gets compiled along with the other files on the command line 
(seems reasonable).

But, if it is NOT a command line argument, then when does it get 
compiled??
(I believe that at first cut, only the public functions and types 
are "grabbed").

But, when and how are the subsequent and necessary module 
compilation (and linking) steps performed?

Finally, we use a "-I" for telling dmd to look for imports in a 
directory, but why only
"look for imports", and not "grab anything from there that is 
used, AND compile them"?

(This question is prompted by some recent desires to use ldc and 
"fastmath"... which, if I
understand correctly, will require me to include all of the 
related modules on the command
line for immediate compilation).  But, in a broader sense, I need 
to understand the related issues better.

Thank for all responses, as well as continued patience with my 
questions.

Best Regards,
James
Sep 27 2021
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 27 September 2021 at 17:38:29 UTC, james.p.leblanc 
wrote:

 Dear D-ers,

 I have trouble understanding "module imports" vs. "module 
 compilations".


A module is implemented in a source file. Though we often use the 
term "module" to refer to both, it may help to think in terms of 
importing modules and compiling source files.

Given the source files `A.d` and `B.d`, which implement the 
modules A and B respectively, and given that module A uses 
symbols from module B, then we can say the following:

1. When the compiler is compiling `A.d`, it must be aware of 
which symbols from module B are accessible from module A. This is 
what imports are for and has no relation to the compilation 
status of `B.d`.
2. The compiler will produce a `A.o/obj` object file that it will 
then pass to the linker,  including references to the symbols in 
module B. At that point, the linker will also need an object file 
from a compiled `B.d` in order to fix up the symbol references 
and produce the final executable.


`B.d` or a D interface file, `B.di`, in order to know which 
symbols are available to module A. There are a couple of ways 
this can happen:

```
dmd A.d B.d
```

Here, when the compiler encounters `import B` in `A.d`, it will 
recognize that `B.d` has been passed on the command line. If 
`B.d` has no module statement, then the file name `B` is used as 
the module name. If it has a module statement, it the file can be 
named anything when it's passed on the command line like this. It 
could be `foo.d`, but as long as it has a `module B` at the top, 
then `A.d` can `import B`.

```
dmd -c A.d
```

Here, when the compiler encounters `import B` in `A.d`, it will 
see that no `module B` declaration has been encountered in any 
other files on the command line, so it will search for `B.di` 
and, if it's not found, `B.d` on the import path (to which we can 
append directories with `-I`). I've included `-c` here, which 
will just compile `A.d` and not attempt to link it, because 
without it the linker will spew errors for every missing symbol 
from module B.

This is how D supports separate compilation. Assuming object 
files with the `.obj` extension on Windows, you could do this:

```
dmd -c B.d
dmd A.d B.obj
```

Now, the compiler uses the source of `B.d` to assess symbol 
accessibility as before, and it will pass both `A.obj` and 
`B.obj` to the linker to produce the executable.

Or you could compile `B.d` into `B.lib` and pass that on the 
command line as well.


 Finally, there have been discussions about allowing new ways of 
 "compiling a module" by including its name on the command line. 
  For example this from 2017:

 https://forum.dlang.org/post/tcrdpvqvwxffnewzohuj forum.dlang.org


This is what resulted in the `-i` compiler switch. Modifying the 
example above:

```
dmd -i A.d
```

Now, when the compiler encounters `import B` in `A.d`, if there 
is no `B.di` and it finds `B.d`, it will compile `B.d` alongside 
`A.d`, just as if the command line had been `dmd A.d B.d`.

Does that help?
Sep 27 2021
parent james.p.leblanc <james.p.leblanc gmail.com> writes:
On Tuesday, 28 September 2021 at 02:05:43 UTC, Mike Parker wrote:

 On Monday, 27 September 2021 at 17:38:29 UTC, james.p.leblanc
mpilations".



 Does that help?


**Yes!...**
=======
  ... this helped immensely!


This explanation gave me a much better understanding of how the
whole process works.

Mike, thanks very much for the time and thought you put into this.

Best Regards,
James
Sep 27 2021
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/27/21 10:38 AM, james.p.leblanc wrote:


 I have trouble understanding "module imports" vs. "module compilations".


In addition to what Mike Parker said, templates do complicate matters 
here: Templates are instantiated (i.e. compiled for a specific set of 
template arguments) by modules that actually use those templates.

So, if no non-template definition is used from a module, that module 
need not appear separately on the build line.

For example, 'module a' defines just a function template:

```
module a;

auto doubled(T)(T value) {
   return value * 2;
}
```

The main module imports 'a' and uses and instantiation of a template 
from that module:

```
module main;

import a;

void main() {
   assert(42.doubled == 84);
}
```

The build line may be confusing because in this case a.d need not appear 
on the build line:

$ dmd main.d

Everything works because the compilation of instance doubled!int is 
compiled when compiling main.d and the linker happily finds all symbols 
that are called in the program.

Ali
Sep 27 2021
parent reply james.p.leblanc <james.p.leblanc gmail.com> writes:
On Tuesday, 28 September 2021 at 05:26:29 UTC, Ali Çehreli wrote:

 On 9/27/21 10:38 AM, james.p.leblanc wrote:



 In addition to what Mike Parker said, templates do complicate 
 matters here: Templates are instantiated (i.e. compiled for a 
 specific set of template arguments) by modules that actually 
 use those templates.

 
 Ali


Ali, this is great! ...I had been tempted to also ask about how 
templates
figure into this, but realized that including this in my question 
would be
over complicating the question, so it remained unasked.

But, now I have this part answered as well.  I very much 
appreciate the
mind-reading tricks going on here on the forum!

Thank You, and Best Regards,
James
Sep 27 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/28/21 1:59 AM, james.p.leblanc wrote:

 On Tuesday, 28 September 2021 at 05:26:29 UTC, Ali Çehreli wrote:

 On 9/27/21 10:38 AM, james.p.leblanc wrote:


 

 In addition to what Mike Parker said, templates do complicate matters 
 here: Templates are instantiated (i.e. compiled for a specific set of 
 template arguments) by modules that actually use those templates.


 
 Ali, this is great! ...I had been tempted to also ask about how templates
 figure into this, but realized that including this in my question would be
 over complicating the question, so it remained unasked.
 
 But, now I have this part answered as well.  I very much appreciate the
 mind-reading tricks going on here on the forum!
 


Be aware that the compiler might not include the code for the template 
in the instantiating module if it detects that the instantiation could 
already have been generated in an imported module (not one passed on the 
command line).

For example, if Ali's module `a` contained an alias:

```d
module a;

auto doubled(T)(T value) {
    return value * 2;
}

alias doubleInt = doubled!int;
```

Now the compiler might say "hey, a.d already has instantiated that one, 
and it's not being built by me, so I'll assume it has already generated 
the code" and doesn't do it.

-Steve
Sep 28 2021