• Stefan Koch (66/66) Oct 01 2021 Good day everyone,
• Stefan Koch (5/72) Oct 01 2021 Note. Those are only the _direct_ dependencies as I don't crawl
• Basile B. (7/9) Oct 02 2021 To which extent self-linting would be work with core.reflect ?
• Stefan Koch (5/14) Oct 02 2021 If you can inject code as a trigger it would work just fine.
• Stefan Koch (17/22) Oct 02 2021 You don't even need to have an explicit list of symbols.

Stefan Koch <uplink.coder googlemail.com> writes:

Good day everyone,

I know multiple programs have been written in order to make sense 
of phobos dependencies.

I am going to show how you can get the set of modules a module is 
'statically' dependent upon.  (this means the set won't include 
non-instantiated templates. or failing `static if` statements)

```D
import core.reflect.reflect;
import core.reflect.transitiveVisitor;

import std_traits = std.traits;
import std_stdio = std.stdio;
import core.stdc.stdio;

static immutable node = nodeFromName("std_traits", 
ReflectFlags.Everything);
static immutable node2 = nodeFromName("std_stdio", 
ReflectFlags.Everything);

import std; // for basename join and writeln

void main()
{
     auto std_traits_deps = referencedFiles(node);
     auto std_stdio_deps = referencedFiles(node2);

     writeln("std.traits deps: ", std_traits_deps.map!(e => 
baseName(e)).join("\n"));
     writeln("std.stdio deps: ", std_stdio_deps.map!(e => 
baseName(e)).join(", "));
}

string[] referencedFiles(const Node n)
{
     class FileLocationVisitor : TransitiveVisitor
     {
         alias visit = TransitiveVisitor.visit;

         override void visit(Location loc)
         {
             if (loc.filename !in fileSet)
             {
                 fileSet[loc.filename] = 1;
             }
         }

         int[string] fileSet;
     }

     scope fileLocationCollector = new FileLocationVisitor();
     (cast()n).accept(fileLocationCollector);

     return fileLocationCollector.fileSet.keys;
}
```

and the result of these few lines is the following output.

```
std.traits deps: traits.d
std.stdio deps: , time.d, string.d, memory.d-mixin-32, errno.d, 
checkedint.d, uio.d, time.d, stdio.d, capacity.d, functional.d, 
array.d, atomic.d, comparison.d, utf.d, memory.d, inet.d, 
string.d, atomic.d-mixin-324, lifetime.d, time.d, unistd.d, 
atomic.d-mixin-271, destruction.d, equality.d, mutation.d, 
traits.d, conv.d, atomic.d, comparison.d, demangle.d, cstring.d, 
functional.d-mixin-446, bitop.d, in_.d, sysv_x64.d, hash.d, 
stdlib.d, fcntl.d, memory.d, lifetime.d, exception.d, 
exception.d, stdio.d, wchar_.d, netdb.d, ascii.d, stdio.d, 
socket.d, typecons.d, object.d, convert.d, stdio.d-mixin-5219, 
traits.d-mixin-127, signal.d
```

Which could surely be prettified. with the appropriate Phobos 
range functions.
So now you know :) why hello world takes ages. importing 
std.stdio pulls in the world :)

Please let me know if this needs more explanation and if you 
found the usage of core.reflect intuitive.

Stefan Koch <uplink.coder googlemail.com> writes:

On Friday, 1 October 2021 at 14:44:24 UTC, Stefan Koch wrote:
 Good day everyone,

 I know multiple programs have been written in order to make 
 sense of phobos dependencies.

 I am going to show how you can get the set of modules a module 
 is 'statically' dependent upon.  (this means the set won't 
 include non-instantiated templates. or failing `static if` 
 statements)

 ```D
 import core.reflect.reflect;
 import core.reflect.transitiveVisitor;

 import std_traits = std.traits;
 import std_stdio = std.stdio;
 import core.stdc.stdio;

 static immutable node = nodeFromName("std_traits", 
 ReflectFlags.Everything);
 static immutable node2 = nodeFromName("std_stdio", 
 ReflectFlags.Everything);

 import std; // for basename join and writeln

 void main()
 {
     auto std_traits_deps = referencedFiles(node);
     auto std_stdio_deps = referencedFiles(node2);

     writeln("std.traits deps: ", std_traits_deps.map!(e => 
 baseName(e)).join("\n"));
     writeln("std.stdio deps: ", std_stdio_deps.map!(e => 
 baseName(e)).join(", "));
 }

 string[] referencedFiles(const Node n)
 {
     class FileLocationVisitor : TransitiveVisitor
     {
         alias visit = TransitiveVisitor.visit;

         override void visit(Location loc)
         {
             if (loc.filename !in fileSet)
             {
                 fileSet[loc.filename] = 1;
             }
         }

         int[string] fileSet;
     }

     scope fileLocationCollector = new FileLocationVisitor();
     (cast()n).accept(fileLocationCollector);

     return fileLocationCollector.fileSet.keys;
 }
 ```

 and the result of these few lines is the following output.

 ```
 std.traits deps: traits.d
 std.stdio deps: , time.d, string.d, memory.d-mixin-32, errno.d, 
 checkedint.d, uio.d, time.d, stdio.d, capacity.d, functional.d, 
 array.d, atomic.d, comparison.d, utf.d, memory.d, inet.d, 
 string.d, atomic.d-mixin-324, lifetime.d, time.d, unistd.d, 
 atomic.d-mixin-271, destruction.d, equality.d, mutation.d, 
 traits.d, conv.d, atomic.d, comparison.d, demangle.d, 
 cstring.d, functional.d-mixin-446, bitop.d, in_.d, sysv_x64.d, 
 hash.d, stdlib.d, fcntl.d, memory.d, lifetime.d, exception.d, 
 exception.d, stdio.d, wchar_.d, netdb.d, ascii.d, stdio.d, 
 socket.d, typecons.d, object.d, convert.d, stdio.d-mixin-5219, 
 traits.d-mixin-127, signal.d
 ```

 Which could surely be prettified. with the appropriate Phobos 
 range functions.
 So now you know :) why hello world takes ages. importing 
 std.stdio pulls in the world :)

 Please let me know if this needs more explanation and if you 
 found the usage of core.reflect intuitive.

Note. Those are only the _direct_ dependencies as I don't crawl 
the import-graph right now.
Reflecting over import graphs is currently being implemented.
However the trees get so huge that I have trouble debugging it.

Basile B. <b2.temp gmx.com> writes:

On Friday, 1 October 2021 at 14:44:24 UTC, Stefan Koch wrote:
 Please let me know if this needs more explanation and if you 
 found the usage of core.reflect intuitive.

To which extent self-linting would be work with core.reflect ?

I'm thinking to something like add `import self_lint;`. That 
module would implement several visitors, like the D-scanner 
checks. Then during compilation you can also lint, for example 
using `enum _ = lintSymbols!(SequenceOfSymbolToLint).` to trigger 
the checks.

Stefan Koch <uplink.coder googlemail.com> writes:

On Saturday, 2 October 2021 at 10:16:56 UTC, Basile B. wrote:
 On Friday, 1 October 2021 at 14:44:24 UTC, Stefan Koch wrote:
 Please let me know if this needs more explanation and if you 
 found the usage of core.reflect intuitive.

 To which extent self-linting would be work with core.reflect ?

 I'm thinking to something like add `import self_lint;`. That 
 module would implement several visitors, like the D-scanner 
 checks. Then during compilation you can also lint, for example 
 using `enum _ = lintSymbols!(SequenceOfSymbolToLint).` to 
 trigger the checks.

If you can inject code as a trigger it would work just fine.
However it cannot just be the import.
Adding an import must never change the module that is the 
importer.

Stefan Koch <uplink.coder googlemail.com> writes:

On Saturday, 2 October 2021 at 10:45:29 UTC, Stefan Koch wrote:
 If you can inject code as a trigger it would work just fine.
 However it cannot just be the import.
 Adding an import must never change the module that is the 
 importer.

 using `enum _ = lintSymbols!(SequenceOfSymbolToLint).`


You don't even need to have an explicit list of symbols.
adding an

```D
static immutable complaintString = lintModule.complaintString;
static assert(!complaintString, complaintString);
```
would be all that's required, and it could optionally match UDAs
which you would do using
```D

static immutable complaintString = 
lintModuleWithUDA(nodeFromName("myLintUDA")).complaintString;
static assert(!complaintString, complaintString);
```

note that this use would store a complaintString in the binary 
but since it'll abort compilation if there are complaints that 
would never actually bloat anything ;)