www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Compile-time global mutables

reply HuskyNator <HuskyNator protonmail.ch> writes:
I've found several instances in which incrementation during 
compile time is required.
One example is generation of unique ID's (although there might be 
some workarounds), another interesting idea the registration of a 
function's call-count to some image loader;

 Anyone know of a way in c++ to track static resource usage at 
 compile time, basically i am trying to find a way so an routine 
 can just do Image("foobar.jpeg2000"), at an arbitrary point in 
 operation and I'll make sure the resources are resident before 
 the Job gets run.


Although this references c++, I have attempted to make this work 
in D. It seems there are no workarounds that would make this work 
sadly. (I've even tried writing to files instead but this is not 
possible during ctfe)

Essentially speaking, there are numerous instances in which 
generating a list or incrementing a value is useful during 
compile-time. I find this should be possible.

One main argument against this seems to be the non-deterministic 
nature of this, though this is not an issue in virtually all 
cases in which doing this is useful or necessary I find. 
Essentially speaking when order does not matter and membership is 
only required during run-time.

Are there other limiting factors preventing this possibility? It 
would allow for some powerful things that currently seem 
impossible to implement (save for some case-specific workarounds).
Jun 30 2022
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Thursday, 30 June 2022 at 12:26:54 UTC, HuskyNator wrote:

 I've found several instances in which incrementation during 
 compile time is required.
 One example is generation of unique ID's (although there might 
 be some workarounds), another interesting idea the registration 
 of a function's call-count to some image loader;

 Anyone know of a way in c++ to track static resource usage at 
 compile time, basically i am trying to find a way so an 
 routine can just do Image("foobar.jpeg2000"), at an arbitrary 
 point in operation and I'll make sure the resources are 
 resident before the Job gets run.


 Although this references c++, I have attempted to make this 
 work in D. It seems there are no workarounds that would make 
 this work sadly. (I've even tried writing to files instead but 
 this is not possible during ctfe)

 Essentially speaking, there are numerous instances in which 
 generating a list or incrementing a value is useful during 
 compile-time. I find this should be possible.

 One main argument against this seems to be the 
 non-deterministic nature of this, though this is not an issue 
 in virtually all cases in which doing this is useful or 
 necessary I find. Essentially speaking when order does not 
 matter and membership is only required during run-time.

 Are there other limiting factors preventing this possibility? 
 It would allow for some powerful things that currently seem 
 impossible to implement (save for some case-specific 
 workarounds).


You can already do that.

```d
enum counter(size_t x = [__traits(allMembers, 
mixin(__MODULE__))].length)=x;
char[] member(ulong x)
{
     char[] buf = "void[0] _X0000000000000000;".dup;
     enum mapping = "0123456789abcdef";
     foreach_reverse(i; buf.length-17 .. buf.length-1)
     {
         buf[i] = mapping[x & 0xf];
         x >>= 4;
     }
     return buf;
}
mixin template increment()
{
     mixin(member(counter!()));
}

pragma(msg, counter!());
mixin increment;
pragma(msg, counter!());
mixin increment;
pragma(msg, counter!());
```

Credits: p0nce
Jun 30 2022
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Thursday, 30 June 2022 at 12:55:43 UTC, bauss wrote:

 You can already do that.

 ```d
 enum counter(size_t x = [__traits(allMembers, 
 mixin(__MODULE__))].length)=x;
 char[] member(ulong x)
 {
     char[] buf = "void[0] _X0000000000000000;".dup;
     enum mapping = "0123456789abcdef";
     foreach_reverse(i; buf.length-17 .. buf.length-1)
     {
         buf[i] = mapping[x & 0xf];
         x >>= 4;
     }
     return buf;
 }
 mixin template increment()
 {
     mixin(member(counter!()));
 }

 pragma(msg, counter!());
 mixin increment;
 pragma(msg, counter!());
 mixin increment;
 pragma(msg, counter!());
 ```

 Credits: p0nce


I'm assuming this works for counters in general? (although I do 
not see how the member function works, this seems like a very 
sophisticated / fault-sensitive workaround)
Do you know there's workarounds for the other case(s) I have 
described?
Jun 30 2022
parent HuskyNator <HuskyNator protonmail.ch> writes:
On Thursday, 30 June 2022 at 13:04:39 UTC, HuskyNator wrote:

 I'm assuming this works for counters in general? (although I do 
 not see how the member function works, this seems like a very 
 sophisticated / fault-sensitive workaround)
 Do you know there's workarounds for the other case(s) I have 
 described?


Sidenote, would this require separate files for every counter one 
requires? Seeing as allMembers is used. (assuming I understand 
the code correctly)
Jun 30 2022
prev sibling next sibling parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 30 June 2022 at 12:26:54 UTC, HuskyNator wrote:

 One example is generation of unique ID's (although there might 
 be some workarounds)


Here's a mixin that works for this particular use-case:

```d
enum gensym = q{"_gensym" ~ __traits(identifier, 
{})["__lambda".length .. $]};

// Works multiple times on the same line
pragma(msg, mixin(gensym)); pragma(msg, mixin(gensym));
```
Jun 30 2022
prev sibling parent reply Adam Ruppe <destructionator gmail.com> writes:
On Thursday, 30 June 2022 at 12:26:54 UTC, HuskyNator wrote:

 Essentially speaking, there are numerous instances in which 
 generating a list or incrementing a value is useful during 
 compile-time. I find this should be possible.


What are those instances?

The file example you showed is done with a perfectly normal 
constructor, it loads the file before moving on.

If you want to collect a bunch of things and load them at once, 
you can use a module constructor; a  `static this` in a template 
and the compiler adds them all together.

Those module constructors can also increment normal runtime 
variables.

I've done a lot of things in D and managed to get by never using 
a compile time counter...
Jun 30 2022
next sibling parent bauss <jj_1337 live.dk> writes:
On Thursday, 30 June 2022 at 13:16:48 UTC, Adam Ruppe wrote:

 On Thursday, 30 June 2022 at 12:26:54 UTC, HuskyNator wrote:

 Essentially speaking, there are numerous instances in which 
 generating a list or incrementing a value is useful during 
 compile-time. I find this should be possible.


 What are those instances?

 The file example you showed is done with a perfectly normal 
 constructor, it loads the file before moving on.

 If you want to collect a bunch of things and load them at once, 
 you can use a module constructor; a  `static this` in a 
 template and the compiler adds them all together.

 Those module constructors can also increment normal runtime 
 variables.

 I've done a lot of things in D and managed to get by never 
 using a compile time counter...


Yeah, me too. The only time I have ever needed it was for 
demonstrating concepts that were specific to the language and not 
for any productive work.
Jun 30 2022
prev sibling parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Thursday, 30 June 2022 at 13:16:48 UTC, Adam Ruppe wrote:

 If you want to collect a bunch of things and load them at once, 
 you can use a module constructor; a  `static this` in a 
 template and the compiler adds them all together.


Not sure I see how that would apply to this example.
How would you write a static constructor that loads in all files 
used somewhere in the source code? This seems like a prerequisite 
to being able to do that.
Jun 30 2022
next sibling parent reply Adam Ruppe <destructionator gmail.com> writes:
On Thursday, 30 June 2022 at 13:39:57 UTC, HuskyNator wrote:

 How would you write a static constructor that loads in all 
 files used somewhere in the source code? This seems like a 
 prerequisite to being able to do that.


template file(string name) {
     import yourproject.allfiles;

     shared static this() {
         allfiles ~= name;
     }

     enum file = name;
}


Use it like:

auto img = Image(file!"thing.jpg");


And then yourproject.allfiles module has:

__gshared string[] allfiles; // you MUST only write to it from 
those constructors but afterwards can iterate it since you know 
it is immutable from that point on


And somewhere in your main() function:


void main() {
     import yourpoject.allfiles;
     foreach(file; allfiles)
         loadFile(file);
}



Or similar. The key point is the magic `file` template builds an 
array of all files in its static constructors which you then 
process later at runtime.
Jun 30 2022
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Thursday, 30 June 2022 at 13:47:47 UTC, Adam Ruppe wrote:

 Or similar. The key point is the magic `file` template builds 
 an array of all files in its static constructors which you then 
 process later at runtime.


I had no idea templates could contain static constructors!
Did I miss some piece of documentation?
Thanks a lot, this basically behaves as I would hope!

The only part about this that seems a little disappointing is it 
still having to work at run-time. I'm also curious what this 
would create in the application. Are you left with potentially 
dozens if not hundreds of useless functions in the code? The 
`static this` methods are all separate instances correct?
Jun 30 2022
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 30 June 2022 at 14:38:30 UTC, HuskyNator wrote:

 Did I miss some piece of documentation?


Key concept is a template is just another aggregate...


 I'm also curious what this would create in the application. Are 
 you left with potentially dozens if not hundreds of useless 
 functions in the code? The `static this` methods are all 
 separate instances correct?


No, it combines them all into a single one per-module in the 
compiler. Then the modules are combined at run time by walking an 
array of them and running each function in turn.
Jun 30 2022
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Thursday, 30 June 2022 at 14:45:49 UTC, Adam D Ruppe wrote:

 On Thursday, 30 June 2022 at 14:38:30 UTC, HuskyNator wrote:

 Did I miss some piece of documentation?


 Key concept is a template is just another aggregate...


I'm not sure saying 'its self-evident' is a great way of writing 
documentation.
It's not evident to me, and none of the tutorials / documentation 
seems to even mention it.
Jun 30 2022
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 30 June 2022 at 15:37:20 UTC, HuskyNator wrote:

 It's not evident to me, and none of the tutorials / 
 documentation seems to even mention it.


I talked about it in my book, but yeah, it could probably use a 
call out somewhere else.
Jun 30 2022
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Thursday, 30 June 2022 at 15:49:07 UTC, Adam D Ruppe wrote:

 I talked about it in my book, but yeah, it could probably use a 
 call out somewhere else.


The github page on the specs does not seem to have an `issues` 
section on github.
Is there any place to suggest this addition?

---
_On a similar note the language spec does not put `.capacity` in 
the Array properties table 
(https://dlang.org/spec/arrays.html#array-properties). My ide 
does not even suggest it exists (code-d in vscode)._
Jul 01 2022
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Friday, 1 July 2022 at 12:13:03 UTC, HuskyNator wrote:

 Is there any place to suggest this addition?


the bugzilla issues.dlang.org


 _On a similar note the language spec does not put `.capacity` 
 in the Array properties table 
 (https://dlang.org/spec/arrays.html#array-properties).


try my search engine

dpldocs.info/capacity

it will bring up

http://druntime.dpldocs.info/object.capacity.html

I think this isn't technically defined by the language and is a 
pure library construct. Just the auto-imported library thing/


 My ide does not even suggest it exists (code-d in vscode)._


can't trust ides tbh.
Jul 01 2022
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Friday, 1 July 2022 at 12:24:55 UTC, Adam D Ruppe wrote:

 On Friday, 1 July 2022 at 12:13:03 UTC, HuskyNator wrote:
 I think this isn't technically defined by the language and is a 
 pure library construct. Just the auto-imported library thing/


It is references, but hidden in the text somewhere. (also note it 
seems to be applicable to static arrays as well, though it yields 
0) 
https://dlang.org/spec/arrays.html#dynamic-arrays:~:text=To%20guarantee%20copying%20behavior%2C%20use%20the%20.dup%20property%20to%20ensure%20a%20unique%20array%20that%20can%20be%20resized.%20Also%2C%20one%20may%20use%20the%20.capacity%20property%20to%20determine%20how%20many%20elements%20can%20be%20appended%20to%20the%20array%20without%20reallocating.
Jul 01 2022
parent reply HuskyNator <HuskyNator protonmail.ch> writes:
On Friday, 1 July 2022 at 12:44:01 UTC, HuskyNator wrote:

 (also note it seems to be applicable to static arrays as well, 
 though it yields 0)


I'm assuming this isn't part of a library in any case.
Jul 01 2022
parent HuskyNator <HuskyNator protonmail.ch> writes:
On Friday, 1 July 2022 at 12:47:49 UTC, HuskyNator wrote:

 On Friday, 1 July 2022 at 12:44:01 UTC, HuskyNator wrote:

 (also note it seems to be applicable to static arrays as well, 
 though it yields 0)


 I'm assuming this isn't part of a library in any case.


Never mind, I hadn't considered UFC.
Didn't make that out from the documentation either.
Apologies for the slight post overdose 😅
Jul 01 2022
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 30 June 2022 at 13:39:57 UTC, HuskyNator wrote:

 On Thursday, 30 June 2022 at 13:16:48 UTC, Adam Ruppe wrote:

 If you want to collect a bunch of things and load them at 
 once, you can use a module constructor; a  `static this` in a 
 template and the compiler adds them all together.


 Not sure I see how that would apply to this example.
 How would you write a static constructor that loads in all 
 files used somewhere in the source code? This seems like a 
 prerequisite to being able to do that.


```d
string[] files;

template register(string file)
{
     static this()
     {
         // TODO: actually load the file
         files ~= file;
     }
     void register() {}
}

void foo()
{
     register!"foo";
}

void bar()
{
     register!"bar";
}

void main()
{
     import std.stdio;
     writeln(files); // ["foo", "bar"]
}
```
Jun 30 2022