www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Accessing static data of functions

reply Ogi <ogion.art gmail.com> writes:
Is there any way to get/set static variables defined in functions 
from the outside? Also, is there any way to access the types 
defined inside in function to get/set their static members?

I’m writing a game engine that should support a dynamic plugin 
system similar to what Manu Evans described in his [Using D 
Alongside a Game Engine](https://youtu.be/FKceA691Wcg) talk. 
Basically, game objects are defined in modules to be compiled as 
DLLs and loaded dynamically. This allows making modifications 
without even restarting the game. Plugin manager detects a 
modification in a plugin source file, serializes all game objects 
into text, unloads a DLL, recompiles it, loads it again and 
deserialize the game objects back.

So far so good, I’m now able to serialize/deserialize game 
objects (including private fields), and any global variables 
defined in a plugin, and any static members of the types defined 
in a plugin. But I don’t know what to do with the stuff that 
could be defined inside of a function, I can’t find any trait or 
a template that could access it.
Oct 14 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 14 October 2021 at 14:21:21 UTC, Ogi wrote:

 Is there any way to get/set static variables defined in 
 functions from the outside? Also, is there any way to access 
 the types defined inside in function to get/set their static 
 members?


No.


 So far so good, I’m now able to serialize/deserialize game 
 objects (including private fields), and any global variables 
 defined in a plugin, and any static members of the types 
 defined in a plugin. But I don’t know what to do with the stuff 
 that could be defined inside of a function, I can’t find any 
 trait or a template that could access it.


Avoid storing any of your game's state in static variables.

You can refactor any function with a static variable into a 
struct method:

```d
// Before:
int fun()
{
     static int n = 123;
     return n;
}

// After:
struct S
{
     int n;
     int fun() { return n; }
}

auto s = S(123);
// call s.fun()
```
Oct 14 2021
parent reply Ogi <ogion.art gmail.com> writes:
On Thursday, 14 October 2021 at 14:32:49 UTC, Paul Backus wrote:

 No.


Is there any solid reasoning behind this gaping hole in the 
language reflection capabilities? I understand that in general it 
is not a good idea to meddle with static variables but 
serialization/deserialization is a good example where this is 
required.


 Avoid storing any of your game's state in static variables.


What are we talking about is not a game but an engine to be used 
for making different games, potentially by people other than me. 
So this is a limitation that the users will have to be aware of 
and work around it, and failing to do so will result in silent 
bugs in their code because there’s no way to enforce it at 
compile time.
Oct 19 2021
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Tuesday, 19 October 2021 at 08:15:01 UTC, Ogi wrote:

 What are we talking about is not a game but an engine to be 
 used for making different games, potentially by people other 
 than me. So this is a limitation that the users will have to be 
 aware of and work around it, and failing to do so will result 
 in silent bugs in their code because there’s no way to enforce 
 it at compile time.


Personally, this isn't something I would expect, nor would I 
worry about it. Static variables in functions are generally there 
because they are scoped to that function, to be used within that 
function over the run time of the program. There shouldn't be any 
reason whatsoever for them to be accessible outside the function.

The idea that you are even considering serializing them is just a 
foreign concept to me. If I wanted to make something available to 
a serializer, I wouldn't declare it inside a function. I'd put it 
at module scope, or in an aggregate. I'm not saying no one would 
ever think of it (you obviously did), but in 25 years of 
following various online programming communities (C, Java, D, 
game development, etc.) I can't recall seeing anyone bring it up 
before now. I can't imagine it would be common enough to be an 
issue.
Oct 19 2021
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 19.10.21 10:31, Mike Parker wrote:

 On Tuesday, 19 October 2021 at 08:15:01 UTC, Ogi wrote:

 What are we talking about is not a game but an engine to be used for 
 making different games, potentially by people other than me. So this 
 is a limitation that the users will have to be aware of and work 
 around it, and failing to do so will result in silent bugs in their 
 code because there’s no way to enforce it at compile time.


 
 Personally, this isn't something I would expect, nor would I worry about 
 it. Static variables in functions are generally there because they are 
 scoped to that function, to be used within that function over the run 
 time of the program. There shouldn't be any reason whatsoever for them 
 to be accessible outside the function.
 
 The idea that you are even considering serializing them is just a 
 foreign concept to me. If I wanted to make something available to a 
 serializer, I wouldn't declare it inside a function. I'd put it at 
 module scope, or in an aggregate. I'm not saying no one would ever think 
 of it (you obviously did), but in 25 years of following various online 
 programming communities (C, Java, D, game development, etc.) I can't 
 recall seeing anyone bring it up before now. I can't imagine it would be 
 common enough to be an issue.


The OP has a well-motivated use case. I don't understand at all why you 
are being this dismissive. The inability to serialize static variables 
scoped inside functions is clearly an arbitrary limitation that should 
be lifted.
Oct 19 2021
next sibling parent Kagamin <spam here.lot> writes:
On Tuesday, 19 October 2021 at 11:12:12 UTC, Timon Gehr wrote:

 The OP has a well-motivated use case. I don't understand at all 
 why you are being this dismissive. The inability to serialize 
 static variables scoped inside functions is clearly an 
 arbitrary limitation that should be lifted.


As an option a virtual machine should be able to serialize 
everything or at least RAM.
Oct 19 2021
prev sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Tuesday, 19 October 2021 at 11:12:12 UTC, Timon Gehr wrote:


 The OP has a well-motivated use case. I don't understand at all 
 why you are being this dismissive. The inability to serialize 
 static variables scoped inside functions is clearly an 
 arbitrary limitation that should be lifted.


Sorry that I came off that way. I'm not opposed to the feature at 
all. If someone wants to implement it, I expect it can get by 
without a DIP. An enhancement request in Bugzilla should suffice.

I was responding to the OP's concern about users of the library. 
Based on my experience, I just can't imagine it ever being an 
issue. Of course, I may well be wrong.
Oct 19 2021
parent WebFreak001 <d.forum webfreak.org> writes:
On Tuesday, 19 October 2021 at 13:36:24 UTC, Mike Parker wrote:

 On Tuesday, 19 October 2021 at 11:12:12 UTC, Timon Gehr wrote:


 The OP has a well-motivated use case. I don't understand at 
 all why you are being this dismissive. The inability to 
 serialize static variables scoped inside functions is clearly 
 an arbitrary limitation that should be lifted.


 Sorry that I came off that way. I'm not opposed to the feature 
 at all. If someone wants to implement it, I expect it can get 
 by without a DIP. An enhancement request in Bugzilla should 
 suffice.

 I was responding to the OP's concern about users of the 
 library. Based on my experience, I just can't imagine it ever 
 being an issue. Of course, I may well be wrong.


I made a library before that would store the whole application 
state in a file, restart the application when an update on the 
filesystem exists and restore the state from the file 
automatically on startup in the new process using traits. I only 
had some fixed global variables but if you made that a library I 
can see serializing static variables as a good inclusion.
Oct 19 2021
prev sibling next sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Tuesday, 19 October 2021 at 08:31:47 UTC, Mike Parker wrote:

 On Tuesday, 19 October 2021 at 08:15:01 UTC, Ogi wrote:

 What are we talking about is not a game but an engine to be 
 used for making different games, potentially by people other 
 than me. So this is a limitation that the users will have to 
 be aware of and work around it, and failing to do so will 
 result in silent bugs in their code because there’s no way to 
 enforce it at compile time.


 Personally, this isn't something I would expect, nor would I 
 worry about it. Static variables in functions are generally 
 there because they are scoped to that function, to be used 
 within that function over the run time of the program. There 
 shouldn't be any reason whatsoever for them to be accessible 
 outside the function.

 The idea that you are even considering serializing them is just 
 a foreign concept to me. If I wanted to make something 
 available to a serializer, I wouldn't declare it inside a 
 function. I'd put it at module scope, or in an aggregate. I'm 
 not saying no one would ever think of it (you obviously did), 
 but in 25 years of following various online programming 
 communities (C, Java, D, game development, etc.) I can't recall 
 seeing anyone bring it up before now. I can't imagine it would 
 be common enough to be an issue.


I was actually thinking something similar. Have never heard of 
that requirement before. You're serializing the object, not the 
class, and the static variable doesn't belong to the object (in 
many languages) but the class. I guess D looked at Java? Have no 
evidence of that atm tho.

That doesn't mean D *should* "do what Java does" ofc.
Oct 19 2021
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 19.10.21 18:43, Imperatorn wrote:

 On Tuesday, 19 October 2021 at 08:31:47 UTC, Mike Parker wrote:

 On Tuesday, 19 October 2021 at 08:15:01 UTC, Ogi wrote:

 What are we talking about is not a game but an engine to be used for 
 making different games, potentially by people other than me. So this 
 is a limitation that the users will have to be aware of and work 
 around it, and failing to do so will result in silent bugs in their 
 code because there’s no way to enforce it at compile time.


 Personally, this isn't something I would expect, nor would I worry 
 about it. Static variables in functions are generally there because 
 they are scoped to that function, to be used within that function over 
 the run time of the program. There shouldn't be any reason whatsoever 
 for them to be accessible outside the function.

 The idea that you are even considering serializing them is just a 
 foreign concept to me. If I wanted to make something available to a 
 serializer, I wouldn't declare it inside a function. I'd put it at 
 module scope, or in an aggregate. I'm not saying no one would ever 
 think of it (you obviously did), but in 25 years of following various 
 online programming communities (C, Java, D, game development, etc.) I 
 can't recall seeing anyone bring it up before now. I can't imagine it 
 would be common enough to be an issue.


 
 I was actually thinking something similar. Have never heard of that 
 requirement before. You're serializing the object, not the class, and 
 the static variable doesn't belong to the object (in many languages) but 
 the class. I guess D looked at Java? Have no evidence of that atm tho.
 
 That doesn't mean D *should* "do what Java does" ofc.


This is about static variables at function scope. Java does not have 
static variables at function scope.

string uniqueName(){
     static int id = 0; // this is the value that OP wants to preserve 
on reload
     return text("__tmp", id++);
}
Oct 19 2021
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Tuesday, 19 October 2021 at 17:04:32 UTC, Timon Gehr wrote:

 On 19.10.21 18:43, Imperatorn wrote:

 On Tuesday, 19 October 2021 at 08:31:47 UTC, Mike Parker wrote:

 [...]


 
 I was actually thinking something similar. Have never heard of 
 that requirement before. You're serializing the object, not 
 the class, and the static variable doesn't belong to the 
 object (in many languages) but the class. I guess D looked at 
 Java? Have no evidence of that atm tho.
 
 That doesn't mean D *should* "do what Java does" ofc.


 This is about static variables at function scope. Java does not 
 have static variables at function scope.

 string uniqueName(){
     static int id = 0; // this is the value that OP wants to 
 preserve on reload
     return text("__tmp", id++);
 }


Ok, I know nothing about Java other than D got some inspiration
Oct 19 2021
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Tuesday, 19 October 2021 at 17:08:41 UTC, Imperatorn wrote:

 On Tuesday, 19 October 2021 at 17:04:32 UTC, Timon Gehr wrote:

 On 19.10.21 18:43, Imperatorn wrote:

 [...]


 This is about static variables at function scope. Java does 
 not have static variables at function scope.

 string uniqueName(){
     static int id = 0; // this is the value that OP wants to 
 preserve on reload
     return text("__tmp", id++);
 }


 Ok, I know nothing about Java other than D got some inspiration 



Well, ok, I know *some* things. Wrote some mobile Java apps but 
that's about it.
Oct 19 2021
prev sibling parent Ogi <ogion.art gmail.com> writes:
On Tuesday, 19 October 2021 at 08:31:47 UTC, Mike Parker wrote:

 Personally, this isn't something I would expect, nor would I 
 worry about it. Static variables in functions are generally 
 there because they are scoped to that function, to be used 
 within that function over the run time of the program. There 
 shouldn't be any reason whatsoever for them to be accessible 
 outside the function.


The way I see it, static variables are just global variables with 
a restricted visibility, and we should be able to bypass 
visibility rules if we really need the same way we can bypass 
member visibility with `getMember`.
Oct 19 2021
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 19 October 2021 at 08:15:01 UTC, Ogi wrote:

 On Thursday, 14 October 2021 at 14:32:49 UTC, Paul Backus wrote:

 No.


 Is there any solid reasoning behind this gaping hole in the 
 language reflection capabilities? I understand that in general 
 it is not a good idea to meddle with static variables but 
 serialization/deserialization is a good example where this is 
 required.


You can't serialize everything. The standard approach to 
serialization is to use DTOs - types tailored for serialization.
Oct 19 2021
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 19.10.21 11:19, Kagamin wrote:

 On Tuesday, 19 October 2021 at 08:15:01 UTC, Ogi wrote:

 On Thursday, 14 October 2021 at 14:32:49 UTC, Paul Backus wrote:

 No.


 Is there any solid reasoning behind this gaping hole in the language 
 reflection capabilities? I understand that in general it is not a good 
 idea to meddle with static variables but serialization/deserialization 
 is a good example where this is required.


 
 You can't serialize everything.


Why not?


 The standard approach to serialization is to use DTOs - types tailored for
serialization.


Clearly the ambition of the OP is to do something that's better than the 
existing standard.
Oct 19 2021
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 19.10.21 10:15, Ogi wrote:

 
 What are we talking about is not a game but an engine to be used for 
 making different games, potentially by people other than me. So this is 
 a limitation that the users will have to be aware of and work around it, 
 and failing to do so will result in silent bugs in their code because 
 there’s no way to enforce it at compile time.


Without the necessary language extension, what you could do as a 
workaround is require all functions in the plugin to be `pure`. Then 
they can't access static variables. Of course that also rules out 
accessing globals, so you'd have to pass around global state explicitly 
within some object.
Oct 19 2021
prev sibling next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 10/14/21 7:21 AM, Ogi wrote:


 is there any way to access the types defined inside
 in function to get/set their static members?


Possible through instances of those types (not through type names 
because those type names are not visible from the outside):

auto foo() {
   struct S {
     static int s;
   }

   return S();
}

void main() {
   auto s = foo();
   s.s = 42;

   foo().s++;
   assert(foo().s == 43);
}

Ali
Oct 14 2021
prev sibling next sibling parent Tejas <notrealemail gmail.com> writes:
On Thursday, 14 October 2021 at 14:21:21 UTC, Ogi wrote:

 Is there any way to get/set static variables defined in 
 functions from the outside? Also, is there any way to access 
 the types defined inside in function to get/set their static 
 members?

 I’m writing a game engine that should support a dynamic plugin 
 system similar to what Manu Evans described in his [Using D 
 Alongside a Game Engine](https://youtu.be/FKceA691Wcg) talk. 
 Basically, game objects are defined in modules to be compiled 
 as DLLs and loaded dynamically. This allows making 
 modifications without even restarting the game. Plugin manager 
 detects a modification in a plugin source file, serializes all 
 game objects into text, unloads a DLL, recompiles it, loads it 
 again and deserialize the game objects back.

 So far so good, I’m now able to serialize/deserialize game 
 objects (including private fields), and any global variables 
 defined in a plugin, and any static members of the types 
 defined in a plugin. But I don’t know what to do with the stuff 
 that could be defined inside of a function, I can’t find any 
 trait or a template that could access it.


Seeing the turn this thread has taken, could you file an 
enhancement request at bugzilla for this particular case? It 
doesn't seem like it was a design choice to disable your 
requested behaviour, nor will it break anything.

Hopefully a like-minded individual can make a PR.
Oct 19 2021
prev sibling next sibling parent Kagamin <spam here.lot> writes:
On Thursday, 14 October 2021 at 14:21:21 UTC, Ogi wrote:

 I’m writing a game engine that should support a dynamic plugin 
 system similar to what Manu Evans described in his talk. 
 Basically, game objects are defined in modules to be compiled 
 as DLLs and loaded dynamically. This allows making 
 modifications without even restarting the game. Plugin manager 
 detects a modification in a plugin source file, serializes all 
 game objects into text, unloads a DLL, recompiles it, loads it 
 again and deserialize the game objects back.


Back in the days games had quick save and quick restore 
functionality, a plugin should be able to save its state in such 
savegame, then you can combine quick save + plugin swap + quick 
restore to get what you want.
Oct 19 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/14/21 10:21 AM, Ogi wrote:

 Is there any way to get/set static variables defined in functions from 
 the outside? Also, is there any way to access the types defined inside 
 in function to get/set their static members?
 
 I’m writing a game engine that should support a dynamic plugin system 
 similar to what Manu Evans described in his [Using D Alongside a Game 
 Engine](https://youtu.be/FKceA691Wcg) talk. Basically, game objects are 
 defined in modules to be compiled as DLLs and loaded dynamically. This 
 allows making modifications without even restarting the game. Plugin 
 manager detects a modification in a plugin source file, serializes all 
 game objects into text, unloads a DLL, recompiles it, loads it again and 
 deserialize the game objects back.
 
 So far so good, I’m now able to serialize/deserialize game objects 
 (including private fields), and any global variables defined in a 
 plugin, and any static members of the types defined in a plugin. But I 
 don’t know what to do with the stuff that could be defined inside of a 
 function, I can’t find any trait or a template that could access it.


As long as it has a mangle, you can access it.

https://run.dlang.io/is/uukAPa

Now, how to get at that mangle? I think it's impossible to do 
introspectively (or even detect that those things exist). But if you 
have control of the code, it's possible.

-Steve
Oct 19 2021
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/19/21 4:19 PM, Steven Schveighoffer wrote:

 I think it's impossible to do introspectively (or even detect that those 
 things exist).


import(__FILE__)

;)
Oct 19 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/19/21 4:04 PM, Timon Gehr wrote:

 On 10/19/21 4:19 PM, Steven Schveighoffer wrote:

 I think it's impossible to do introspectively (or even detect that 
 those things exist).


 
 import(__FILE__)
 
 ;)


Ok, so not *impossible*. Just horrific ;)

It reminds me of an early tango library called 
[flectioned](http://dsource.org/projects/flectioned), which actually 
opened the executable file and parsed the debug info so you could do 
runtime reflection without needing the source.

-Steve
Oct 20 2021