www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to get compatible symbol names and runtime typeid names for

reply cc <cc nevernet.com> writes:
This produces compatible strings between symbol and runtime type:
```d
class Foo {}
void main() {
	alias Foo F;
	writeln(fullyQualifiedName!F);
	auto f = new F;
	writeln(typeid(f).name);
}
```
```
test.Foo
test.Foo
```

But if the class is a template, the strings different:
```d
class Foo(bool b) {}
void main() {
	alias Foo!true F;
	writeln(fullyQualifiedName!F);
	auto f = new F;
	writeln(typeid(f).name);
}
```
```
test.Foo!(true)
test.Foo!true.Foo
```

Given a runtime typeid, how can I get the equivalent 
fullyQualifiedName without attempting to mangle the string myself 
manually?  e.g. something I can pass to `Object.factory`.
May 03 2022
next sibling parent reply cc <cc nevernet.com> writes:
On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote:
 Given a runtime typeid, how can I get the equivalent 
 fullyQualifiedName without attempting to mangle the string 
 myself manually?  e.g. something I can pass to `Object.factory`.
Actually, looking at this further, does Object.factory even support templates? I'm getting null returned from any attempt to instantiate a templated classname.
May 03 2022
parent reply bauss <jj_1337 live.dk> writes:
On Tuesday, 3 May 2022 at 09:52:56 UTC, cc wrote:
 On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote:
 Given a runtime typeid, how can I get the equivalent 
 fullyQualifiedName without attempting to mangle the string 
 myself manually?  e.g. something I can pass to 
 `Object.factory`.
Actually, looking at this further, does Object.factory even support templates? I'm getting null returned from any attempt to instantiate a templated classname.
It does not. Object.factory calls TypeInfo_Class.find which just loops through ModuleInfo and then looks if any of the entries in localClasses has a name that matches. So for your example it does this check: ``` if (c.name == "test.Foo!(true)") { return c; // c is the TypeInfo_Class that matches the given class name } ``` https://github.com/dlang/druntime/blob/master/src/object.d#L1661 Afterwards it calls the create function on the TypeInfo_Class which of course isn't "generic" by any means. This is where compile-time has its limits compared to runtime type creation, because templates only live during compile-time then it isn't really that easy to do something like this, where
May 03 2022
next sibling parent Arafel <er.krali gmail.com> writes:
On 3/5/22 12:48, bauss wrote:
 This is where compile-time has its limits compared to runtime type 
 creation, because templates only live during compile-time then it isn't 
 really that easy to do something like this, where it would be trivial in 

That's something I don't really get. I totally understand that you can't instantiate the template during runtime, but why can't already instantiated classes be registered just like non-templated ones? I tried the following snippet, and couldn't find C!int.C anywhere, although it **must** be there: I can get the `TypeInfo_Class` object, so I can clearly create new instances at runtime: ```d import std.stdio : writeln; class C(T) {} class D {} void main() { auto c = new C!int(); auto c2 = typeid(c).create(); auto d = new D(); writeln(typeid(c).name); writeln(typeid(c2).name); writeln(typeid(d).name); writeln("----"); writeln; writeln; foreach (m; ModuleInfo) { if (m) { writeln(m.name); writeln("--------------------"); foreach (c; m.localClasses) { if (c) { writeln(c.name); } } writeln; } } } ```
May 03 2022
prev sibling parent cc <cc nevernet.com> writes:
On Tuesday, 3 May 2022 at 10:48:53 UTC, bauss wrote:
 Object.factory calls TypeInfo_Class.find which just loops 
 through ModuleInfo and then looks if any of the entries in 
 localClasses has a name that matches.

 Afterwards it calls the create function on the TypeInfo_Class 
 which of course isn't "generic" by any means.

 This is where compile-time has its limits compared to runtime 
 type creation, because templates only live during compile-time 
 then it isn't really that easy to do something like this, where 

On Tuesday, 3 May 2022 at 12:46:56 UTC, Adam D Ruppe wrote:
 On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote:
 something I can pass to `Object.factory`.
Object.factory is useless and will hopefully be removed someday. Instead, make your own factory registration function. Put a static constructor in the class which appends a factory delegate to an array or something you can use later. Then you can use your own thing to construct registered objects.
Yeah, that's unfortunate. Actually I was already doing something similar for serialization/encoding to get the true type of an object (making sure `Animal an = new Cat();` encodes a Cat and not an Animal), took me a second to put two and two together and realize I could just instantiate objects via new that way instead of calling Object.factory. At the moment I try to register as many relevant symbols as I can automatically when encoding is called for a given object, such as: ```d private mixin template RegisterModule(alias MOD) { void RegisterModule() { static foreach (SYM; getSymbolsByUDA!(MOD, Coder)) { static if (is(SYM == class)) { RegisterSerializer!SYM(); } } } } private static void[0][string] registeredModules; private void registerModules(T)() { enum string MODULENAME = moduleName!T; if (MODULENAME !in registeredModules) { registeredModules.require(MODULENAME); mixin("import "~MODULENAME~";"); mixin("mixin RegisterModule!"~MODULENAME~";"); RegisterModule(); } } IPtr encode(T)(T obj) { registerModules!T; ... } ``` I'll have to get a little more creative for registering templated classes then, something like this works: ```d static void RegisterSerializer(alias SYM)(string runtimeName = null) { enum sym = fullyQualifiedName!SYM; if (sym !in serialTypes) { auto st = new SerialType!SYM; serialTypes[sym] = st; if (runtimeName.length && runtimeName !in serialTypes) serialTypes[runtimeName] = st; } } static void RegisterSerializer(T : Object)(T obj) { RegisterSerializer!T(typeid(obj).name); } ``` but I'd rather not have to instantiate an actual object just to get its typeid().name, I suppose I can just manually construct it from the fullyQualifiedName inserting the parenthesis and appended portion so it matches.
May 03 2022
prev sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote:
 something I can pass to `Object.factory`.
Object.factory is useless and will hopefully be removed someday. Instead, make your own factory registration function. Put a static constructor in the class which appends a factory delegate to an array or something you can use later. Then you can use your own thing to construct registered objects.
May 03 2022
parent reply Arafel <er.krali gmail.com> writes:
On 3/5/22 14:46, Adam D Ruppe wrote:
 Put a static constructor in the class which appends a factory delegate 
 to an array or something you can use later. Then you can use your own 
 thing to construct registered objects.
I'd like to do a runtime registration system myself, using a "template this" static constructor. A simple version supporting only default constructors would be: ```d module test; import std.stdio : writeln; class MyObject { /* static */ this(this T)() { string type = typeid(T).name; if (type !in generators) { generators[type] = () => new T(); } } static MyObject factory(string type) { if(type in generators) { return generators[type](); } else { return null; } } private: static MyObject function()[string] generators; } class MyClass : MyObject { this() { writeln("Creating MyClass"); } } void main() { auto _ = new MyClass(); // Shouldn't be needed auto myClass = MyObject.factory("test.MyClass"); } ``` Unfortunately, this isn't currently possible: https://issues.dlang.org/show_bug.cgi?id=10488 https://issues.dlang.org/show_bug.cgi?id=20277 (notice the big number of duplicates). The closest feasible option is to put it in a non-static constructor, and that's suboptimal: it forces an instantiation of the class, and it will be run at every instantiation. Alternatively, instruct the users to create a static constructor for each of the classes they'd like registered (perhaps through a mixin), but that's also quite cumbersome.
May 03 2022
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 3 May 2022 at 13:25:14 UTC, Arafel wrote:
 I'd like to do a runtime registration system myself, using a 
 "template this" static constructor. A simple version supporting 
 only default constructors would be:
Yeah, you can't template this a static constructor, but you can just use a static constructor. It will have to be mixed into each child class though, and the compiler won't help to remind you. But do something along the lines of: ```d module factory.register; private Object function()[string] factories; Object construct(string name) { if(auto f = name in factories) return (*f)(); return null; } mixin template Register() { static this() { import factory.register; alias This = typeof(this); // bypassing private __traits(getMember, factory.register, "factories") [This.mangleof] = function Object() { // you could even delegate to a static // function if one is present, or pass arguments // etc. this impossible with Object.factory return new This(); }; } } ``` That code is your library. Then, to use it: ```d import factory.register; class MyThing { // you have to remember to do this in each child mixin Register; } void main() { auto t = new MyThing(); // I used the mangle instead of the FQN since it // is easier. Object o = construct(typeof(t).mangleof); MyThing t2 = cast(MyThing) o; assert(t2 !is null); // assert it actually worked } ``` Now, you can extend this a little if you're willing to add an interface too. And if you forget to register the base class, the interface method being not implemented will remind user they did something wrong, and you can runtime assert to check child classes. Check this out: ```d module factory.register; private Object function()[string] factories; Object construct(string name) { if(auto f = name in factories) return (*f)(); return null; } // adding this for the assert bool typeIsRegistered(string name) { return (name in factories) !is null; } // this interface gives runtime access to the info we need interface Serializable { string typeCode() const; } mixin template Register() { // interface implementation override string typeCode() const { // casting away const for more consistent names alias no_const = typeof(cast() this); auto name = no_const.mangleof; // a runtime check to help remind you if something not registered import factory.register; assert(typeIsRegistered(name), "Type "~typeof(this).stringof~" not registered!"); // also making sure the child class was registered // by ensuring the runtime type is the same as the static type assert(typeid(this) == typeid(no_const), "Child class "~typeid(this).toString()~" was not registered!"); return name; } static this() { import factory.register; alias This = typeof(this); // bypassing private __traits(getMember, factory.register, "factories") [This.mangleof] = function Object() { // you could even delegate to a static // function if one is present, or pass arguments // etc. this impossible with Object.factory return new This(); }; } } ``` And the usage: ```d import factory.register; class MyThing : Serializable { mixin Register; } class Child : MyThing { // forgot to register uh oh // mixin Register; } void main() { auto t = new MyThing(); Object o = construct(typeof(t).mangleof); MyThing t2 = cast(MyThing) o; assert(t2 !is null); auto child = new Child(); // if we called this in the serialize function or even one of those constructors' contracts // it can verify things work by triggering the asserts back in the library implementation child.typeCode(); } ``` So doing things yourself gives you some control.
May 03 2022
parent reply Arafel <er.krali gmail.com> writes:
On 3/5/22 15:57, Adam D Ruppe wrote:
 So doing things yourself gives you some control.
Yes, it is indeed possible (I acknowledged it), but I think it's much more cumbersome than it should, and puts the load on the user. If templated this worked in static context (ideally everywhere else too), then we'd be able to implement RTTI in a 100% "pay as you go" way: just inherit from SerializableObject, or perhaps add a mixin to your own root class, and that'd be it. Actually, it would be cool to do it through an interface, although I don't think an interface's static constructors are invoked by the implementing classes... it would be cool, though. And, in one of the bugs, you argue yourself that according to the spec, it *should* work. So please let me just whine... I mean, raise awareness ;-), in case somebody thinks it's interesting and feels brave enough to have a go at it. I'd try it myself, but I wouldn't know where to start. Compiler internals are way beyond my comfort zone...
May 03 2022
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 3 May 2022 at 14:38:53 UTC, Arafel wrote:
 Actually, it would be cool to do it through an interface, 
 although I don't think an interface's static constructors are 
 invoked by the implementing classes... it would be cool, though.
yeah interfaces can't have constructors.
 I'd try it myself, but I wouldn't know where to start. Compiler 
 internals are way beyond my comfort zone...
Believe it or not, you don't need to touch the compiler. Open your druntime's object.d and search for `RTInfo` http://druntime.dpldocs.info/object.RTInfo.html That is instantiated for every user defined type in the program and you have the compile time info..... all druntime uses it for is a tiny bit of GC info and even then only sometimes. But it could do so so so much more. Including doing custom factories and runtime reflection buildups!
May 03 2022
parent Arafel <er.krali gmail.com> writes:
On 3/5/22 16:48, Adam D Ruppe wrote:
 Believe it or not, you don't need to touch the compiler. Open your 
 druntime's object.d and search for `RTInfo`
 
 http://druntime.dpldocs.info/object.RTInfo.html
 
 That is instantiated for every user defined type in the program and you 
 have the compile time info..... all druntime uses it for is a tiny bit 
 of GC info and even then only sometimes.
 
 But it could do so so so much more. Including doing custom factories and 
 runtime reflection buildups!
This looks nice, but I actually meant to allow "template this" in static contexts, as in the bug reports. I think that might indeed need compiler support? You'll make me happy if that's possible without touching the compiler!
May 03 2022
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, May 03, 2022 at 04:38:53PM +0200, Arafel via Digitalmars-d-learn wrote:
 On 3/5/22 15:57, Adam D Ruppe wrote:
 So doing things yourself gives you some control.
Yes, it is indeed possible (I acknowledged it), but I think it's much more cumbersome than it should, and puts the load on the user. If templated this worked in static context (ideally everywhere else too), then we'd be able to implement RTTI in a 100% "pay as you go" way: just inherit from SerializableObject, or perhaps add a mixin to your own root class, and that'd be it. Actually, it would be cool to do it through an interface, although I don't think an interface's static constructors are invoked by the implementing classes... it would be cool, though.
The way I did it in my own serialization code is to use CRTP with static ctors in templated wrapper structs. Namely, replace: class Base { ... } class Derived : Base { ... } with: class Base : Serializable!(Base) { ... } class Derived : Serializable!(Base, Derived) { ... } That's the only thing user code classes need to do. The rest is done in the Serializable proxy base class using compile-time introspection. In a nutshell, what Serializable does is to inject serialize() and deserialize() methods into the class hierarchy. Here's a brief sketch of what it looks like: class Serializable(Base, Derived = Base) : Base { static if (is(Base == Derived)) // this is the base of the hierarchy { // Base class declarations void serialize(...) { ... // use introspection to extract data members } void deserialize(...) { ... // use introspection to reconstitute data members } } else // this is a derived class in the hierarchy { override void serialize(...) { ... // use introspection to extract data members } override void deserialize(...) { ... // use introspection to reconstitute data members } } // How does the deserializer recreate an instance of // Derived? By registering the string name of the class // into a global hash: static struct Proxy // N.B.: this is instantiated for each Derived class { // This static this gets instantiated per // Derived class, and uses compile-time // knowledge about Derived to generate code for // reconstructing an instance of Derived. static this() { deserializers[Derived.stringof] = { auto obj = new Derived(); obj.deserialize(...); return obj; }; } } } // This is module-global. /*shared*/ static Object delegate(...)[string] deserializers; // Global deserialize method that returns an instance of the // class hierarchy. Object deserialize(...) { // Obtain class name from serialized data string classname = ...; // Dispatch to the correct method registered by Proxy's // static this, that recreates the class of the required // type. return deserializers[classname](...); } The nice thing about this approach is that you have full compile-time information about the target type `Derived`, in both the serialization and deserialization methods. So you can use introspection to automate away most of the boilerplate associated with serialization code. E.g., iterate over __traits(allMembers) to extract data fields, inspect UDAs that allow user classes to specify how the serialization should proceed, etc.. T -- Doubtless it is a good thing to have an open mind, but a truly open mind should be open at both ends, like the food-pipe, with the capacity for excretion as well as absorption. -- Northrop Frye
May 03 2022
parent reply cc <cc nevernet.com> writes:
On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
 	class Base : Serializable!(Base) { ... }
 	class Derived : Serializable!(Base, Derived) { ... }
This is really interesting syntax, I'm surprised that works!
May 03 2022
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, May 03, 2022 at 04:38:23PM +0000, cc via Digitalmars-d-learn wrote:
 On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
 	class Base : Serializable!(Base) { ... }
 	class Derived : Serializable!(Base, Derived) { ... }
This is really interesting syntax, I'm surprised that works!
https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern It's a bit counterintuitive at first, but once you "get" how it works, it's an extremely powerful technique for leveraging D's compile-time introspection capabilities. And translating compile-time information into runtime using static this(). ;-) T -- Never step over a puddle, always step around it. Chances are that whatever made it is still dripping.
May 03 2022
parent reply cc <cc nevernet.com> writes:
On Tuesday, 3 May 2022 at 16:51:33 UTC, H. S. Teoh wrote:
 On Tue, May 03, 2022 at 04:38:23PM +0000, cc via 
 Digitalmars-d-learn wrote:
 On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
 	class Base : Serializable!(Base) { ... }
 	class Derived : Serializable!(Base, Derived) { ... }
This is really interesting syntax, I'm surprised that works!
https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern It's a bit counterintuitive at first, but once you "get" how it works, it's an extremely powerful technique for leveraging D's compile-time introspection capabilities. And translating compile-time information into runtime using static this(). ;-) T
Hm although I am having trouble with that particular implementation: ```d class Base : Serializable!(Base) {} class Derived : Serializable!(Base, Derived) {} class Serializable(Base, Derived = Base) : Base {} ``` ``` Error: class `test.Base` circular inheritance Error: template instance `test.Serializable!(Base, Base)` error instantiating ```
May 03 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, May 03, 2022 at 04:59:42PM +0000, cc via Digitalmars-d-learn wrote:
 On Tuesday, 3 May 2022 at 16:51:33 UTC, H. S. Teoh wrote:
[...]
 On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
 	class Base : Serializable!(Base) { ... }
 	class Derived : Serializable!(Base, Derived) { ... }
[...]
 Hm although I am having trouble with that particular implementation:
 ```d
 class Base : Serializable!(Base) {}
 class Derived : Serializable!(Base, Derived) {}
 class Serializable(Base, Derived = Base) : Base {}
 ```
 ```
 Error: class `test.Base` circular inheritance
 Error: template instance `test.Serializable!(Base, Base)` error
 instantiating
 ```
Oops, sorry, I made a mistake. The definition of Serializable should be: class Serializable(Base, Derived = Object) : Base {} and the corresponding static if in the implementation should test for Object instead of Base. Basically, it's just some way of differentiating the base class from the derived classes, because you need to declare the serialize methods in the base class without `override` but in the derived classes you need `override`. T -- Turning your clock 15 minutes ahead won't cure lateness---you're just making time go faster!
May 03 2022
parent reply cc <cc nevernet.com> writes:
On Tuesday, 3 May 2022 at 17:05:09 UTC, H. S. Teoh wrote:
 Oops, sorry, I made a mistake. The definition of Serializable 
 should be:

 	class Serializable(Base, Derived = Object) : Base {}
There we go, works with this, now I get what it's trying to do: ```d class Serializable(Base, Derived = Object) : Derived { ``` What's the purpose of the `static struct Proxy`? The `static this()` seems to work without being enclosed in a structure.
May 03 2022
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, May 03, 2022 at 05:25:06PM +0000, cc via Digitalmars-d-learn wrote:
 On Tuesday, 3 May 2022 at 17:05:09 UTC, H. S. Teoh wrote:
 Oops, sorry, I made a mistake. The definition of Serializable should be:
 
 	class Serializable(Base, Derived = Object) : Base {}
There we go, works with this, now I get what it's trying to do: ```d class Serializable(Base, Derived = Object) : Derived { ``` What's the purpose of the `static struct Proxy`? The `static this()` seems to work without being enclosed in a structure.
Actually, come to think of it, Proxy isn't strictly necessary. You could move the static this into Serializable and it'd work. The main purpose here is to create a separate instance of static this() per instantiation of Serializable, i.e., there'd be a separate instance of static this() for each Derived class. Since the compiler collects all static this()'s into a list of functions that run at program startup, this allows us to initialize the deserializers AA with functions that understand how to create an instance of Derived at runtime. Since in the scope of the static this we have direct access to compile-time information about Derived, it can use compile-time introspection to inspect Derived and take the appropriate action at runtime based on this compile-time knowledge. In this way, we "translate" compile-time knowledge into runtime knowledge. T -- PNP = Plug 'N' Pray
May 03 2022
prev sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 3 May 2022 at 16:38:23 UTC, cc wrote:
 This is really interesting syntax, I'm surprised that works!
Can read a little more on my blog about it: http://dpldocs.info/this-week-in-d/Blog.Posted_2019_06_10.html#tip-of-the-week pretty cool little pattern.
May 03 2022