www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ClassInfo

reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
So I am working on persistence, and being a newbie, its tough. I have 
this method:

void Set(Object o) {
	file.Stream.writeBlock(cast(ubyte*)o, o.classinfo.init.length);
}

One specific question I have is, is classinfo populated with the info 
for the class that the object really is? For example:

MyObject m = new MyObject;
ObjectDatabase o = Dope.Open("filename");
o.Set(m);

Of course I would like to have it work that way. Any tips, pointers, 
smacks on the head for being an idiot are welcome!

Kyle
Apr 28 2005
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
news:d4scbl$1ik8$1 digitaldaemon.com...
 One specific question I have is, is classinfo populated with the info for 
 the class that the object really is? For example:

Yep.
Apr 29 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Jarrett Billingsley wrote:
 "Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
 news:d4scbl$1ik8$1 digitaldaemon.com...
 
One specific question I have is, is classinfo populated with the info for 
the class that the object really is? For example:

Yep.

So the classinfo.init.length property is the current size of the object in memory?
Apr 29 2005
parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Kyle Furlong wrote:
 Jarrett Billingsley wrote:
 
 "Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
 news:d4scbl$1ik8$1 digitaldaemon.com...

 One specific question I have is, is classinfo populated with the info 
 for the class that the object really is? For example:

Yep.

So the classinfo.init.length property is the current size of the object in memory?

But... the object doesn't change it's size in memory... it's data can be allocated dynamically, yet init.length doesn't change because only pointers/references are assigned some values. Note that you wouldn't like to serialize the object just plain writing its bytes to a file. Sure, you'd store some data, but whilst loading it in another program run, you'd load pointers/refernces that would no longer be valid. You might take a look at the Mango library, which has some serialization support. I've written my own lib, yet it needs some readme/tutorial... -- Tomasz Stachowiak /+ a.k.a. h3r3tic +/
Apr 29 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Tom S wrote:
 Kyle Furlong wrote:
 
 Jarrett Billingsley wrote:

 "Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
 news:d4scbl$1ik8$1 digitaldaemon.com...

 One specific question I have is, is classinfo populated with the 
 info for the class that the object really is? For example:

Yep.

So the classinfo.init.length property is the current size of the object in memory?

But... the object doesn't change it's size in memory... it's data can be allocated dynamically, yet init.length doesn't change because only pointers/references are assigned some values. Note that you wouldn't like to serialize the object just plain writing its bytes to a file. Sure, you'd store some data, but whilst loading it in another program run, you'd load pointers/refernces that would no longer be valid. You might take a look at the Mango library, which has some serialization support. I've written my own lib, yet it needs some readme/tutorial...

I have looked at mango, but it seems that Kris only provides the IPickle interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code? Kyle
Apr 29 2005
next sibling parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Kyle, I'm afraid this is what I meant when I talked about "reflection" 
and "serialization" and how D didn't support them like Java and C#.  You 
are going to have a real fun time making it completely transparent, as I 
don't know that you can.

Now, if classes are self-aware and can serialize/unserialize itself with 
methods written for each class, that's more than possible.  But if you 
want to automate that part transparently... well, that's specifically 
why some languages have reflection.

Your method would work fine for any class that had no arrays, pointers, 
or other classes as members.  Failing that, you'd have to "find" the 
pointers in a way much the same as garbage collecting, I suppose, and 
serialize those too.

-[Unknown]


 I have looked at mango, but it seems that Kris only provides the IPickle 
  interface, and leaves it to the user to implement the actually 
 serializing code. I want to make a lib which can take any object passed 
 to it without knowing what class it is, push it to a file, and be able 
 to reconstitute it. I understand that the pointers wont be valid accross 
 sessions, but there should be a way to substitute virtual pointers when 
 writing to file, so that when trying to deserialize, you would take the 
 root object, follow any of your "virtual pointers" to other objects 
 stored, construct those objects in the same way, and pass back the 
 "real" pointer to the newly created object to the root object. Does that 
 scheme sound plausible? What I am looking for is something other than a 
 class specific implementation, where you for example write a serialize 
 method for each class. Maybe this isnt possible in D currently, but any 
 other pointers anyone has, feel free to speak out. BTW Tom, what does 
 your library actually do? Could you post some sample code?
 
 Kyle

Apr 29 2005
parent Kyle Furlong <ky220 umail.ucsb.edu> writes:
Unknown W. Brackets wrote:
 Kyle, I'm afraid this is what I meant when I talked about "reflection" 
 and "serialization" and how D didn't support them like Java and C#.  You 
 are going to have a real fun time making it completely transparent, as I 
 don't know that you can.
 
 Now, if classes are self-aware and can serialize/unserialize itself with 
 methods written for each class, that's more than possible.  But if you 
 want to automate that part transparently... well, that's specifically 
 why some languages have reflection.
 
 Your method would work fine for any class that had no arrays, pointers, 
 or other classes as members.  Failing that, you'd have to "find" the 
 pointers in a way much the same as garbage collecting, I suppose, and 
 serialize those too.
 
 -[Unknown]
 
 
 I have looked at mango, but it seems that Kris only provides the 
 IPickle  interface, and leaves it to the user to implement the 
 actually serializing code. I want to make a lib which can take any 
 object passed to it without knowing what class it is, push it to a 
 file, and be able to reconstitute it. I understand that the pointers 
 wont be valid accross sessions, but there should be a way to 
 substitute virtual pointers when writing to file, so that when trying 
 to deserialize, you would take the root object, follow any of your 
 "virtual pointers" to other objects stored, construct those objects in 
 the same way, and pass back the "real" pointer to the newly created 
 object to the root object. Does that scheme sound plausible? What I am 
 looking for is something other than a class specific implementation, 
 where you for example write a serialize method for each class. Maybe 
 this isnt possible in D currently, but any other pointers anyone has, 
 feel free to speak out. BTW Tom, what does your library actually do? 
 Could you post some sample code?

 Kyle


For the scheme I described above to work, each class would have to provide a method to tell the library which fields are which types. I can get at the fields in memory, but I cant tell what type they are. So you are right, it cannot be completely transparent without some outside metadata. Kyle
Apr 30 2005
prev sibling next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 But... the object doesn't change it's size in memory... it's data can be 
 allocated dynamically, yet init.length doesn't change because only 
 pointers/references are assigned some values.
 Note that you wouldn't like to serialize the object just plain writing 
 its bytes to a file. Sure, you'd store some data, but whilst loading it 
 in another program run, you'd load pointers/refernces that would no 
 longer be valid.
 You might take a look at the Mango library, which has some serialization 
 support.
 I've written my own lib, yet it needs some readme/tutorial...

I have looked at mango, but it seems that Kris only provides the IPickle interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code? Kyle

One of the tools I want to eventually use the "DMD front end starter kit" for was a program (I've been calling it "deflect") that takes a class source file (say A) and generates a "metadata" class output source (for say MetaA) that can be compiled and linked with A so that code that wants to serialize and deserialize A can ask MetaA to do it. A meta class is a bit like the ClassInfo class but it has more hooks - eg the entire reflection API in Java would be possible. You can ask MetaA for the fields in A, their offsets, their types etc etc. If you are feeling ambitious you can grab the DMD Front End starter kit from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. Note that such a program would do more than support serialization so it might be too much for your particular needs. -Ben
Apr 29 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Ben Hinkle wrote:
But... the object doesn't change it's size in memory... it's data can be 
allocated dynamically, yet init.length doesn't change because only 
pointers/references are assigned some values.
Note that you wouldn't like to serialize the object just plain writing 
its bytes to a file. Sure, you'd store some data, but whilst loading it 
in another program run, you'd load pointers/refernces that would no 
longer be valid.
You might take a look at the Mango library, which has some serialization 
support.
I've written my own lib, yet it needs some readme/tutorial...

I have looked at mango, but it seems that Kris only provides the IPickle interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code? Kyle

One of the tools I want to eventually use the "DMD front end starter kit" for was a program (I've been calling it "deflect") that takes a class source file (say A) and generates a "metadata" class output source (for say MetaA) that can be compiled and linked with A so that code that wants to serialize and deserialize A can ask MetaA to do it. A meta class is a bit like the ClassInfo class but it has more hooks - eg the entire reflection API in Java would be possible. You can ask MetaA for the fields in A, their offsets, their types etc etc. If you are feeling ambitious you can grab the DMD Front End starter kit from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. Note that such a program would do more than support serialization so it might be too much for your particular needs. -Ben

I saw dmdfe, but was unsure how to incorporate it, could you give examples, or instructions on how to go about getting at that meta class you talk about? Like I said, I am very new to the C/D paradigm, growing up as a programmer in Java/C#. Kyle
Apr 30 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 One of the tools I want to eventually use the "DMD front end starter kit" 
 for was a program (I've been calling it "deflect") that takes a class 
 source file (say A) and generates a "metadata" class output source (for 
 say MetaA) that can be compiled and linked with A so that code that wants 
 to serialize and deserialize A can ask MetaA to do it. A meta class is a 
 bit like the ClassInfo class but it has more hooks - eg the entire 
 reflection API in Java would be possible. You can ask MetaA for the 
 fields in A, their offsets, their types etc etc.
 If you are feeling ambitious you can grab the DMD Front End starter kit 
 from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. 
 Note that such a program would do more than support serialization so it 
 might be too much for your particular needs.

I saw dmdfe, but was unsure how to incorporate it, could you give examples, or instructions on how to go about getting at that meta class you talk about? Like I said, I am very new to the C/D paradigm, growing up as a programmer in Java/C#.

The metadata is stored in the parse tree created in memory while reading in the source file. That is what the dmd front end is for. The dmd back end (which Walter has but not dmdfe) takes that tree and generates code and object files. What I'm suggesting is taking that tree and generating source code that would allow D programmers to query and manipulate classes and/or class instances. So there would be alot of work to do just to get to the point of writing the class info to a file much less a D source file that obeys some API. My guess is that given your D experience it would be too much unless you have experience with compiler internals (which I don't).
Apr 30 2005
parent Kyle Furlong <ky220 umail.ucsb.edu> writes:
Ben Hinkle wrote:
One of the tools I want to eventually use the "DMD front end starter kit" 
for was a program (I've been calling it "deflect") that takes a class 
source file (say A) and generates a "metadata" class output source (for 
say MetaA) that can be compiled and linked with A so that code that wants 
to serialize and deserialize A can ask MetaA to do it. A meta class is a 
bit like the ClassInfo class but it has more hooks - eg the entire 
reflection API in Java would be possible. You can ask MetaA for the 
fields in A, their offsets, their types etc etc.
If you are feeling ambitious you can grab the DMD Front End starter kit 
from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. 
Note that such a program would do more than support serialization so it 
might be too much for your particular needs.

I saw dmdfe, but was unsure how to incorporate it, could you give examples, or instructions on how to go about getting at that meta class you talk about? Like I said, I am very new to the C/D paradigm, growing up as a programmer in Java/C#.

The metadata is stored in the parse tree created in memory while reading in the source file. That is what the dmd front end is for. The dmd back end (which Walter has but not dmdfe) takes that tree and generates code and object files. What I'm suggesting is taking that tree and generating source code that would allow D programmers to query and manipulate classes and/or class instances. So there would be alot of work to do just to get to the point of writing the class info to a file much less a D source file that obeys some API. My guess is that given your D experience it would be too much unless you have experience with compiler internals (which I don't).

Gonna need more than that. I dont even know what to do with these stubs. When to they get called, what data is available when they do, etc. etc. I really am a newbie.
Apr 30 2005
prev sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Kyle Furlong wrote:
 BTW Tom, what does 
 your library actually do? Could you post some sample code?
 
 Kyle

I guess it's not what you are looking for, since the user of my lib has to implement a 'describe' function to enable serialization, e.g.: # void describe(Serializer s) # { # super.describe(s); # s.str!(Texture)(diffuseTex); # s.str!(Texture)(lightmapTex); # } or # void describe(Serializer s) # { # if (Serializer.SMode.R == s.mode) # { # ISceneNode[] chLst; # s.objArr!(ISceneNode)(chLst); # # foreach(ISceneNode ch; chLst) # attachChild(ch); # # delete chLst; # } # else # { # s.objArr!(ISceneNode)(m_children); # } # # s.arr!(char)(m_name); # } or # void describe(Serializer s) # { # s.strArr!(vec3)(vertices); # s.strArr!(vec3)(normals); # s.strArr2!(vec2)(texCoords); # s.strArr!(vec3ub)(colors); # s.arr!(uint)(indices); # s.obj!(Effect)(effect); # } And classes have to be registered in a factory: # static this() # { # new factory.type!(DirectionalLight); # new factory.type!(PointLight); # } Yet for simple structs it's enough to use a mixin: # mixin Serializer.RawStruct; ... and it's capable of serializing whole complicated object hierarchies just like this: # ISceneNode level = Pickle!(ISceneNode).read("level.sg"); # Pickle!(ISceneNode).read("optimizedLevel.sg").setParent(root); Note: Circular referencing classes will serialize just fine, as class instances are serialized just once. But this is only implemented for classes. If you have structs that recursively reference each other, you'd get a nice infinite loop. I didn't need that functionality yet, but it's gonna get there. The API is probably also going to change. Anyways, if you want to take a look ( to get some ideas or find out how not to do it ;) ), the sources are here: http://codeinsane.info/listDir.spy?viewType=normal&path=code/heresy The file is called serializer.d but I've given you a link to a wider code base because serializer.d is dependent on some other modules. -- Tomasz Stachowiak /+ a.k.a. h3r3tic +/
Apr 30 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Tom S wrote:
 Kyle Furlong wrote:
 
 BTW Tom, what does your library actually do? Could you post some 
 sample code?

 Kyle

I guess it's not what you are looking for, since the user of my lib has to implement a 'describe' function to enable serialization, e.g.: # void describe(Serializer s) # { # super.describe(s); # s.str!(Texture)(diffuseTex); # s.str!(Texture)(lightmapTex); # } or # void describe(Serializer s) # { # if (Serializer.SMode.R == s.mode) # { # ISceneNode[] chLst; # s.objArr!(ISceneNode)(chLst); # # foreach(ISceneNode ch; chLst) # attachChild(ch); # # delete chLst; # } # else # { # s.objArr!(ISceneNode)(m_children); # } # # s.arr!(char)(m_name); # } or # void describe(Serializer s) # { # s.strArr!(vec3)(vertices); # s.strArr!(vec3)(normals); # s.strArr2!(vec2)(texCoords); # s.strArr!(vec3ub)(colors); # s.arr!(uint)(indices); # s.obj!(Effect)(effect); # } And classes have to be registered in a factory: # static this() # { # new factory.type!(DirectionalLight); # new factory.type!(PointLight); # } Yet for simple structs it's enough to use a mixin: # mixin Serializer.RawStruct; ... and it's capable of serializing whole complicated object hierarchies just like this: # ISceneNode level = Pickle!(ISceneNode).read("level.sg"); # Pickle!(ISceneNode).read("optimizedLevel.sg").setParent(root); Note: Circular referencing classes will serialize just fine, as class instances are serialized just once. But this is only implemented for classes. If you have structs that recursively reference each other, you'd get a nice infinite loop. I didn't need that functionality yet, but it's gonna get there. The API is probably also going to change. Anyways, if you want to take a look ( to get some ideas or find out how not to do it ;) ), the sources are here: http://codeinsane.info/listDir.spy?viewType=normal&path=code/heresy The file is called serializer.d but I've given you a link to a wider code base because serializer.d is dependent on some other modules.

I think that for the meantime I would settle for the functionality you provide? Will you allow others to use your code? Let me know.
Apr 30 2005
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Kyle Furlong wrote:
 I think that for the meantime I would settle for the functionality you 
 provide? Will you allow others to use your code? Let me know.

This code is public domain, you're welcome to use it. -- Tomasz Stachowiak /+ a.k.a. h3r3tic +/
Apr 30 2005