www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ClassInfo

reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
So I am working on persistence, and being a newbie, its tough. I have 
this method:

void Set(Object o) {
	file.Stream.writeBlock(cast(ubyte*)o, o.classinfo.init.length);
}

One specific question I have is, is classinfo populated with the info 
for the class that the object really is? For example:

MyObject m = new MyObject;
ObjectDatabase o = Dope.Open("filename");
o.Set(m);

Of course I would like to have it work that way. Any tips, pointers, 
smacks on the head for being an idiot are welcome!

Kyle
Apr 28 2005
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
news:d4scbl$1ik8$1 digitaldaemon.com...

 One specific question I have is, is classinfo populated with the info for 
 the class that the object really is? For example:


Yep.
Apr 29 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Jarrett Billingsley wrote:

 "Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
 news:d4scbl$1ik8$1 digitaldaemon.com...
 

One specific question I have is, is classinfo populated with the info for 
the class that the object really is? For example:


 
 
 Yep. 
 
 


So the classinfo.init.length property is the current size of the object 
in memory?
Apr 29 2005
parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Kyle Furlong wrote:

 Jarrett Billingsley wrote:
 

 "Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
 news:d4scbl$1ik8$1 digitaldaemon.com...


 One specific question I have is, is classinfo populated with the info 
 for the class that the object really is? For example:




 Yep.


 
 So the classinfo.init.length property is the current size of the object 
 in memory?


But... the object doesn't change it's size in memory... it's data can be 
allocated dynamically, yet init.length doesn't change because only 
pointers/references are assigned some values.
Note that you wouldn't like to serialize the object just plain writing 
its bytes to a file. Sure, you'd store some data, but whilst loading it 
in another program run, you'd load pointers/refernces that would no 
longer be valid.
You might take a look at the Mango library, which has some serialization 
support.
I've written my own lib, yet it needs some readme/tutorial...

-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/
Apr 29 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Tom S wrote:

 Kyle Furlong wrote:
 

 Jarrett Billingsley wrote:


 "Kyle Furlong" <ky220 umail.ucsb.edu> wrote in message 
 news:d4scbl$1ik8$1 digitaldaemon.com...


 One specific question I have is, is classinfo populated with the 
 info for the class that the object really is? For example:





 Yep.


 So the classinfo.init.length property is the current size of the 
 object in memory?


 
 
 But... the object doesn't change it's size in memory... it's data can be 
 allocated dynamically, yet init.length doesn't change because only 
 pointers/references are assigned some values.
 Note that you wouldn't like to serialize the object just plain writing 
 its bytes to a file. Sure, you'd store some data, but whilst loading it 
 in another program run, you'd load pointers/refernces that would no 
 longer be valid.
 You might take a look at the Mango library, which has some serialization 
 support.
 I've written my own lib, yet it needs some readme/tutorial...
 


I have looked at mango, but it seems that Kris only provides the IPickle 
  interface, and leaves it to the user to implement the actually 
serializing code. I want to make a lib which can take any object passed 
to it without knowing what class it is, push it to a file, and be able 
to reconstitute it. I understand that the pointers wont be valid accross 
sessions, but there should be a way to substitute virtual pointers when 
writing to file, so that when trying to deserialize, you would take the 
root object, follow any of your "virtual pointers" to other objects 
stored, construct those objects in the same way, and pass back the 
"real" pointer to the newly created object to the root object. Does that 
scheme sound plausible? What I am looking for is something other than a 
class specific implementation, where you for example write a serialize 
method for each class. Maybe this isnt possible in D currently, but any 
other pointers anyone has, feel free to speak out. BTW Tom, what does 
your library actually do? Could you post some sample code?

Kyle
Apr 29 2005
next sibling parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Kyle, I'm afraid this is what I meant when I talked about "reflection" 

are going to have a real fun time making it completely transparent, as I 
don't know that you can.

Now, if classes are self-aware and can serialize/unserialize itself with 
methods written for each class, that's more than possible.  But if you 
want to automate that part transparently... well, that's specifically 
why some languages have reflection.

Your method would work fine for any class that had no arrays, pointers, 
or other classes as members.  Failing that, you'd have to "find" the 
pointers in a way much the same as garbage collecting, I suppose, and 
serialize those too.

-[Unknown]



 I have looked at mango, but it seems that Kris only provides the IPickle 
  interface, and leaves it to the user to implement the actually 
 serializing code. I want to make a lib which can take any object passed 
 to it without knowing what class it is, push it to a file, and be able 
 to reconstitute it. I understand that the pointers wont be valid accross 
 sessions, but there should be a way to substitute virtual pointers when 
 writing to file, so that when trying to deserialize, you would take the 
 root object, follow any of your "virtual pointers" to other objects 
 stored, construct those objects in the same way, and pass back the 
 "real" pointer to the newly created object to the root object. Does that 
 scheme sound plausible? What I am looking for is something other than a 
 class specific implementation, where you for example write a serialize 
 method for each class. Maybe this isnt possible in D currently, but any 
 other pointers anyone has, feel free to speak out. BTW Tom, what does 
 your library actually do? Could you post some sample code?
 
 Kyle
Apr 29 2005
parent Kyle Furlong <ky220 umail.ucsb.edu> writes:
Unknown W. Brackets wrote:

 Kyle, I'm afraid this is what I meant when I talked about "reflection" 

 are going to have a real fun time making it completely transparent, as I 
 don't know that you can.
 
 Now, if classes are self-aware and can serialize/unserialize itself with 
 methods written for each class, that's more than possible.  But if you 
 want to automate that part transparently... well, that's specifically 
 why some languages have reflection.
 
 Your method would work fine for any class that had no arrays, pointers, 
 or other classes as members.  Failing that, you'd have to "find" the 
 pointers in a way much the same as garbage collecting, I suppose, and 
 serialize those too.
 
 -[Unknown]
 
 

 I have looked at mango, but it seems that Kris only provides the 
 IPickle  interface, and leaves it to the user to implement the 
 actually serializing code. I want to make a lib which can take any 
 object passed to it without knowing what class it is, push it to a 
 file, and be able to reconstitute it. I understand that the pointers 
 wont be valid accross sessions, but there should be a way to 
 substitute virtual pointers when writing to file, so that when trying 
 to deserialize, you would take the root object, follow any of your 
 "virtual pointers" to other objects stored, construct those objects in 
 the same way, and pass back the "real" pointer to the newly created 
 object to the root object. Does that scheme sound plausible? What I am 
 looking for is something other than a class specific implementation, 
 where you for example write a serialize method for each class. Maybe 
 this isnt possible in D currently, but any other pointers anyone has, 
 feel free to speak out. BTW Tom, what does your library actually do? 
 Could you post some sample code?

 Kyle




For the scheme I described above to work, each class would have to 
provide a method to tell the library which fields are which types. I can 
get at the fields in memory, but I cant tell what type they are. So you 
are right, it cannot be completely transparent without some outside 
metadata.

Kyle
Apr 30 2005
prev sibling next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 But... the object doesn't change it's size in memory... it's data can be 
 allocated dynamically, yet init.length doesn't change because only 
 pointers/references are assigned some values.
 Note that you wouldn't like to serialize the object just plain writing 
 its bytes to a file. Sure, you'd store some data, but whilst loading it 
 in another program run, you'd load pointers/refernces that would no 
 longer be valid.
 You might take a look at the Mango library, which has some serialization 
 support.
 I've written my own lib, yet it needs some readme/tutorial...


 I have looked at mango, but it seems that Kris only provides the IPickle 
 interface, and leaves it to the user to implement the actually serializing 
 code. I want to make a lib which can take any object passed to it without 
 knowing what class it is, push it to a file, and be able to reconstitute 
 it. I understand that the pointers wont be valid accross sessions, but 
 there should be a way to substitute virtual pointers when writing to file, 
 so that when trying to deserialize, you would take the root object, follow 
 any of your "virtual pointers" to other objects stored, construct those 
 objects in the same way, and pass back the "real" pointer to the newly 
 created object to the root object. Does that scheme sound plausible? What 
 I am looking for is something other than a class specific implementation, 
 where you for example write a serialize method for each class. Maybe this 
 isnt possible in D currently, but any other pointers anyone has, feel free 
 to speak out. BTW Tom, what does your library actually do? Could you post 
 some sample code?

 Kyle


One of the tools I want to eventually use the "DMD front end starter kit" 
for was a program (I've been calling it "deflect") that takes a class source 
file (say A) and generates a "metadata" class output source (for say MetaA) 
that can be compiled and linked with A so that code that wants to serialize 
and deserialize A can ask MetaA to do it. A meta class is a bit like the 
ClassInfo class but it has more hooks - eg the entire reflection API in Java 
would be possible. You can ask MetaA for the fields in A, their offsets, 
their types etc etc.
If you are feeling ambitious you can grab the DMD Front End starter kit from 
http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. Note that 
such a program would do more than support serialization so it might be too 
much for your particular needs.

-Ben
Apr 29 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Ben Hinkle wrote:

But... the object doesn't change it's size in memory... it's data can be 
allocated dynamically, yet init.length doesn't change because only 
pointers/references are assigned some values.
Note that you wouldn't like to serialize the object just plain writing 
its bytes to a file. Sure, you'd store some data, but whilst loading it 
in another program run, you'd load pointers/refernces that would no 
longer be valid.
You might take a look at the Mango library, which has some serialization 
support.
I've written my own lib, yet it needs some readme/tutorial...


I have looked at mango, but it seems that Kris only provides the IPickle 
interface, and leaves it to the user to implement the actually serializing 
code. I want to make a lib which can take any object passed to it without 
knowing what class it is, push it to a file, and be able to reconstitute 
it. I understand that the pointers wont be valid accross sessions, but 
there should be a way to substitute virtual pointers when writing to file, 
so that when trying to deserialize, you would take the root object, follow 
any of your "virtual pointers" to other objects stored, construct those 
objects in the same way, and pass back the "real" pointer to the newly 
created object to the root object. Does that scheme sound plausible? What 
I am looking for is something other than a class specific implementation, 
where you for example write a serialize method for each class. Maybe this 
isnt possible in D currently, but any other pointers anyone has, feel free 
to speak out. BTW Tom, what does your library actually do? Could you post 
some sample code?

Kyle


 
 
 One of the tools I want to eventually use the "DMD front end starter kit" 
 for was a program (I've been calling it "deflect") that takes a class source 
 file (say A) and generates a "metadata" class output source (for say MetaA) 
 that can be compiled and linked with A so that code that wants to serialize 
 and deserialize A can ask MetaA to do it. A meta class is a bit like the 
 ClassInfo class but it has more hooks - eg the entire reflection API in Java 
 would be possible. You can ask MetaA for the fields in A, their offsets, 
 their types etc etc.
 If you are feeling ambitious you can grab the DMD Front End starter kit from 
 http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. Note that 
 such a program would do more than support serialization so it might be too 
 much for your particular needs.
 
 -Ben 
 
 


I saw dmdfe, but was unsure how to incorporate it, could you give 
examples, or instructions on how to go about getting at that meta class 
you talk about? Like I said, I am very new to the C/D paradigm, growing 


Kyle
Apr 30 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 One of the tools I want to eventually use the "DMD front end starter kit" 
 for was a program (I've been calling it "deflect") that takes a class 
 source file (say A) and generates a "metadata" class output source (for 
 say MetaA) that can be compiled and linked with A so that code that wants 
 to serialize and deserialize A can ask MetaA to do it. A meta class is a 
 bit like the ClassInfo class but it has more hooks - eg the entire 
 reflection API in Java would be possible. You can ask MetaA for the 
 fields in A, their offsets, their types etc etc.
 If you are feeling ambitious you can grab the DMD Front End starter kit 
 from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. 
 Note that such a program would do more than support serialization so it 
 might be too much for your particular needs.


 I saw dmdfe, but was unsure how to incorporate it, could you give 
 examples, or instructions on how to go about getting at that meta class 
 you talk about? Like I said, I am very new to the C/D paradigm, growing up 



The metadata is stored in the parse tree created in memory while reading in 
the source file. That is what the dmd front end is for. The dmd back end 
(which Walter has but not dmdfe) takes that tree and generates code and 
object files. What I'm suggesting is taking that tree and generating source 
code that would allow D programmers to query and manipulate classes and/or 
class instances. So there would be alot of work to do just to get to the 
point of writing the class info to a file much less a D source file that 
obeys some API. My guess is that given your D experience it would be too 
much unless you have experience with compiler internals (which I don't).
Apr 30 2005
parent Kyle Furlong <ky220 umail.ucsb.edu> writes:
Ben Hinkle wrote:

One of the tools I want to eventually use the "DMD front end starter kit" 
for was a program (I've been calling it "deflect") that takes a class 
source file (say A) and generates a "metadata" class output source (for 
say MetaA) that can be compiled and linked with A so that code that wants 
to serialize and deserialize A can ask MetaA to do it. A meta class is a 
bit like the ClassInfo class but it has more hooks - eg the entire 
reflection API in Java would be possible. You can ask MetaA for the 
fields in A, their offsets, their types etc etc.
If you are feeling ambitious you can grab the DMD Front End starter kit 
from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. 
Note that such a program would do more than support serialization so it 
might be too much for your particular needs.


I saw dmdfe, but was unsure how to incorporate it, could you give 
examples, or instructions on how to go about getting at that meta class 
you talk about? Like I said, I am very new to the C/D paradigm, growing up 



 
 
 The metadata is stored in the parse tree created in memory while reading in 
 the source file. That is what the dmd front end is for. The dmd back end 
 (which Walter has but not dmdfe) takes that tree and generates code and 
 object files. What I'm suggesting is taking that tree and generating source 
 code that would allow D programmers to query and manipulate classes and/or 
 class instances. So there would be alot of work to do just to get to the 
 point of writing the class info to a file much less a D source file that 
 obeys some API. My guess is that given your D experience it would be too 
 much unless you have experience with compiler internals (which I don't). 
 
 


Gonna need more than that. I dont even know what to do with these stubs. 
  When to they get called, what data is available when they do, etc. 
etc. I really am a newbie.
Apr 30 2005
prev sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Kyle Furlong wrote:

 BTW Tom, what does 
 your library actually do? Could you post some sample code?
 
 Kyle


I guess it's not what you are looking for, since the user of my lib has
to implement a 'describe' function to enable serialization, e.g.:








or





















or











And classes have to be registered in a factory:







Yet for simple structs it's enough to use a mixin:




... and it's capable of serializing whole complicated object hierarchies
just like this:





Note: Circular referencing classes will serialize just fine, as class 
instances are serialized just once. But this is only implemented for 
classes. If you have structs that recursively reference each other, 
you'd get a nice infinite loop. I didn't need that functionality yet, 
but it's gonna get there. The API is probably also going to change.

Anyways, if you want to take a look ( to get some ideas or find out how 
not to do it ;) ), the sources are here:
http://codeinsane.info/listDir.spy?viewType=normal&path=code/heresy

The file is called serializer.d but I've given you a link to a wider 
code base because serializer.d is dependent on some other modules.

-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/
Apr 30 2005
parent reply Kyle Furlong <ky220 umail.ucsb.edu> writes:
Tom S wrote:

 Kyle Furlong wrote:
 

 BTW Tom, what does your library actually do? Could you post some 
 sample code?

 Kyle


 
 
 I guess it's not what you are looking for, since the user of my lib has
 to implement a 'describe' function to enable serialization, e.g.:
 






 
 or
 



















 
 or
 









 
 And classes have to be registered in a factory:
 





 
 Yet for simple structs it's enough to use a mixin:
 

 
 
 ... and it's capable of serializing whole complicated object hierarchies
 just like this:
 

 

 
 Note: Circular referencing classes will serialize just fine, as class 
 instances are serialized just once. But this is only implemented for 
 classes. If you have structs that recursively reference each other, 
 you'd get a nice infinite loop. I didn't need that functionality yet, 
 but it's gonna get there. The API is probably also going to change.
 
 Anyways, if you want to take a look ( to get some ideas or find out how 
 not to do it ;) ), the sources are here:
 http://codeinsane.info/listDir.spy?viewType=normal&path=code/heresy
 
 The file is called serializer.d but I've given you a link to a wider 
 code base because serializer.d is dependent on some other modules.
 


I think that for the meantime I would settle for the functionality you 
provide? Will you allow others to use your code? Let me know.
Apr 30 2005
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Kyle Furlong wrote:

 I think that for the meantime I would settle for the functionality you 
 provide? Will you allow others to use your code? Let me know.


This code is public domain, you're welcome to use it.


-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/
Apr 30 2005