www.digitalmars.com         C & C++   DMDScript  

D - TypeInfo

reply Burton Radons <loth users.sourceforge.net> writes:
As I mentioned here and described on d.gnu, I'm working on a port of DMD 
to Linux.  The big part of the work is complete, so now I'm playing with 
a more complete TypeInfo/ClassInfo, and I'm wondering what people 
concretely want from introspection.  Specific things you've needed to do 
that have been stymied by C and C++'s opacity, or things from other 
languages with introspection that you've found helpful.
Aug 04 2002
next sibling parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
Some way to get a unique ID for each class, preferrably one that won't
change on a recompile (i.e. based on the class name) and also preferrably
short (32 bits maybe).  For persistence.

A way to look up a classinfo given its unique ID.

Some way to create a new instance of a class given its classinfo.  It would
likely only be able to call a default constructor (one with no parameters).

Given those 3 things, implementing persistence ourselves is easy.  With



well (but it wouldn't be nearly so nice; it'd have to be grafted into each
class via some kind of factory function lookup.  Am I just lazy or what?
The class name should have full scope including module name available so you
can tell classes with the same name in different modules apart from each
other.  And ID collisions could be problematic too, though unlikely.  Even
for a program with 10000 classes, the likelihood of a collision would be low
if the hash is good.

Test for IsA (is instance of A) and IsDerivedFromA (is instance of A or is
instance of class derived from A)

Of course dynamic cast.


of parameters to each function.  But I can live without that.

I'm sure there are some more.

Sean

"Burton Radons" <loth users.sourceforge.net> wrote in message
news:3D4E1F35.5000703 users.sourceforge.net...

 As I mentioned here and described on d.gnu, I'm working on a port of DMD
 to Linux.  The big part of the work is complete, so now I'm playing with
 a more complete TypeInfo/ClassInfo, and I'm wondering what people
 concretely want from introspection.  Specific things you've needed to do
 that have been stymied by C and C++'s opacity, or things from other
 languages with introspection that you've found helpful.
Aug 05 2002
next sibling parent reply "Richard Krehbiel" <rich kastle.com> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:ailcd5$1cfq$1 digitaldaemon.com...

 Some way to get a unique ID for each class, preferrably one that won't
 change on a recompile (i.e. based on the class name) and also preferrably
 short (32 bits maybe).  For persistence.

 A way to look up a classinfo given its unique ID.


What's wrong with the 128-bit GUID concept?



 of parameters to each function.  But I can live without that.


Good.  D isn't a scripting language, it's for generating optimized machine
code.

I still believe that the symbol tables (number of class members, names,
types, etc) should be available to a D programmer using a compile-time macro
language, and I really don't think executables should be burdened with this
stuff.
Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 07:39:34 -0400 "Richard Krehbiel" <rich kastle.com> wrote:


 A way to look up a classinfo given its unique ID.


 
 What's wrong with the 128-bit GUID concept?


Wouldn't it be simplier for the compiler to assign a unique integer to each
class declared in the program? 4 times smaller, and much faster when it
comes to comparison.
Aug 05 2002
next sibling parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
Don't want it to change on recompile (breaks existing data files).  So it
should be some kind of hash on the name.

Sean

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN37473681461331 news.digitalmars.com...

 On Mon, 5 Aug 2002 07:39:34 -0400 "Richard Krehbiel" <rich kastle.com>


wrote:

 A way to look up a classinfo given its unique ID.


 What's wrong with the 128-bit GUID concept?


 Wouldn't it be simplier for the compiler to assign a unique integer to


each

 class declared in the program? 4 times smaller, and much faster when it
 comes to comparison.
Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 10:09:57 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:


 Don't want it to change on recompile (breaks existing data files).  So it
 should be some kind of hash on the name.


Then GUID will probably be a better idea. Never heard of those duplicating.
Aug 05 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
So how (and when) do you generate it?  GUID's have timestamp and local NIC
info.  That means a data file I make on my machine with an exe I compile, if
you have identical source and you recompile and try to read my data file it
won't match.

No good.

A crc32 on the fully explicit scoped name (with dots as separators) would
likely be plenty sufficient.

Sean

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374739687560995 news.digitalmars.com...

 On Mon, 5 Aug 2002 10:09:57 -0700 "Sean L. Palmer"


<seanpalmer earthlink.net>

 wrote:


 Don't want it to change on recompile (breaks existing data files).  So




it

 should be some kind of hash on the name.


 Then GUID will probably be a better idea. Never heard of those


duplicating.
Aug 05 2002
parent reply "Walter" <walter digitalmars.com> writes:
I think odds of a collision on 32 bit values are high enough to make it
unreliable.

"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:ainr9r$1o9e$1 digitaldaemon.com...

 So how (and when) do you generate it?  GUID's have timestamp and local NIC
 info.  That means a data file I make on my machine with an exe I compile,


if

 you have identical source and you recompile and try to read my data file


it

 won't match.

 No good.

 A crc32 on the fully explicit scoped name (with dots as separators) would
 likely be plenty sufficient.
Aug 09 2002
parent Burton Radons <loth users.sourceforge.net> writes:
Walter wrote:


 I think odds of a collision on 32 bit values are high enough to make it
 unreliable.



It's not difficult to provide the full name of a class (with module) the 
first time it's transmitted and then just a handle subsequent times. 
This should be unique in any data space.  This will also include the 
data signature describing how the class was saved, so it's important 
information that need be conveyed in any case, as the reader has to do 
some involved data matching on the initial instance.

The same thing is done with pointers.  First time the pointer's seen on 
this stream we transmit the full data, subsequent times are just a 
handle.  Old hat.
Aug 10 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN37473681461331 news.digitalmars.com...

 Wouldn't it be simplier for the compiler to assign a unique integer to


each

 class declared in the program? 4 times smaller, and much faster when it
 comes to comparison.


Already done, it's the pointer to the classinfo <g>. The downside is this
won't work across dll's or processes.
Aug 09 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com> wrote:


 Already done, it's the pointer to the classinfo <g>. The downside is this
 won't work across dll's or processes.


It also won't work for persistance, because one can't guarantee
that ClassInfo pointer will remain the same on the second run
of the program.
Aug 10 2002
next sibling parent "anderson" <anderson firestar.com.au> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374785206735301 news.digitalmars.com...

 On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com> wrote:


 Already done, it's the pointer to the classinfo <g>. The downside is




this

 won't work across dll's or processes.


 It also won't work for persistance, because one can't guarantee
 that ClassInfo pointer will remain the same on the second run
 of the program.


Exactly. Persistance.

Parhaps there could be a way to allow users to specify numbers for ClassInfo
(or something else) at complie time.

Something else other then (but simular to) ClassInfo, where users could
specify which group it belongs to.

ie

//Belongs to group Agroup and Bgroup
persistant Agroup, Bgroup class somethingA
{
...
}

persistant Agroup class somethingB

...
}

//Inherits this ability
class somethingC : somethingB
{
...
}

Agroup.getID()

A file, which the user could change could be created (automaticly),
something like:
Agroup:



Bgroup:


Which the complie would add to each time new classes are added.  If a class
was removed it would leave it in the listing in case the user wants to
included it latter. When a user transports code, this file would be taken
with it. Of course the users can already do this manually, but it's
difficult when working in groups. You wouldn't always need it and this
version would only need 16 - 32 bits. Often projects need to be combined and
having something standard like this would be a good help.

Anyhow
If we had "static virtual constructors" and the order remained the same at
runtime this would be an easy job for a programmer.
Aug 10 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374785206735301 news.digitalmars.com...

 On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com> wrote:


 Already done, it's the pointer to the classinfo <g>. The downside is




this

 won't work across dll's or processes.


 It also won't work for persistance, because one can't guarantee
 that ClassInfo pointer will remain the same on the second run
 of the program.


You're right. I think GUIDs are the right solution - and they'd mesh
naturally in with COM support, too.
Aug 10 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
GUIDs are a bad solution for aforementioned reasons.  Look at my earlier
post in this thread.  If you go that route, the programmer has to generate
GUID's for every class probably and those GUID's would I guess have to live
in the source code.  If I wanted that I could do it myself.  I'd like to
avoid that PITA in day-to-day work if possible.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:aj3osj$2p6p$1 digitaldaemon.com...

 "Pavel Minayev" <evilone omen.ru> wrote in message
 news:CFN374785206735301 news.digitalmars.com...

 On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com>




wrote:

 Already done, it's the pointer to the classinfo <g>. The downside is




 this

 won't work across dll's or processes.


 It also won't work for persistance, because one can't guarantee
 that ClassInfo pointer will remain the same on the second run
 of the program.


 You're right. I think GUIDs are the right solution - and they'd mesh
 naturally in with COM support, too.
Aug 11 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:aj5669$13bs$1 digitaldaemon.com...

 GUIDs are a bad solution for aforementioned reasons.  Look at my earlier
 post in this thread.  If you go that route, the programmer has to generate
 GUID's for every class probably and those GUID's would I guess have to


live

 in the source code.  If I wanted that I could do it myself.  I'd like to
 avoid that PITA in day-to-day work if possible.


The GUIDs would be generated by default by the compiler, and so for
intra-process stuff, should work fine. Only for cross-process etc. would
they need to be hand generated. The need to support GUIDs to work with COM
isn't going to go away, and 3 schemes coexisting together is too much.
Aug 11 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
For streaming, you 99% of the time want to write a datum to a file, that
will be read by a later run of that same program.  Sometimes a different
program altogether.

I have a hard time imagining a scheme where the compiler could auto generate
GUID's such that they stay the same from one compile to the next, let alone
generate the same GUID on different machines if your friend decides to make
a small change to your program, compile it, then try to read an existing
data file generated by your program on your machine.

Yes, GUID's are necessary for COM.  However, COM is not the entire
programming universe, and I don't see the entire programming industry
standardizing on COM anytime soon.

GUID's are overkill for this problem...  that's the main point I'm trying to
make.  I'd rather see the full string of the class/module name embedded in
my datafiles than have GUID's.

Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!) bits
would do?  Most programs will contain fewer than 100 classes.  Large
programs may have 1000.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:aj6fp2$2bi8$2 digitaldaemon.com...

 "Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
 news:aj5669$13bs$1 digitaldaemon.com...

 GUIDs are a bad solution for aforementioned reasons.  Look at my earlier
 post in this thread.  If you go that route, the programmer has to




generate

 GUID's for every class probably and those GUID's would I guess have to


 live

 in the source code.  If I wanted that I could do it myself.  I'd like to
 avoid that PITA in day-to-day work if possible.


 The GUIDs would be generated by default by the compiler, and so for
 intra-process stuff, should work fine. Only for cross-process etc. would
 they need to be hand generated. The need to support GUIDs to work with COM
 isn't going to go away, and 3 schemes coexisting together is too much.
Aug 11 2002
next sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Sun, 11 Aug 2002 13:58:25 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:


 I have a hard time imagining a scheme where the compiler could auto generate
 GUID's such that they stay the same from one compile to the next, let alone
 generate the same GUID on different machines if your friend decides to make
 a small change to your program, compile it, then try to read an existing
 data file generated by your program on your machine.


Yes, that's the problem.
 

 GUID's are overkill for this problem...  that's the main point I'm trying to
 make.  I'd rather see the full string of the class/module name embedded in
 my datafiles than have GUID's.


So far I don't see any other way but use module.class string. Only it can
guarantee uniqueness.


 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!) bits
 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.


Yes, but there's still a chance that two of those 100 classes might have
the same CRC32 (or whatever hashfunc you decide to use).
Aug 11 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
True re: the hash function.  Since identifiers are only allowed to use a
small subset of valid ANSI characters, a couple bits per character could be
eliminated.  In 32 bits with 6 bits per character you can get only 5
characters with no possibility of loss of information.  For 128 bits you can
get 21 characters.

Ahh well just put the whole string in.  Pays to use short class and module
names I guess.  ;)

Sean

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374801163376736 news.digitalmars.com...

 On Sun, 11 Aug 2002 13:58:25 -0700 "Sean L. Palmer"


<seanpalmer earthlink.net>

 wrote:


 I have a hard time imagining a scheme where the compiler could auto




generate

 GUID's such that they stay the same from one compile to the next, let




alone

 generate the same GUID on different machines if your friend decides to




make

 a small change to your program, compile it, then try to read an existing
 data file generated by your program on your machine.


 Yes, that's the problem.


 GUID's are overkill for this problem...  that's the main point I'm




trying to

 make.  I'd rather see the full string of the class/module name embedded




in

 my datafiles than have GUID's.


 So far I don't see any other way but use module.class string. Only it can
 guarantee uniqueness.


 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!)




bits

 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.


 Yes, but there's still a chance that two of those 100 classes might have
 the same CRC32 (or whatever hashfunc you decide to use).
Aug 13 2002
next sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Tue, 13 Aug 2002 00:36:04 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:


 True re: the hash function.  Since identifiers are only allowed to use a
 small subset of valid ANSI characters, a couple bits per character could be
 eliminated.  In 32 bits with 6 bits per character you can get only 5
 characters with no possibility of loss of information.  For 128 bits you can
 get 21 characters.


And what will you do with Unicode identifiers? =)


 Ahh well just put the whole string in.  Pays to use short class and module
 names I guess.  ;)


Strings can be compressed, after all. Huffman or something.
Aug 13 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374820383640162 news.digitalmars.com...

 On Tue, 13 Aug 2002 00:36:04 -0700 "Sean L. Palmer"


<seanpalmer earthlink.net>

 wrote:


 True re: the hash function.  Since identifiers are only allowed to use a
 small subset of valid ANSI characters, a couple bits per character could




be

 eliminated.  In 32 bits with 6 bits per character you can get only 5
 characters with no possibility of loss of information.  For 128 bits you




can

 get 21 characters.


 And what will you do with Unicode identifiers? =)


Compile them, I hope!!  ;)


 Ahh well just put the whole string in.  Pays to use short class and




module

 names I guess.  ;)


 Strings can be compressed, after all. Huffman or something.


With pregenerated statistics about frequency of occurrences of symbols and
pairs of symbols, they could actually compress quite well.  Probably just
about as fast as calculating a CRC32.  Just as with the original string, the
variable memory requirement is kinda annoying, though.

Sean
Aug 15 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Thu, 15 Aug 2002 01:43:44 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:


 With pregenerated statistics about frequency of occurrences of symbols and
 pairs of symbols, they could actually compress quite well.  Probably just
 about as fast as calculating a CRC32.  Just as with the original string, the
 variable memory requirement is kinda annoying, though.


Since length of D identifiers is not limited in specification, I
guess there is no workaround. We'll have to live with it.
Aug 15 2002
parent "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374836441665856 news.digitalmars.com...

 Since length of D identifiers is not limited in specification, I
 guess there is no workaround. We'll have to live with it.


Unfortunately, I'm stuck a bit with the OMF limitations on name lengths. But
it's fairly large.
Aug 15 2002
prev sibling parent "anderson" <anderson firestar.com.au> writes:
What about, writing info to disk about the present class structure in debug
mode. When the class structure change you simply need to update the list,
which should contain all the class info. To determine what is which, it's
simply a matter of loading the list at initation and passing out the numbers
to the classes. This way you can use the entire 32 bits = 4294967296 entries
per group. When you distribute the code you also distribute the data file. I
mentioned something like this in a pervious post.

However, I would still like this to be a standard in the complier. If I had
time I'd write a standard lib to support it (although it wouldn't be as
effecient as a complie time version).


"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:ajacej$fv8$1 digitaldaemon.com...

 True re: the hash function.  Since identifiers are only allowed to use a
 small subset of valid ANSI characters, a couple bits per character could


be

 eliminated.  In 32 bits with 6 bits per character you can get only 5
 characters with no possibility of loss of information.  For 128 bits you


can

 get 21 characters.

 Ahh well just put the whole string in.  Pays to use short class and module
 names I guess.  ;)

 Sean

 "Pavel Minayev" <evilone omen.ru> wrote in message
 news:CFN374801163376736 news.digitalmars.com...

 On Sun, 11 Aug 2002 13:58:25 -0700 "Sean L. Palmer"


 <seanpalmer earthlink.net>

 wrote:


 I have a hard time imagining a scheme where the compiler could auto




 generate

 GUID's such that they stay the same from one compile to the next, let




 alone

 generate the same GUID on different machines if your friend decides to




 make

 a small change to your program, compile it, then try to read an






existing

 data file generated by your program on your machine.


 Yes, that's the problem.


 GUID's are overkill for this problem...  that's the main point I'm




 trying to

 make.  I'd rather see the full string of the class/module name






embedded

 in

 my datafiles than have GUID's.


 So far I don't see any other way but use module.class string. Only it




can

 guarantee uniqueness.


 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!)




 bits

 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.


 Yes, but there's still a chance that two of those 100 classes might have
 the same CRC32 (or whatever hashfunc you decide to use).
Aug 13 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:aj6in3$2e87$1 digitaldaemon.com...

 For streaming, you 99% of the time want to write a datum to a file, that
 will be read by a later run of that same program.  Sometimes a different
 program altogether.


Yes.


 I have a hard time imagining a scheme where the compiler could auto


generate

 GUID's such that they stay the same from one compile to the next, let


alone

 generate the same GUID on different machines if your friend decides to


make

 a small change to your program, compile it, then try to read an existing
 data file generated by your program on your machine.


Actually, autogenerated GUIDs should deliberately not be the same from
compile to compile. Only if they are hardcoded will they be the same.


 Yes, GUID's are necessary for COM.  However, COM is not the entire
 programming universe, and I don't see the entire programming industry
 standardizing on COM anytime soon.


COM is actually a brilliant idea. The implementation of it went off course
somewhere, but I don't think it was the GUIDs that did that.


 GUID's are overkill for this problem...  that's the main point I'm trying


to

 make.  I'd rather see the full string of the class/module name embedded in
 my datafiles than have GUID's.


The full name doesn't allow for different versions of the same class.


 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!) bits
 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.


I don't think you can guarantee uniqueness with only 32 bits without knowing
every other D class.
Aug 15 2002
next sibling parent reply "anderson" <anderson firestar.com.au> writes:
 I don't think you can guarantee uniqueness with only 32 bits without


knowing

 every other D class.


What about if you retricted it to just a group of classes (put each class in
a group). That way there you'd only need to store unique ids within that
group of classes.

I also suggested a solution (however inefficient) to this in a previous
email, using the class names an saving them to disk between complies. This
method wouldn't rely on compression. However, this technique could be done
by the programmer or in a standard lib.

PS - Is it possible to get the name of a class as a string in C++?
Aug 16 2002
parent Pavel Minayev <evilone omen.ru> writes:
On Fri, 16 Aug 2002 15:15:08 +0800 "anderson" <anderson firestar.com.au> wrote:


 PS - Is it possible to get the name of a class as a string in C++?


typeid(Foo).name()

And I believe it is mangled.
Aug 16 2002
prev sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Thu, 15 Aug 2002 23:22:44 -0700 "Walter" <walter digitalmars.com> wrote:


 Actually, autogenerated GUIDs should deliberately not be the same from
 compile to compile. Only if they are hardcoded will they be the same.


Then, you have a problem. You release version 1.0 of your program,
end-users save documents made by it (and it uses persistence). Then,
you release version 1.1 - and oops! it cannot read old files,
because all the GUIDs were changed during recompile!


be the compiler which does all the dirty work like this, not 
the programmer.

I still don't understand, why not use the name of the class?
What do you mean by "different versions of the class"?
Aug 16 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374844840487037 news.digitalmars.com...

 On Thu, 15 Aug 2002 23:22:44 -0700 "Walter" <walter digitalmars.com>


wrote:

 Actually, autogenerated GUIDs should deliberately not be the same from
 compile to compile. Only if they are hardcoded will they be the same.


 Then, you have a problem. You release version 1.0 of your program,
 end-users save documents made by it (and it uses persistence). Then,
 you release version 1.1 - and oops! it cannot read old files,
 because all the GUIDs were changed during recompile!


 be the compiler which does all the dirty work like this, not
 the programmer.


An alternate solution would be to write a ClassInfo.eq() function - classes
with the same ClassInfo
contents should be the same.


 I still don't understand, why not use the name of the class?
 What do you mean by "different versions of the class"?


Let's say, from version 1.0 to version 1.1, you add a member to the class
without changing the class's name. Now, all binary references compiled with
the old class layout will crash.
Aug 18 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Sun, 18 Aug 2002 22:17:50 -0700 "Walter" <walter digitalmars.com> wrote:


 An alternate solution would be to write a ClassInfo.eq() function - classes
 with the same ClassInfo
 contents should be the same.


But, ClassInfo is practically a name, vtbl pointer, and constructor pointer.
So, it is simplier to store just the name. =)
 

 Let's say, from version 1.0 to version 1.1, you add a member to the class
 without changing the class's name. Now, all binary references compiled with
 the old class layout will crash.


This is "binary incompatibility", and there's just nothing you can do with
that. Only to remember not to break code so it can't read files generated
by earlier versions. After all, object can have some version property,
and check it when restoring itself from file, to catch this kind of errors.
But I don't see how you can _fix_ it.
Aug 19 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374878540987731 news.digitalmars.com...

 This is "binary incompatibility", and there's just nothing you can do with
 that. Only to remember not to break code so it can't read files generated
 by earlier versions. After all, object can have some version property,
 and check it when restoring itself from file, to catch this kind of


errors.

 But I don't see how you can _fix_ it.


At least by comparing the contents of ClassInfo, you'll know when you broke
it <g>.
Aug 19 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 19 Aug 2002 16:35:23 -0700 "Walter" <walter digitalmars.com> wrote:


 At least by comparing the contents of ClassInfo, you'll know when you broke
 it <g>.


 
ClassInfo contains a string (class name), initializer, and a couple of
pointers. It is probably not the best idea to save pointers to a disk
file...
Aug 20 2002
next sibling parent "Martin M. Pedersen" <mmp www.moeller-pedersen.dk> writes:
Hi,

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374884983818403 news.digitalmars.com...

 ClassInfo contains a string (class name), initializer, and a couple of
 pointers. It is probably not the best idea to save pointers to a disk
 file...


Especially not with a garbage collecting language. However, saving pointers
to disk may have its utility. I remember an application I wrote once. It had
a small set of data structures fixed in memory, and a potentially very large
data structure referencing the fixed data structure. The large data
structure overflowed to a temporary file with pointers and all. It worked
very well, and there was no reason it shouldn't, as the temporary file
didn't live any longer than the fixed data structures.

Another programming practice I recall, that requires special attention with
GC is this: Thread A creates a structure on the heap and sends a pointer to
it to thread B over a message queue at OS-level, thereby having a queue with
synchronization for free.

Regards,
Martin M. Pedersen
Aug 20 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374884983818403 news.digitalmars.com...

 On Mon, 19 Aug 2002 16:35:23 -0700 "Walter" <walter digitalmars.com>


wrote:

 At least by comparing the contents of ClassInfo, you'll know when you




broke

 it <g>.


 ClassInfo contains a string (class name), initializer, and a couple of
 pointers. It is probably not the best idea to save pointers to a disk
 file...


ClassInfo will eventually be extended to know about the members and their
offsets. An overloaded eq() for it would be able to follow the pointers and
properly compare the contents.
Aug 20 2002
next sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Tue=2C 20 Aug 2002 11=3A31=3A28 -0700 =22Walter=22
=3Cwalter=40digitalmars=2Ecom=3E wrote=3A

=3E ClassInfo will eventually be extended to know about the members and their
=3E offsets=2E An overloaded eq=28=29 for it would be able to follow the
pointers and
=3E properly compare the contents=2E

But this means that ANY change to interface of class would make
it =22incompatible=22 to any previous version - which doesn't make
sense if I just add a new function=2C or maybe a new property which
isn't persistent=2E

So either it should use class name=2C and let the programmer control
versions=2C or provide some way to indicate whether attribute is
persistant or not=2E Something like this=3A

=09class Mesh
=09{
=09=09Vertex=5B=5D vertices=3B
=09=09Face=5B=5D faces=3B
=09=09=2E=2E=2E
=09=09=2F=2F auto-generated OpenGL display list
=09=09transient GLuint displist=3B=09
=09}

But then it seems like a step to persistance built into the language
=28which would IMO be great!=29=2E

By the way=2C anybody knows of some good reliable way to store object
references into file=2C so they can be retrieved later=3F
Aug 20 2002
next sibling parent "anderson" <anderson firestar.com.au> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374890214802662 news.digitalmars.com...
On Tue, 20 Aug 2002 11:31:28 -0700 "Walter" <walter digitalmars.com> wrote:


 By the way, anybody knows of some good reliable way to store object
 references into file, so they can be retrieved later?


I'm sure this will be way below what your looking for.

--

In a previous project (I couldn't find the source), I built a tree (you
could of course use a hash table) of all the object references (well
pointers in C++) and saved them into the file.

It worked something like this.

To save...

Whenever the program came across a pointer the programmer would call this in
the save method of the object.
X = savePointer(char * Type, Object Pointer);
//Note that at this point I didn't know that you could get the names of
//objects using typeid(Foo).name().

X would return the number location in the tree the object existed. The X is
saved in place where the pointer would be. If the object hadn't been saved
yet, it would be saved on to a tempary store and inserted in the binary
tree.

Finally.
saveAll( ... );
Would save the tree of tree of data and then all the object entries. At the
end of the file the main objects tree references are saved.

To load...

The tree is loaded, then each object in the tree is loaded (calling the
objects load method).
Finally fixPointers is called for each new object (note that at this stage
pointers are actually array indexes to the tree) which forces it to call
getPointer(reference) for each pointer it contains.

Finally all the linkages to the main program are set using
getPointer(reference) again.

Note that each object had a save and load function derived from a base
class.

This technique is  reliable, but slightly complicated. Of course this
technique could probably be improved in D (and probably even in C++). It
would be nice if you could cycle though each property member in a class,
determine if it was a pointer or not, and perform the appropriate action.
That way this technique would only need to be written for the base class and
all objects would inherit the ability to save themselves.
Aug 20 2002
prev sibling parent "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374890214802662 news.digitalmars.com...

But then it seems like a step to persistance built into the language
(which would IMO be great!).


I wanted to do that originally, but gave up on it as too complicated for
now.


By the way, anybody knows of some good reliable way to store object
references into file, so they can be retrieved later?


A technique I've used is to store memory mapped files at fixed addresses.
Aug 22 2002
prev sibling parent Burton Radons <loth users.sourceforge.net> writes:
Walter wrote:


 "Pavel Minayev" <evilone omen.ru> wrote in message 


news:CFN374884983818403 news.digitalmars.com...



 On Mon, 19 Aug 2002 16:35:23 -0700 "Walter"
 <walter digitalmars.com>


 wrote:


 At least by comparing the contents of ClassInfo, you'll know
 when you




 broke


 it <g>.


 ClassInfo contains a string (class name), initializer, and a
 couple of pointers. It is probably not the best idea to save
 pointers to a disk file...


 ClassInfo will eventually be extended to know about the members and
 their offsets. An overloaded eq() for it would be able to follow the


pointers and

 properly compare the contents.


Sigh.  I've been waiting for someone to ask, but you three keep on
circling around it, so this is what I do for my pickling.

The class name is saved, which contains the full name up to the module.
   Then the parent class is saved.  Then the field names and types are
saved.  Everything is done once only.  A small class takes about thirty
bytes.

For loading, we take in that name and find it or not, giving a solid
error message if there's a problem.  Finding the class to load into is
just a name search, as we match each field individually, allowing new
fields to be added or fields to be moved around.  If a field changes
type or is deleted, it throws up, again with a good error message.

This transfer is also done for types.  From then on, the type is
referenced with a single byte, or more if necessary (variable-sized
integers are used everywhere).  Pointers, class references, and arrays
also use these single-transfer handles, so that aliasing is retained.

Array range aliasing is not retained - if an array is a range of another 
array, and they're both saved, the arrays will be made unaliased in loading.



 Aug 21 2002



prev sibling next sibling parent  Pavel Minayev <evilone omen.ru>  writes:


On Mon, 5 Aug 2002 01:26:38 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:


 Test for IsA (is instance of A) and IsDerivedFromA (is instance of A or is
 instance of class derived from A)


Very easy to write even now. Just look at ClassInfo;


 Of course dynamic cast.


All reference casts in D are dynamic.
 



 Aug 05 2002



prev sibling  parent  Pavel Minayev <evilone omen.ru>  writes:


On Mon, 5 Aug 2002 01:26:38 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:


 Test for IsA (is instance of A) and IsDerivedFromA (is instance of A or is
 instance of class derived from A)


These are very easily written by user - just look at ClassInfo. =)
 

 Of course dynamic cast.


All reference casts in D are dynamic.
 



 Aug 05 2002



prev sibling next sibling parent  "Richard Krehbiel" <rich kastle.com>  writes:


"Burton Radons" <loth users.sourceforge.net> wrote in message
news:3D4E1F35.5000703 users.sourceforge.net...

 As I mentioned here and described on d.gnu, I'm working on a port of DMD
 to Linux.  The big part of the work is complete, so now I'm playing with
 a more complete TypeInfo/ClassInfo, and I'm wondering what people
 concretely want from introspection.  Specific things you've needed to do
 that have been stymied by C and C++'s opacity, or things from other
 languages with introspection that you've found helpful.


I for one specifically do NOT want complete introspection; this is a feature
for interpreted (or nearly) languages.  A means for a programmer to label
specific classes as "serializable" might be nice, but I don't want D to
become so unweildy that executables carry around a massive type database.

C++'s RTTI is just about right.  You can ask if a pointer is to a class of a
certain type, or is derived from a certain type.  Run-time support is all
there using the vtable pointer, which object instances must carry around
anyway.

Most of the type info should *stay* in the compiler, and *not* be added to
the executable, except if the programmer asks for it.

--
Richard Krehbiel, Arlington, VA, USA
rich kastle.com (work) or krehbiel3 comcast.net  (personal)



 Aug 05 2002



prev sibling  parent reply Patrick Down <pat codemoon.com>  writes:


Burton Radons <loth users.sourceforge.net> wrote in news:3D4E1F35.5000703
 users.sourceforge.net:


 As I mentioned here and described on d.gnu, I'm working on a port of 


DMD 

 to Linux.  The big part of the work is complete, so now I'm playing 


with 

 a more complete TypeInfo/ClassInfo, and I'm wondering what people 
 concretely want from introspection.  Specific things you've needed to 


do 

 that have been stymied by C and C++'s opacity, or things from other 
 languages with introspection that you've found helpful.
 


For those that are complaining about addtional executable bulk 
I suggest a compile option to excude introspection information.

However I can find use for both introspection of member variables 
and of functions.

Introspection of member variables in useful for lots of things.  
I've used in in Python to build automatic persistance.  I have 
also used to automatically tie database fields to object members.  

There are a couple of reasons why I want to be able to inspect 
objects for functions.  It would provide support similar to Java 
Beans for code building tools.  Second it would make it much
easier to attach D to various scripting engines.

On a personal side I have a library that builds GUIs on the fly 
via a GUI description language.  The code and language have gone 
thru several incarnations.  One of the things I'd like to do in
the D version is reference callback functions from directly 
within the GUI language.

<button text="Ok" OnClick="DoOkClick()" />



 Aug 05 2002



  parent reply Pavel Minayev <evilone omen.ru>  writes:


On Mon, 5 Aug 2002 13:34:38 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:


 Introspection of member variables in useful for lots of things.  
 I've used in in Python to build automatic persistance.  I have 
 also used to automatically tie database fields to object members.  


Python is an interpreted language. D is not.
 

 There are a couple of reasons why I want to be able to inspect 
 objects for functions.  It would provide support similar to Java 
 Beans for code building tools.  Second it would make it much
 easier to attach D to various scripting engines.
 
 On a personal side I have a library that builds GUIs on the fly 
 via a GUI description language.  The code and language have gone 
 thru several incarnations.  One of the things I'd like to do in
 the D version is reference callback functions from directly 
 within the GUI language.
 
 <button text="Ok" OnClick="DoOkClick()" />


This can be done in a better way - look at Delphi for an example.
There, you can declare properties and methods of class as "published",
which means that information about them is saved. So, typically
properties of GUI components like width, height, caption etc are
all declared as published, so GUI designer can see them, and you
declare event handlers as published so symbolic names can later
be used.



 Aug 05 2002



  parent reply Patrick Down <pat codemoon.com>  writes:


Pavel Minayev <evilone omen.ru> wrote in
news:CFN374737995338773 news.digitalmars.com: 


 On Mon, 5 Aug 2002 13:34:38 +0000 (UTC) Patrick Down
 <pat codemoon.com> wrote: 
 

 Introspection of member variables in useful for lots of things.  
 I've used in in Python to build automatic persistance.  I have 
 also used to automatically tie database fields to object members.  


 
 Python is an interpreted language. D is not.


So? Does this mean that D should not have introspection so that
you can do similar things?  I'm not asking for run time modifiable 
code or structures.  I'm just asking to be able to list the names and 
types of members.


 On a personal side I have a library that builds GUIs on the fly 
 via a GUI description language.  The code and language have gone 
 thru several incarnations.  One of the things I'd like to do in
 the D version is reference callback functions from directly 
 within the GUI language.
 
 <button text="Ok" OnClick="DoOkClick()" />


 
 This can be done in a better way - look at Delphi for an example.
 There, you can declare properties and methods of class as "published",
 which means that information about them is saved. So, typically
 properties of GUI components like width, height, caption etc are
 all declared as published, so GUI designer can see them, and you
 declare event handlers as published so symbolic names can later
 be used.


Ok, I can see this as useful but it really just seems like
introspection where the user can turn it on and off at the
function and member level.



 Aug 05 2002



  parent reply Pavel Minayev <evilone omen.ru>  writes:


On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:


 Python is an interpreted language. D is not.


 
 So? Does this mean that D should not have introspection so that
 you can do similar things?  I'm not asking for run time modifiable 
 code or structures.  I'm just asking to be able to list the names and 
 types of members.


.. which adds significant bloat to compiled executable.


 Ok, I can see this as useful but it really just seems like
 introspection where the user can turn it on and off at the
 function and member level.


Yes, it is exactly that. And I like it: where it is really needed, you
turn it on; where it is not, it doesn't waste space. 



 Aug 05 2002



 next sibling parent reply Patrick Down <pat codemoon.com>  writes:


Pavel Minayev <evilone omen.ru> wrote in
news:CFN374739675806829 news.digitalmars.com: 


 On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down
 <pat codemoon.com> wrote: 
 

 Ok, I can see this as useful but it really just seems like
 introspection where the user can turn it on and off at the
 function and member level.


 
 Yes, it is exactly that. And I like it: where it is really needed, you
 turn it on; where it is not, it doesn't waste space. 


Ok, that seems reasonable.  Perhaps you should be able to turn
in on and off at the class and module level too.

Also if you go as far as to add new keyword for member introspection 




 Aug 05 2002



  parent reply Pavel Minayev <evilone omen.ru>  writes:


On Mon, 5 Aug 2002 19:45:39 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:


 Ok, that seems reasonable.  Perhaps you should be able to turn
 in on and off at the class and module level too.


Yes, right.
 

 Also if you go as far as to add new keyword for member introspection 



Could you please remind me about it?



 Aug 05 2002



  parent  Patrick Down <pat codemoon.com>  writes:


Pavel Minayev <evilone omen.ru> wrote in
news:CFN374743695890046 news.digitalmars.com: 


 On Mon, 5 Aug 2002 19:45:39 +0000 (UTC) Patrick Down
 <pat codemoon.com> wrote: 
 

 Ok, that seems reasonable.  Perhaps you should be able to turn
 in on and off at the class and module level too.


 
 Yes, right.
  

 Also if you go as far as to add new keyword for member introspection 



 
 Could you please remind me about it?


I meant attributes, I had to get the book out and
remind myself.  They are extra information available
via reflection besides the name and type information.
The syntax is like this.

[Serializable]
class Foo { }

There are a number of built in attributes but like
Serializable but you can also define your own.  
For example you might attach a human readable name 
for a runtime property browser.

class Foo
{
  [ Serializable, PropName("The yearly interest rate") ]
  public int yearlyRate
  {
    get { return yrRate; }
    set ( yrRate= value; }
  }
}

I'm not necessarily advocating them for D but they




 Aug 06 2002



prev sibling next sibling parent reply "anderson" <anderson firestar.com.au>  writes:


This sounds like a cool idea. This would be useful in saving/loading
objects.

<Babble>
I'd also like it if D provided other faculties for save/loading classes from
file. Some features to aid in quickly save parts of the hierarchies of class
objects to files. Some support in saving objects that have multiple
references and then in restoring their pointer locations on a load. (I've
done that before, but it would be nice to have some sort of support either
in lib form or complier form for it).

-PS Lucky this a digitalmars newgroup and hasn't been attacked by that
newgroup porn thing. What's with that Evil.

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374739675806829 news.digitalmars.com...

 On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down <pat codemoon.com>


wrote:

 Python is an interpreted language. D is not.


 So? Does this mean that D should not have introspection so that
 you can do similar things?  I'm not asking for run time modifiable
 code or structures.  I'm just asking to be able to list the names and
 types of members.


 .. which adds significant bloat to compiled executable.


 Ok, I can see this as useful but it really just seems like
 introspection where the user can turn it on and off at the
 function and member level.


 Yes, it is exactly that. And I like it: where it is really needed, you
 turn it on; where it is not, it doesn't waste space.





 Aug 06 2002



  parent reply "Sean L. Palmer" <seanpalmer earthlink.net>  writes:


I propose a standard "convert to byte array" function (and corresponding
"convert byte array to object" function).  Not sure what the syntax should
be.  Something along the lines of standard conversion to string, just a
binary version.  Actual file I/O isn't the hard part, and
streaming/persistence doesn't always go to a disk file anyway (i.e. message
pipe, network socket, some kind of custom file, compressed file, etc)

Sean

"anderson" <anderson firestar.com.au> wrote in message
news:aioclq$11f4$1 digitaldaemon.com...

 This sounds like a cool idea. This would be useful in saving/loading
 objects.

 <Babble>
 I'd also like it if D provided other faculties for save/loading classes


from

 file. Some features to aid in quickly save parts of the hierarchies of


class

 objects to files. Some support in saving objects that have multiple
 references and then in restoring their pointer locations on a load. (I've
 done that before, but it would be nice to have some sort of support either
 in lib form or complier form for it).

 -PS Lucky this a digitalmars newgroup and hasn't been attacked by that
 newgroup porn thing. What's with that Evil.





 Aug 07 2002



  parent reply "anderson" <anderson firestar.com.au>  writes:


"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:aiqleb$13s0$1 digitaldaemon.com...

 I propose a standard "convert to byte array" function (and corresponding
 "convert byte array to object" function).  Not sure what the syntax should
 be.  Something along the lines of standard conversion to string, just a
 binary version.  Actual file I/O isn't the hard part, and
 streaming/persistence doesn't always go to a disk file anyway (i.e.


message

 pipe, network socket, some kind of custom file, compressed file, etc)

 Sean


That's something along the lines I was considering. I'm don't think I/O is
that hard either, it was mainly the proccess of converting the class object
structure to file. How would you handle pointers?

In C++ you could simply save the entire class (unless you had pointers), in
one go (or convert it to a binary string as easy as pie). So I hope that's
not what your suggesting. However, I don't trust D here, because it has the
power to rearrange members. However that's not my point.

Your right, it should be able to be use it with anything, parhaps even used
for in copy constuctors and overloaded operators. The point is that it would
make coding alot easier if there was some support for dealing with the class
object linkages as a whole.


 "anderson" <anderson firestar.com.au> wrote in message
 news:aioclq$11f4$1 digitaldaemon.com...

 This sounds like a cool idea. This would be useful in saving/loading
 objects.

 <Babble>
 I'd also like it if D provided other faculties for save/loading classes


 from

 file. Some features to aid in quickly save parts of the hierarchies of


 class

 objects to files. Some support in saving objects that have multiple
 references and then in restoring their pointer locations on a load.




(I've

 done that before, but it would be nice to have some sort of support




either

 in lib form or complier form for it).

 -PS Lucky this a digitalmars newgroup and hasn't been attacked by that
 newgroup porn thing. What's with that Evil.







 Aug 07 2002



  parent  "Sean L. Palmer" <seanpalmer earthlink.net>  writes:


Yah, the pointers are a bitch.  ;)  Didn't think of that.

Sean

"anderson" <anderson firestar.com.au> wrote in message
news:air25s$1geh$1 digitaldaemon.com...

 "Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
 news:aiqleb$13s0$1 digitaldaemon.com...

 I propose a standard "convert to byte array" function (and corresponding
 "convert byte array to object" function).  Not sure what the syntax




should

 be.  Something along the lines of standard conversion to string, just a
 binary version.  Actual file I/O isn't the hard part, and
 streaming/persistence doesn't always go to a disk file anyway (i.e.


 message

 pipe, network socket, some kind of custom file, compressed file, etc)

 Sean


 That's something along the lines I was considering. I'm don't think I/O is
 that hard either, it was mainly the proccess of converting the class


object

 structure to file. How would you handle pointers?

 In C++ you could simply save the entire class (unless you had pointers),


in

 one go (or convert it to a binary string as easy as pie). So I hope that's
 not what your suggesting. However, I don't trust D here, because it has


the

 power to rearrange members. However that's not my point.

 Your right, it should be able to be use it with anything, parhaps even


used

 for in copy constuctors and overloaded operators. The point is that it


would

 make coding alot easier if there was some support for dealing with the


class

 object linkages as a whole.





 Aug 07 2002



prev sibling  parent  Burton Radons <loth users.sourceforge.net>  writes:


Pavel Minayev wrote:


 On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:
 
 

Python is an interpreted language. D is not.


So? Does this mean that D should not have introspection so that
you can do similar things?  I'm not asking for run time modifiable 
code or structures.  I'm just asking to be able to list the names and 
types of members.


 
 .. which adds significant bloat to compiled executable.



It's not going to be compile-time optional, so this subthread can die.



 Aug 06 2002