www.digitalmars.com         C & C++   DMDScript  

D - TypeInfo

reply Burton Radons <loth users.sourceforge.net> writes:
As I mentioned here and described on d.gnu, I'm working on a port of DMD 
to Linux.  The big part of the work is complete, so now I'm playing with 
a more complete TypeInfo/ClassInfo, and I'm wondering what people 
concretely want from introspection.  Specific things you've needed to do 
that have been stymied by C and C++'s opacity, or things from other 
languages with introspection that you've found helpful.
Aug 04 2002
next sibling parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
Some way to get a unique ID for each class, preferrably one that won't
change on a recompile (i.e. based on the class name) and also preferrably
short (32 bits maybe).  For persistence.

A way to look up a classinfo given its unique ID.

Some way to create a new instance of a class given its classinfo.  It would
likely only be able to call a default constructor (one with no parameters).

Given those 3 things, implementing persistence ourselves is easy.  With
access to the class name, #1 can be done user side (though it may as well be
done during compile time automatically).  #2 could be done with an
associative array on the user side, and #3 could be implemented user side as
well (but it wouldn't be nearly so nice; it'd have to be grafted into each
class via some kind of factory function lookup.  Am I just lazy or what?
The class name should have full scope including module name available so you
can tell classes with the same name in different modules apart from each
other.  And ID collisions could be problematic too, though unlikely.  Even
for a program with 10000 classes, the likelihood of a collision would be low
if the hash is good.

Test for IsA (is instance of A) and IsDerivedFromA (is instance of A or is
instance of class derived from A)

Of course dynamic cast.

Scripting support is easier if you can examine members and get # and types
of parameters to each function.  But I can live without that.

I'm sure there are some more.

Sean

"Burton Radons" <loth users.sourceforge.net> wrote in message
news:3D4E1F35.5000703 users.sourceforge.net...
 As I mentioned here and described on d.gnu, I'm working on a port of DMD
 to Linux.  The big part of the work is complete, so now I'm playing with
 a more complete TypeInfo/ClassInfo, and I'm wondering what people
 concretely want from introspection.  Specific things you've needed to do
 that have been stymied by C and C++'s opacity, or things from other
 languages with introspection that you've found helpful.

Aug 05 2002
next sibling parent reply "Richard Krehbiel" <rich kastle.com> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:ailcd5$1cfq$1 digitaldaemon.com...
 Some way to get a unique ID for each class, preferrably one that won't
 change on a recompile (i.e. based on the class name) and also preferrably
 short (32 bits maybe).  For persistence.

 A way to look up a classinfo given its unique ID.

What's wrong with the 128-bit GUID concept?
 Scripting support is easier if you can examine members and get # and types
 of parameters to each function.  But I can live without that.

Good. D isn't a scripting language, it's for generating optimized machine code. I still believe that the symbol tables (number of class members, names, types, etc) should be available to a D programmer using a compile-time macro language, and I really don't think executables should be burdened with this stuff.
Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 07:39:34 -0400 "Richard Krehbiel" <rich kastle.com> wrote:

 A way to look up a classinfo given its unique ID.

What's wrong with the 128-bit GUID concept?

Wouldn't it be simplier for the compiler to assign a unique integer to each class declared in the program? 4 times smaller, and much faster when it comes to comparison.
Aug 05 2002
next sibling parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
Don't want it to change on recompile (breaks existing data files).  So it
should be some kind of hash on the name.

Sean

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN37473681461331 news.digitalmars.com...
 On Mon, 5 Aug 2002 07:39:34 -0400 "Richard Krehbiel" <rich kastle.com>

 A way to look up a classinfo given its unique ID.

What's wrong with the 128-bit GUID concept?

Wouldn't it be simplier for the compiler to assign a unique integer to

 class declared in the program? 4 times smaller, and much faster when it
 comes to comparison.

Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 10:09:57 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:

 Don't want it to change on recompile (breaks existing data files).  So it
 should be some kind of hash on the name.

Then GUID will probably be a better idea. Never heard of those duplicating.
Aug 05 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
So how (and when) do you generate it?  GUID's have timestamp and local NIC
info.  That means a data file I make on my machine with an exe I compile, if
you have identical source and you recompile and try to read my data file it
won't match.

No good.

A crc32 on the fully explicit scoped name (with dots as separators) would
likely be plenty sufficient.

Sean

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374739687560995 news.digitalmars.com...
 On Mon, 5 Aug 2002 10:09:57 -0700 "Sean L. Palmer"

 wrote:

 Don't want it to change on recompile (breaks existing data files).  So


 should be some kind of hash on the name.

Then GUID will probably be a better idea. Never heard of those

Aug 05 2002
parent reply "Walter" <walter digitalmars.com> writes:
I think odds of a collision on 32 bit values are high enough to make it
unreliable.

"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:ainr9r$1o9e$1 digitaldaemon.com...
 So how (and when) do you generate it?  GUID's have timestamp and local NIC
 info.  That means a data file I make on my machine with an exe I compile,

 you have identical source and you recompile and try to read my data file

 won't match.

 No good.

 A crc32 on the fully explicit scoped name (with dots as separators) would
 likely be plenty sufficient.

Aug 09 2002
parent Burton Radons <loth users.sourceforge.net> writes:
Walter wrote:

 I think odds of a collision on 32 bit values are high enough to make it
 unreliable.

It's not difficult to provide the full name of a class (with module) the first time it's transmitted and then just a handle subsequent times. This should be unique in any data space. This will also include the data signature describing how the class was saved, so it's important information that need be conveyed in any case, as the reader has to do some involved data matching on the initial instance. The same thing is done with pointers. First time the pointer's seen on this stream we transmit the full data, subsequent times are just a handle. Old hat.
Aug 10 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN37473681461331 news.digitalmars.com...
 Wouldn't it be simplier for the compiler to assign a unique integer to

 class declared in the program? 4 times smaller, and much faster when it
 comes to comparison.

Already done, it's the pointer to the classinfo <g>. The downside is this won't work across dll's or processes.
Aug 09 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com> wrote:

 Already done, it's the pointer to the classinfo <g>. The downside is this
 won't work across dll's or processes.

It also won't work for persistance, because one can't guarantee that ClassInfo pointer will remain the same on the second run of the program.
Aug 10 2002
next sibling parent "anderson" <anderson firestar.com.au> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374785206735301 news.digitalmars.com...
 On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com> wrote:

 Already done, it's the pointer to the classinfo <g>. The downside is


 won't work across dll's or processes.

It also won't work for persistance, because one can't guarantee that ClassInfo pointer will remain the same on the second run of the program.

Exactly. Persistance. Parhaps there could be a way to allow users to specify numbers for ClassInfo (or something else) at complie time. Something else other then (but simular to) ClassInfo, where users could specify which group it belongs to. ie //Belongs to group Agroup and Bgroup persistant Agroup, Bgroup class somethingA { ... } persistant Agroup class somethingB ... } //Inherits this ability class somethingC : somethingB { ... } Agroup.getID() A file, which the user could change could be created (automaticly), something like: Agroup: somethingA #1 somethingB #2 somethingC #3 Bgroup: somethingA #1 Which the complie would add to each time new classes are added. If a class was removed it would leave it in the listing in case the user wants to included it latter. When a user transports code, this file would be taken with it. Of course the users can already do this manually, but it's difficult when working in groups. You wouldn't always need it and this version would only need 16 - 32 bits. Often projects need to be combined and having something standard like this would be a good help. Anyhow If we had "static virtual constructors" and the order remained the same at runtime this would be an easy job for a programmer.
Aug 10 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374785206735301 news.digitalmars.com...
 On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com> wrote:

 Already done, it's the pointer to the classinfo <g>. The downside is


 won't work across dll's or processes.

It also won't work for persistance, because one can't guarantee that ClassInfo pointer will remain the same on the second run of the program.

You're right. I think GUIDs are the right solution - and they'd mesh naturally in with COM support, too.
Aug 10 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
GUIDs are a bad solution for aforementioned reasons.  Look at my earlier
post in this thread.  If you go that route, the programmer has to generate
GUID's for every class probably and those GUID's would I guess have to live
in the source code.  If I wanted that I could do it myself.  I'd like to
avoid that PITA in day-to-day work if possible.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:aj3osj$2p6p$1 digitaldaemon.com...
 "Pavel Minayev" <evilone omen.ru> wrote in message
 news:CFN374785206735301 news.digitalmars.com...
 On Fri, 9 Aug 2002 20:22:17 -0700 "Walter" <walter digitalmars.com>


 Already done, it's the pointer to the classinfo <g>. The downside is


 won't work across dll's or processes.

It also won't work for persistance, because one can't guarantee that ClassInfo pointer will remain the same on the second run of the program.

You're right. I think GUIDs are the right solution - and they'd mesh naturally in with COM support, too.

Aug 11 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:aj5669$13bs$1 digitaldaemon.com...
 GUIDs are a bad solution for aforementioned reasons.  Look at my earlier
 post in this thread.  If you go that route, the programmer has to generate
 GUID's for every class probably and those GUID's would I guess have to

 in the source code.  If I wanted that I could do it myself.  I'd like to
 avoid that PITA in day-to-day work if possible.

The GUIDs would be generated by default by the compiler, and so for intra-process stuff, should work fine. Only for cross-process etc. would they need to be hand generated. The need to support GUIDs to work with COM isn't going to go away, and 3 schemes coexisting together is too much.
Aug 11 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
For streaming, you 99% of the time want to write a datum to a file, that
will be read by a later run of that same program.  Sometimes a different
program altogether.

I have a hard time imagining a scheme where the compiler could auto generate
GUID's such that they stay the same from one compile to the next, let alone
generate the same GUID on different machines if your friend decides to make
a small change to your program, compile it, then try to read an existing
data file generated by your program on your machine.

Yes, GUID's are necessary for COM.  However, COM is not the entire
programming universe, and I don't see the entire programming industry
standardizing on COM anytime soon.

GUID's are overkill for this problem...  that's the main point I'm trying to
make.  I'd rather see the full string of the class/module name embedded in
my datafiles than have GUID's.

Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!) bits
would do?  Most programs will contain fewer than 100 classes.  Large
programs may have 1000.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:aj6fp2$2bi8$2 digitaldaemon.com...
 "Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
 news:aj5669$13bs$1 digitaldaemon.com...
 GUIDs are a bad solution for aforementioned reasons.  Look at my earlier
 post in this thread.  If you go that route, the programmer has to


 GUID's for every class probably and those GUID's would I guess have to

 in the source code.  If I wanted that I could do it myself.  I'd like to
 avoid that PITA in day-to-day work if possible.

The GUIDs would be generated by default by the compiler, and so for intra-process stuff, should work fine. Only for cross-process etc. would they need to be hand generated. The need to support GUIDs to work with COM isn't going to go away, and 3 schemes coexisting together is too much.

Aug 11 2002
next sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Sun, 11 Aug 2002 13:58:25 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:

 I have a hard time imagining a scheme where the compiler could auto generate
 GUID's such that they stay the same from one compile to the next, let alone
 generate the same GUID on different machines if your friend decides to make
 a small change to your program, compile it, then try to read an existing
 data file generated by your program on your machine.

Yes, that's the problem.
 GUID's are overkill for this problem...  that's the main point I'm trying to
 make.  I'd rather see the full string of the class/module name embedded in
 my datafiles than have GUID's.

So far I don't see any other way but use module.class string. Only it can guarantee uniqueness.
 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!) bits
 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.

Yes, but there's still a chance that two of those 100 classes might have the same CRC32 (or whatever hashfunc you decide to use).
Aug 11 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
True re: the hash function.  Since identifiers are only allowed to use a
small subset of valid ANSI characters, a couple bits per character could be
eliminated.  In 32 bits with 6 bits per character you can get only 5
characters with no possibility of loss of information.  For 128 bits you can
get 21 characters.

Ahh well just put the whole string in.  Pays to use short class and module
names I guess.  ;)

Sean

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374801163376736 news.digitalmars.com...
 On Sun, 11 Aug 2002 13:58:25 -0700 "Sean L. Palmer"

 wrote:

 I have a hard time imagining a scheme where the compiler could auto


 GUID's such that they stay the same from one compile to the next, let


 generate the same GUID on different machines if your friend decides to


 a small change to your program, compile it, then try to read an existing
 data file generated by your program on your machine.

Yes, that's the problem.
 GUID's are overkill for this problem...  that's the main point I'm


 make.  I'd rather see the full string of the class/module name embedded


 my datafiles than have GUID's.

So far I don't see any other way but use module.class string. Only it can guarantee uniqueness.
 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!)


 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.

Yes, but there's still a chance that two of those 100 classes might have the same CRC32 (or whatever hashfunc you decide to use).

Aug 13 2002
next sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Tue, 13 Aug 2002 00:36:04 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:

 True re: the hash function.  Since identifiers are only allowed to use a
 small subset of valid ANSI characters, a couple bits per character could be
 eliminated.  In 32 bits with 6 bits per character you can get only 5
 characters with no possibility of loss of information.  For 128 bits you can
 get 21 characters.

And what will you do with Unicode identifiers? =)
 Ahh well just put the whole string in.  Pays to use short class and module
 names I guess.  ;)

Strings can be compressed, after all. Huffman or something.
Aug 13 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374820383640162 news.digitalmars.com...
 On Tue, 13 Aug 2002 00:36:04 -0700 "Sean L. Palmer"

 wrote:

 True re: the hash function.  Since identifiers are only allowed to use a
 small subset of valid ANSI characters, a couple bits per character could


 eliminated.  In 32 bits with 6 bits per character you can get only 5
 characters with no possibility of loss of information.  For 128 bits you


 get 21 characters.

And what will you do with Unicode identifiers? =)

Compile them, I hope!! ;)
 Ahh well just put the whole string in.  Pays to use short class and


 names I guess.  ;)

Strings can be compressed, after all. Huffman or something.

With pregenerated statistics about frequency of occurrences of symbols and pairs of symbols, they could actually compress quite well. Probably just about as fast as calculating a CRC32. Just as with the original string, the variable memory requirement is kinda annoying, though. Sean
Aug 15 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Thu, 15 Aug 2002 01:43:44 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:

 With pregenerated statistics about frequency of occurrences of symbols and
 pairs of symbols, they could actually compress quite well.  Probably just
 about as fast as calculating a CRC32.  Just as with the original string, the
 variable memory requirement is kinda annoying, though.

Since length of D identifiers is not limited in specification, I guess there is no workaround. We'll have to live with it.
Aug 15 2002
parent "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374836441665856 news.digitalmars.com...
 Since length of D identifiers is not limited in specification, I
 guess there is no workaround. We'll have to live with it.

Unfortunately, I'm stuck a bit with the OMF limitations on name lengths. But it's fairly large.
Aug 15 2002
prev sibling parent "anderson" <anderson firestar.com.au> writes:
What about, writing info to disk about the present class structure in debug
mode. When the class structure change you simply need to update the list,
which should contain all the class info. To determine what is which, it's
simply a matter of loading the list at initation and passing out the numbers
to the classes. This way you can use the entire 32 bits = 4294967296 entries
per group. When you distribute the code you also distribute the data file. I
mentioned something like this in a pervious post.

However, I would still like this to be a standard in the complier. If I had
time I'd write a standard lib to support it (although it wouldn't be as
effecient as a complie time version).


"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:ajacej$fv8$1 digitaldaemon.com...
 True re: the hash function.  Since identifiers are only allowed to use a
 small subset of valid ANSI characters, a couple bits per character could

 eliminated.  In 32 bits with 6 bits per character you can get only 5
 characters with no possibility of loss of information.  For 128 bits you

 get 21 characters.

 Ahh well just put the whole string in.  Pays to use short class and module
 names I guess.  ;)

 Sean

 "Pavel Minayev" <evilone omen.ru> wrote in message
 news:CFN374801163376736 news.digitalmars.com...
 On Sun, 11 Aug 2002 13:58:25 -0700 "Sean L. Palmer"

 wrote:

 I have a hard time imagining a scheme where the compiler could auto


 GUID's such that they stay the same from one compile to the next, let


 generate the same GUID on different machines if your friend decides to


 a small change to your program, compile it, then try to read an



 data file generated by your program on your machine.

Yes, that's the problem.
 GUID's are overkill for this problem...  that's the main point I'm


 make.  I'd rather see the full string of the class/module name



 in
 my datafiles than have GUID's.

So far I don't see any other way but use module.class string. Only it


 guarantee uniqueness.

 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!)


 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.

Yes, but there's still a chance that two of those 100 classes might have the same CRC32 (or whatever hashfunc you decide to use).


Aug 13 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:aj6in3$2e87$1 digitaldaemon.com...
 For streaming, you 99% of the time want to write a datum to a file, that
 will be read by a later run of that same program.  Sometimes a different
 program altogether.

Yes.
 I have a hard time imagining a scheme where the compiler could auto

 GUID's such that they stay the same from one compile to the next, let

 generate the same GUID on different machines if your friend decides to

 a small change to your program, compile it, then try to read an existing
 data file generated by your program on your machine.

Actually, autogenerated GUIDs should deliberately not be the same from compile to compile. Only if they are hardcoded will they be the same.
 Yes, GUID's are necessary for COM.  However, COM is not the entire
 programming universe, and I don't see the entire programming industry
 standardizing on COM anytime soon.

COM is actually a brilliant idea. The implementation of it went off course somewhere, but I don't think it was the GUIDs that did that.
 GUID's are overkill for this problem...  that's the main point I'm trying

 make.  I'd rather see the full string of the class/module name embedded in
 my datafiles than have GUID's.

The full name doesn't allow for different versions of the same class.
 Besides that, GUID's are large.  Why use 128 bits when 32 (or fewer!) bits
 would do?  Most programs will contain fewer than 100 classes.  Large
 programs may have 1000.

I don't think you can guarantee uniqueness with only 32 bits without knowing every other D class.
Aug 15 2002
next sibling parent reply "anderson" <anderson firestar.com.au> writes:
 I don't think you can guarantee uniqueness with only 32 bits without

 every other D class.

What about if you retricted it to just a group of classes (put each class in a group). That way there you'd only need to store unique ids within that group of classes. I also suggested a solution (however inefficient) to this in a previous email, using the class names an saving them to disk between complies. This method wouldn't rely on compression. However, this technique could be done by the programmer or in a standard lib. PS - Is it possible to get the name of a class as a string in C++?
Aug 16 2002
parent Pavel Minayev <evilone omen.ru> writes:
On Fri, 16 Aug 2002 15:15:08 +0800 "anderson" <anderson firestar.com.au> wrote:

 PS - Is it possible to get the name of a class as a string in C++?

typeid(Foo).name() And I believe it is mangled.
Aug 16 2002
prev sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Thu, 15 Aug 2002 23:22:44 -0700 "Walter" <walter digitalmars.com> wrote:

 Actually, autogenerated GUIDs should deliberately not be the same from
 compile to compile. Only if they are hardcoded will they be the same.

Then, you have a problem. You release version 1.0 of your program, end-users save documents made by it (and it uses persistence). Then, you release version 1.1 - and oops! it cannot read old files, because all the GUIDs were changed during recompile! Hard-coding GUIDs for every class is a pain in the # !. It should be the compiler which does all the dirty work like this, not the programmer. I still don't understand, why not use the name of the class? What do you mean by "different versions of the class"?
Aug 16 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374844840487037 news.digitalmars.com...
 On Thu, 15 Aug 2002 23:22:44 -0700 "Walter" <walter digitalmars.com>

 Actually, autogenerated GUIDs should deliberately not be the same from
 compile to compile. Only if they are hardcoded will they be the same.

Then, you have a problem. You release version 1.0 of your program, end-users save documents made by it (and it uses persistence). Then, you release version 1.1 - and oops! it cannot read old files, because all the GUIDs were changed during recompile! Hard-coding GUIDs for every class is a pain in the # !. It should be the compiler which does all the dirty work like this, not the programmer.

An alternate solution would be to write a ClassInfo.eq() function - classes with the same ClassInfo contents should be the same.
 I still don't understand, why not use the name of the class?
 What do you mean by "different versions of the class"?

Let's say, from version 1.0 to version 1.1, you add a member to the class without changing the class's name. Now, all binary references compiled with the old class layout will crash.
Aug 18 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Sun, 18 Aug 2002 22:17:50 -0700 "Walter" <walter digitalmars.com> wrote:

 An alternate solution would be to write a ClassInfo.eq() function - classes
 with the same ClassInfo
 contents should be the same.

But, ClassInfo is practically a name, vtbl pointer, and constructor pointer. So, it is simplier to store just the name. =)
 Let's say, from version 1.0 to version 1.1, you add a member to the class
 without changing the class's name. Now, all binary references compiled with
 the old class layout will crash.

This is "binary incompatibility", and there's just nothing you can do with that. Only to remember not to break code so it can't read files generated by earlier versions. After all, object can have some version property, and check it when restoring itself from file, to catch this kind of errors. But I don't see how you can _fix_ it.
Aug 19 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374878540987731 news.digitalmars.com...
 This is "binary incompatibility", and there's just nothing you can do with
 that. Only to remember not to break code so it can't read files generated
 by earlier versions. After all, object can have some version property,
 and check it when restoring itself from file, to catch this kind of

 But I don't see how you can _fix_ it.

At least by comparing the contents of ClassInfo, you'll know when you broke it <g>.
Aug 19 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 19 Aug 2002 16:35:23 -0700 "Walter" <walter digitalmars.com> wrote:

 At least by comparing the contents of ClassInfo, you'll know when you broke
 it <g>.

ClassInfo contains a string (class name), initializer, and a couple of pointers. It is probably not the best idea to save pointers to a disk file...
Aug 20 2002
next sibling parent "Martin M. Pedersen" <mmp www.moeller-pedersen.dk> writes:
Hi,

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374884983818403 news.digitalmars.com...
 ClassInfo contains a string (class name), initializer, and a couple of
 pointers. It is probably not the best idea to save pointers to a disk
 file...

Especially not with a garbage collecting language. However, saving pointers to disk may have its utility. I remember an application I wrote once. It had a small set of data structures fixed in memory, and a potentially very large data structure referencing the fixed data structure. The large data structure overflowed to a temporary file with pointers and all. It worked very well, and there was no reason it shouldn't, as the temporary file didn't live any longer than the fixed data structures. Another programming practice I recall, that requires special attention with GC is this: Thread A creates a structure on the heap and sends a pointer to it to thread B over a message queue at OS-level, thereby having a queue with synchronization for free. Regards, Martin M. Pedersen
Aug 20 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374884983818403 news.digitalmars.com...
 On Mon, 19 Aug 2002 16:35:23 -0700 "Walter" <walter digitalmars.com>

 At least by comparing the contents of ClassInfo, you'll know when you


 it <g>.

ClassInfo contains a string (class name), initializer, and a couple of pointers. It is probably not the best idea to save pointers to a disk file...

ClassInfo will eventually be extended to know about the members and their offsets. An overloaded eq() for it would be able to follow the pointers and properly compare the contents.
Aug 20 2002
next sibling parent reply Pavel Minayev <evilone omen.ru> writes:
On Tue=2C 20 Aug 2002 11=3A31=3A28 -0700 =22Walter=22
=3Cwalter=40digitalmars=2Ecom=3E wrote=3A

=3E ClassInfo will eventually be extended to know about the members and their
=3E offsets=2E An overloaded eq=28=29 for it would be able to follow the
pointers and
=3E properly compare the contents=2E

But this means that ANY change to interface of class would make
it =22incompatible=22 to any previous version - which doesn't make
sense if I just add a new function=2C or maybe a new property which
isn't persistent=2E

So either it should use class name=2C and let the programmer control
versions=2C or provide some way to indicate whether attribute is
persistant or not=2E Something like this=3A

=09class Mesh
=09{
=09=09Vertex=5B=5D vertices=3B
=09=09Face=5B=5D faces=3B
=09=09=2E=2E=2E
=09=09=2F=2F auto-generated OpenGL display list
=09=09transient GLuint displist=3B=09
=09}

But then it seems like a step to persistance built into the language
=28which would IMO be great!=29=2E

By the way=2C anybody knows of some good reliable way to store object
references into file=2C so they can be retrieved later=3F
Aug 20 2002
next sibling parent "anderson" <anderson firestar.com.au> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374890214802662 news.digitalmars.com...
On Tue, 20 Aug 2002 11:31:28 -0700 "Walter" <walter digitalmars.com> wrote:

 By the way, anybody knows of some good reliable way to store object
 references into file, so they can be retrieved later?

I'm sure this will be way below what your looking for. -- In a previous project (I couldn't find the source), I built a tree (you could of course use a hash table) of all the object references (well pointers in C++) and saved them into the file. It worked something like this. To save... Whenever the program came across a pointer the programmer would call this in the save method of the object. X = savePointer(char * Type, Object Pointer); //Note that at this point I didn't know that you could get the names of //objects using typeid(Foo).name(). X would return the number location in the tree the object existed. The X is saved in place where the pointer would be. If the object hadn't been saved yet, it would be saved on to a tempary store and inserted in the binary tree. Finally. saveAll( ... ); Would save the tree of tree of data and then all the object entries. At the end of the file the main objects tree references are saved. To load... The tree is loaded, then each object in the tree is loaded (calling the objects load method). Finally fixPointers is called for each new object (note that at this stage pointers are actually array indexes to the tree) which forces it to call getPointer(reference) for each pointer it contains. Finally all the linkages to the main program are set using getPointer(reference) again. Note that each object had a save and load function derived from a base class. This technique is reliable, but slightly complicated. Of course this technique could probably be improved in D (and probably even in C++). It would be nice if you could cycle though each property member in a class, determine if it was a pointer or not, and perform the appropriate action. That way this technique would only need to be written for the base class and all objects would inherit the ability to save themselves.
Aug 20 2002
prev sibling parent "Walter" <walter digitalmars.com> writes:
"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374890214802662 news.digitalmars.com...
But then it seems like a step to persistance built into the language
(which would IMO be great!).

I wanted to do that originally, but gave up on it as too complicated for now.
By the way, anybody knows of some good reliable way to store object
references into file, so they can be retrieved later?

A technique I've used is to store memory mapped files at fixed addresses.
Aug 22 2002
prev sibling parent Burton Radons <loth users.sourceforge.net> writes:
Walter wrote:

 "Pavel Minayev" <evilone omen.ru> wrote in message 


 On Mon, 19 Aug 2002 16:35:23 -0700 "Walter"
 <walter digitalmars.com>

 At least by comparing the contents of ClassInfo, you'll know
 when you


 it <g>.

couple of pointers. It is probably not the best idea to save pointers to a disk file...

ClassInfo will eventually be extended to know about the members and their offsets. An overloaded eq() for it would be able to follow the

 properly compare the contents.

Sigh. I've been waiting for someone to ask, but you three keep on circling around it, so this is what I do for my pickling. The class name is saved, which contains the full name up to the module. Then the parent class is saved. Then the field names and types are saved. Everything is done once only. A small class takes about thirty bytes. For loading, we take in that name and find it or not, giving a solid error message if there's a problem. Finding the class to load into is just a name search, as we match each field individually, allowing new fields to be added or fields to be moved around. If a field changes type or is deleted, it throws up, again with a good error message. This transfer is also done for types. From then on, the type is referenced with a single byte, or more if necessary (variable-sized integers are used everywhere). Pointers, class references, and arrays also use these single-transfer handles, so that aliasing is retained. Array range aliasing is not retained - if an array is a range of another array, and they're both saved, the arrays will be made unaliased in loading.
Aug 21 2002
prev sibling next sibling parent Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 01:26:38 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:

 Test for IsA (is instance of A) and IsDerivedFromA (is instance of A or is
 instance of class derived from A)

Very easy to write even now. Just look at ClassInfo;
 Of course dynamic cast.

All reference casts in D are dynamic.
Aug 05 2002
prev sibling parent Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 01:26:38 -0700 "Sean L. Palmer" <seanpalmer earthlink.net> 
wrote:

 Test for IsA (is instance of A) and IsDerivedFromA (is instance of A or is
 instance of class derived from A)

These are very easily written by user - just look at ClassInfo. =)
 Of course dynamic cast.

All reference casts in D are dynamic.
Aug 05 2002
prev sibling next sibling parent "Richard Krehbiel" <rich kastle.com> writes:
"Burton Radons" <loth users.sourceforge.net> wrote in message
news:3D4E1F35.5000703 users.sourceforge.net...
 As I mentioned here and described on d.gnu, I'm working on a port of DMD
 to Linux.  The big part of the work is complete, so now I'm playing with
 a more complete TypeInfo/ClassInfo, and I'm wondering what people
 concretely want from introspection.  Specific things you've needed to do
 that have been stymied by C and C++'s opacity, or things from other
 languages with introspection that you've found helpful.

I for one specifically do NOT want complete introspection; this is a feature for interpreted (or nearly) languages. A means for a programmer to label specific classes as "serializable" might be nice, but I don't want D to become so unweildy that executables carry around a massive type database. C++'s RTTI is just about right. You can ask if a pointer is to a class of a certain type, or is derived from a certain type. Run-time support is all there using the vtable pointer, which object instances must carry around anyway. Most of the type info should *stay* in the compiler, and *not* be added to the executable, except if the programmer asks for it. -- Richard Krehbiel, Arlington, VA, USA rich kastle.com (work) or krehbiel3 comcast.net (personal)
Aug 05 2002
prev sibling parent reply Patrick Down <pat codemoon.com> writes:
Burton Radons <loth users.sourceforge.net> wrote in news:3D4E1F35.5000703
 users.sourceforge.net:

 As I mentioned here and described on d.gnu, I'm working on a port of 

 to Linux.  The big part of the work is complete, so now I'm playing 

 a more complete TypeInfo/ClassInfo, and I'm wondering what people 
 concretely want from introspection.  Specific things you've needed to 

 that have been stymied by C and C++'s opacity, or things from other 
 languages with introspection that you've found helpful.
 

For those that are complaining about addtional executable bulk I suggest a compile option to excude introspection information. However I can find use for both introspection of member variables and of functions. Introspection of member variables in useful for lots of things. I've used in in Python to build automatic persistance. I have also used to automatically tie database fields to object members. There are a couple of reasons why I want to be able to inspect objects for functions. It would provide support similar to Java Beans for code building tools. Second it would make it much easier to attach D to various scripting engines. On a personal side I have a library that builds GUIs on the fly via a GUI description language. The code and language have gone thru several incarnations. One of the things I'd like to do in the D version is reference callback functions from directly within the GUI language. <button text="Ok" OnClick="DoOkClick()" />
Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 13:34:38 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:

 Introspection of member variables in useful for lots of things.  
 I've used in in Python to build automatic persistance.  I have 
 also used to automatically tie database fields to object members.  

Python is an interpreted language. D is not.
 There are a couple of reasons why I want to be able to inspect 
 objects for functions.  It would provide support similar to Java 
 Beans for code building tools.  Second it would make it much
 easier to attach D to various scripting engines.
 
 On a personal side I have a library that builds GUIs on the fly 
 via a GUI description language.  The code and language have gone 
 thru several incarnations.  One of the things I'd like to do in
 the D version is reference callback functions from directly 
 within the GUI language.
 
 <button text="Ok" OnClick="DoOkClick()" />

This can be done in a better way - look at Delphi for an example. There, you can declare properties and methods of class as "published", which means that information about them is saved. So, typically properties of GUI components like width, height, caption etc are all declared as published, so GUI designer can see them, and you declare event handlers as published so symbolic names can later be used.
Aug 05 2002
parent reply Patrick Down <pat codemoon.com> writes:
Pavel Minayev <evilone omen.ru> wrote in
news:CFN374737995338773 news.digitalmars.com: 

 On Mon, 5 Aug 2002 13:34:38 +0000 (UTC) Patrick Down
 <pat codemoon.com> wrote: 
 
 Introspection of member variables in useful for lots of things.  
 I've used in in Python to build automatic persistance.  I have 
 also used to automatically tie database fields to object members.  

Python is an interpreted language. D is not.

So? Does this mean that D should not have introspection so that you can do similar things? I'm not asking for run time modifiable code or structures. I'm just asking to be able to list the names and types of members.
 On a personal side I have a library that builds GUIs on the fly 
 via a GUI description language.  The code and language have gone 
 thru several incarnations.  One of the things I'd like to do in
 the D version is reference callback functions from directly 
 within the GUI language.
 
 <button text="Ok" OnClick="DoOkClick()" />

This can be done in a better way - look at Delphi for an example. There, you can declare properties and methods of class as "published", which means that information about them is saved. So, typically properties of GUI components like width, height, caption etc are all declared as published, so GUI designer can see them, and you declare event handlers as published so symbolic names can later be used.

Ok, I can see this as useful but it really just seems like introspection where the user can turn it on and off at the function and member level.
Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:

 Python is an interpreted language. D is not.

So? Does this mean that D should not have introspection so that you can do similar things? I'm not asking for run time modifiable code or structures. I'm just asking to be able to list the names and types of members.

.. which adds significant bloat to compiled executable.
 Ok, I can see this as useful but it really just seems like
 introspection where the user can turn it on and off at the
 function and member level.

Yes, it is exactly that. And I like it: where it is really needed, you turn it on; where it is not, it doesn't waste space.
Aug 05 2002
next sibling parent reply Patrick Down <pat codemoon.com> writes:
Pavel Minayev <evilone omen.ru> wrote in
news:CFN374739675806829 news.digitalmars.com: 

 On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down
 <pat codemoon.com> wrote: 
 
 Ok, I can see this as useful but it really just seems like
 introspection where the user can turn it on and off at the
 function and member level.

Yes, it is exactly that. And I like it: where it is really needed, you turn it on; where it is not, it doesn't waste space.

Ok, that seems reasonable. Perhaps you should be able to turn in on and off at the class and module level too. Also if you go as far as to add new keyword for member introspection should it go as far as C# style tagging?
Aug 05 2002
parent reply Pavel Minayev <evilone omen.ru> writes:
On Mon, 5 Aug 2002 19:45:39 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:

 Ok, that seems reasonable.  Perhaps you should be able to turn
 in on and off at the class and module level too.

Yes, right.
 Also if you go as far as to add new keyword for member introspection 
 should it go as far as C# style tagging?

Could you please remind me about it?
Aug 05 2002
parent Patrick Down <pat codemoon.com> writes:
Pavel Minayev <evilone omen.ru> wrote in
news:CFN374743695890046 news.digitalmars.com: 

 On Mon, 5 Aug 2002 19:45:39 +0000 (UTC) Patrick Down
 <pat codemoon.com> wrote: 
 
 Ok, that seems reasonable.  Perhaps you should be able to turn
 in on and off at the class and module level too.

Yes, right.
 Also if you go as far as to add new keyword for member introspection 
 should it go as far as C# style tagging?

Could you please remind me about it?

I meant attributes, I had to get the book out and remind myself. They are extra information available via reflection besides the name and type information. The syntax is like this. [Serializable] class Foo { } There are a number of built in attributes but like Serializable but you can also define your own. For example you might attach a human readable name for a runtime property browser. class Foo { [ Serializable, PropName("The yearly interest rate") ] public int yearlyRate { get { return yrRate; } set ( yrRate= value; } } } I'm not necessarily advocating them for D but they are an interesting language feature of C#.
Aug 06 2002
prev sibling next sibling parent reply "anderson" <anderson firestar.com.au> writes:
This sounds like a cool idea. This would be useful in saving/loading
objects.

<Babble>
I'd also like it if D provided other faculties for save/loading classes from
file. Some features to aid in quickly save parts of the hierarchies of class
objects to files. Some support in saving objects that have multiple
references and then in restoring their pointer locations on a load. (I've
done that before, but it would be nice to have some sort of support either
in lib form or complier form for it).

-PS Lucky this a digitalmars newgroup and hasn't been attacked by that
newgroup porn thing. What's with that Evil.

"Pavel Minayev" <evilone omen.ru> wrote in message
news:CFN374739675806829 news.digitalmars.com...
 On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down <pat codemoon.com>

 Python is an interpreted language. D is not.

So? Does this mean that D should not have introspection so that you can do similar things? I'm not asking for run time modifiable code or structures. I'm just asking to be able to list the names and types of members.

.. which adds significant bloat to compiled executable.
 Ok, I can see this as useful but it really just seems like
 introspection where the user can turn it on and off at the
 function and member level.

Yes, it is exactly that. And I like it: where it is really needed, you turn it on; where it is not, it doesn't waste space.

Aug 06 2002
parent reply "Sean L. Palmer" <seanpalmer earthlink.net> writes:
I propose a standard "convert to byte array" function (and corresponding
"convert byte array to object" function).  Not sure what the syntax should
be.  Something along the lines of standard conversion to string, just a
binary version.  Actual file I/O isn't the hard part, and
streaming/persistence doesn't always go to a disk file anyway (i.e. message
pipe, network socket, some kind of custom file, compressed file, etc)

Sean

"anderson" <anderson firestar.com.au> wrote in message
news:aioclq$11f4$1 digitaldaemon.com...
 This sounds like a cool idea. This would be useful in saving/loading
 objects.

 <Babble>
 I'd also like it if D provided other faculties for save/loading classes

 file. Some features to aid in quickly save parts of the hierarchies of

 objects to files. Some support in saving objects that have multiple
 references and then in restoring their pointer locations on a load. (I've
 done that before, but it would be nice to have some sort of support either
 in lib form or complier form for it).

 -PS Lucky this a digitalmars newgroup and hasn't been attacked by that
 newgroup porn thing. What's with that Evil.

Aug 07 2002
parent reply "anderson" <anderson firestar.com.au> writes:
"Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
news:aiqleb$13s0$1 digitaldaemon.com...
 I propose a standard "convert to byte array" function (and corresponding
 "convert byte array to object" function).  Not sure what the syntax should
 be.  Something along the lines of standard conversion to string, just a
 binary version.  Actual file I/O isn't the hard part, and
 streaming/persistence doesn't always go to a disk file anyway (i.e.

 pipe, network socket, some kind of custom file, compressed file, etc)

 Sean

That's something along the lines I was considering. I'm don't think I/O is that hard either, it was mainly the proccess of converting the class object structure to file. How would you handle pointers? In C++ you could simply save the entire class (unless you had pointers), in one go (or convert it to a binary string as easy as pie). So I hope that's not what your suggesting. However, I don't trust D here, because it has the power to rearrange members. However that's not my point. Your right, it should be able to be use it with anything, parhaps even used for in copy constuctors and overloaded operators. The point is that it would make coding alot easier if there was some support for dealing with the class object linkages as a whole.
 "anderson" <anderson firestar.com.au> wrote in message
 news:aioclq$11f4$1 digitaldaemon.com...
 This sounds like a cool idea. This would be useful in saving/loading
 objects.

 <Babble>
 I'd also like it if D provided other faculties for save/loading classes

 file. Some features to aid in quickly save parts of the hierarchies of

 objects to files. Some support in saving objects that have multiple
 references and then in restoring their pointer locations on a load.


 done that before, but it would be nice to have some sort of support


 in lib form or complier form for it).

 -PS Lucky this a digitalmars newgroup and hasn't been attacked by that
 newgroup porn thing. What's with that Evil.


Aug 07 2002
parent "Sean L. Palmer" <seanpalmer earthlink.net> writes:
Yah, the pointers are a bitch.  ;)  Didn't think of that.

Sean

"anderson" <anderson firestar.com.au> wrote in message
news:air25s$1geh$1 digitaldaemon.com...
 "Sean L. Palmer" <seanpalmer earthlink.net> wrote in message
 news:aiqleb$13s0$1 digitaldaemon.com...
 I propose a standard "convert to byte array" function (and corresponding
 "convert byte array to object" function).  Not sure what the syntax


 be.  Something along the lines of standard conversion to string, just a
 binary version.  Actual file I/O isn't the hard part, and
 streaming/persistence doesn't always go to a disk file anyway (i.e.

 pipe, network socket, some kind of custom file, compressed file, etc)

 Sean

That's something along the lines I was considering. I'm don't think I/O is that hard either, it was mainly the proccess of converting the class

 structure to file. How would you handle pointers?

 In C++ you could simply save the entire class (unless you had pointers),

 one go (or convert it to a binary string as easy as pie). So I hope that's
 not what your suggesting. However, I don't trust D here, because it has

 power to rearrange members. However that's not my point.

 Your right, it should be able to be use it with anything, parhaps even

 for in copy constuctors and overloaded operators. The point is that it

 make coding alot easier if there was some support for dealing with the

 object linkages as a whole.

Aug 07 2002
prev sibling parent Burton Radons <loth users.sourceforge.net> writes:
Pavel Minayev wrote:

 On Mon, 5 Aug 2002 15:46:12 +0000 (UTC) Patrick Down <pat codemoon.com> wrote:
 
 
Python is an interpreted language. D is not.

you can do similar things? I'm not asking for run time modifiable code or structures. I'm just asking to be able to list the names and types of members.

.. which adds significant bloat to compiled executable.

It's not going to be compile-time optional, so this subthread can die.
Aug 06 2002