www.digitalmars.com         C & C++   DMDScript  

D - I Really Need Reflection!

reply "Craig Black" <cblack ara.com> writes:
Thanks and praise to Walter and all those who are sacrificing their time to
make this innovative programming language a reality!

It sounds like reflection (a.k.a. introspection) is taking a backseat to the
many other features that have been added.  Does anyone realize the power of
class meta-data at run-time?  It's huge!  It's a whole new programming
paradigm!  I could give you many examples of how I would use it if anyone
cared to listen to me rant.

I am very interested in D and all of its innovate features, but for my
purposes it would not be a vast improvement over C++ if reflection were
absent.  Maybe I'm all by myself in saying this, but I would use D if it
included reflection.  I could not justify porting from C++ otherwise.

I would REALLY like to know what Walter intends to do regarding reflection.

Thanks,

Craig
Mar 29 2004
next sibling parent Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
"Craig Black" <cblack ara.com> wrote:

 It sounds like reflection (a.k.a. introspection) is taking a backseat
 to the many other features that have been added.  Does anyone realize
 the power of class meta-data at run-time?  It's huge!  It's a whole
 new programming paradigm!  I could give you many examples of how I
 would use it if anyone cared to listen to me rant.

It wouldn't be ranting to me :-) I also want reflection, and find the lack of it in C++ one of its largest drawbacks (among many :-) If D were to have reflection support, it would be killer.
 I am very interested in D and all of its innovate features, but for my
 purposes it would not be a vast improvement over C++ if reflection
 were absent.  Maybe I'm all by myself in saying this, but I would use
 D if it included reflection.  I could not justify porting from C++
 otherwise. 

I agree, although I think porting to D would still be an improvement over C++, and I would consider it even without reflection (no loss, but many gains). My biggest problem now (besides being a D beginner :-) is lack of debugging support. This morning I've been toying with Visual Studio, using a makefile wrapped in a VS.NET project. It compiles and I can run the app in the Visual Studio debugger, and even have it stop on the source line when an exception is thrown. But assert's and other errors don't come through too well -- it's possible I haven't set it all up correctly yet, or that it isn't possible.
 I would REALLY like to know what Walter intends to do regarding
 reflection. 

Me too, but I wouldn't be shocked if Walter replied (or has already) that it isn't in his vision for D. In any case, it would be a major undertaking, not something he could tack on. But I think he could take a day or two and implement it for us, doncha think? <BG> -- dave
Mar 29 2004
prev sibling next sibling parent reply Ilya Minkov <minkov cs.tum.edu> writes:
Throw your thoughts at us! You can be sure some of us read and comment 
on them. Walter said that he may expand the metadata if someone can 
convince him of the merit of it. So far his arguments were, as far as i 
can recall:

- Serialization can already be done using RTTI - you can query the 
fields of a class.
- The only real use he sees is in development enviroments, which may 
just as well parse the code and compile the thunks, because they 
requiere the compiler anyway.
- It would lead to increased program sizes in a much further extent than 
RTTI and would have much less merit. Besides, it would hinder 
whole-program optimization - unused methods cannot be eliminated, 
non-inline versions can not be omitted.

It also seems that current extended RTTI is not part of the spec, 
perhaps other compilers can implement it in an incompatible manner if 
this can increase program performance - for example interleaving fields 
in struct or class arrays may requiere such measure.

When commenting please also recall that there are interfaces, they are 
enough in most cases. And definetely they are *much* more efficient on 
performance! Also some tasks could be taken by an syntax extension 
mechanism (like OpenC++), of which Walter was thinking for future 
versions of D. Just think that when you need more information with 
interfaces, its generation could then be automated by library means, 
only for the parts where this makes sense!

So, i think i have collected the points here which may make this 
discussion much shorter than the previous one on the same topic. ;) Or 
you might also want to wait with it till Version 1 final comes out and 
we would be collecting ideas for the new generation of D. But of course 
we can discuss it now if you like. And yes, D does not justify to port 
existing software from C++, it justifies mostly to write new software 
making it much less effort.

-eye

Craig Black schrieb:
 Thanks and praise to Walter and all those who are sacrificing their time to
 make this innovative programming language a reality!
 
 It sounds like reflection (a.k.a. introspection) is taking a backseat to the
 many other features that have been added.  Does anyone realize the power of
 class meta-data at run-time?  It's huge!  It's a whole new programming
 paradigm!  I could give you many examples of how I would use it if anyone
 cared to listen to me rant.
 
 I am very interested in D and all of its innovate features, but for my
 purposes it would not be a vast improvement over C++ if reflection were
 absent.  Maybe I'm all by myself in saying this, but I would use D if it
 included reflection.  I could not justify porting from C++ otherwise.
 
 I would REALLY like to know what Walter intends to do regarding reflection.
 
 Thanks,
 
 Craig
 
 

Mar 29 2004
parent reply "Craig Black" <cblack ara.com> writes:
 - Serialization can already be done using RTTI - you can query the
 fields of a class.

You already store the fields of a class? I didn't know that! Great! In what form are they stored? What then do you lack? How about the methods? If you already have fields, it would seem to me that the methods would be easy to do also. Further, it seems that it wouldn't be that hard to add a complete introspection system. :-) Actually, having access to fields is the main thing. But the rest would be nice to have also, and it would seem to me that this would not take that long to implement. You could look at C# if you want to see a well-done reflection system.
 - The only real use he sees is in development enviroments, which may
 just as well parse the code and compile the thunks, because they
 requiere the compiler anyway.

As far as the file size and memory overhead, it should be a compiler option to include metadata. That way you could include it only if you need it.
 - It would lead to increased program sizes in a much further extent than
 RTTI and would have much less merit. Besides, it would hinder
 whole-program optimization - unused methods cannot be eliminated,
 non-inline versions can not be omitted.

This is true. If you include the methods in the metadata then you would not be able to discard them. Even so, if it is a compiler option, this overhead will only be incured if the programmer specifies the option. Yes, serialization is one of the main uses of reflection. As far as development environments, I agree with Walter that the DE should be able to parse the code. However, there are many other uses for reflection. As I said, this is a whole new programming paradigm, the power which has yet to be discovered. One cool example of how I would use reflection is for automatic GUI's. With reflection, you could automatically generate a GUI based upon the fields and/or methods in a class. Just send an instance of your particular class into the GUI generator, and it will construct a GUI for you to manipulate the fields, or invoke the methods of that class. In such a situation, it would be nice to attach arbitrary information to a particular field/method so that you could tweak how the GUI behaves. Correct me if I'm wrong, but I think that such information is called an "attribute" in C#. I am confident that there are many more applications for reflection that have yet to be explored. It is like any other programming paradigm. It can take a while for your brain to think in a new ways. Thus, you don't realize the utility of this new approach until you have been using it for a while. Craig
Mar 29 2004
parent reply Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
"Craig Black" <cblack ara.com> wrote:

 You already store the fields of a class?  I didn't know that!  Great! 
 In what form are they stored?  What then do you lack?  How about the
 methods? If you already have fields, it would seem to me that the
 methods would be easy to do also. Further, it seems that it wouldn't
 be that hard to add a complete introspection system. :-)

Hmm, I can't find anywhere in the docs that says information about the fields is available at runtime. In fact, looking at ClassInfo in object.d in the phobos library, there is information on the base class and the interfaces, which would be used for dynamic casting, and a few other things, but no information about the methods or fields.
 Actually, having access to fields is the main thing.  But the rest
 would be nice to have also, and it would seem to me that this would
 not take that long to implement.  You could look at C# if you want to
 see a well-done reflection system.

I agree that C# has a very good reflection system. In fact, if C# could be compiled to native code (and didn't require the .NET runtime) I would probably be using it for most of my development. So I hope it's "D to the rescue!" :-)
 As far as the file size and memory overhead, it should be a compiler
 option to include metadata.  That way you could include it only if you
 need it. 

Yes -- following the C++ philosophy that you don't pay for it if you don't use it. -- dave
Mar 29 2004
next sibling parent reply "Jan-Eric Duden" <jeduden whisset.com> writes:
 I agree that C# has a very good reflection system. In fact, if C# could be
 compiled to native code (and didn't require the .NET runtime) I would
 probably be using it for most of my development. So I hope it's "D to the
 rescue!" :-)

With Mono, I think you can compile your C# program with an embedded .NET runtime.
Mar 31 2004
parent Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
"Jan-Eric Duden" <jeduden whisset.com> wrote:

 Maybe you want to check out Mono. http://www.go-mono.com/
 With Mono, I think you can compile your C# program with an embedded
 .NET runtime.

Yes, it's been a while since I've looked at it. But I am starting to like D, maybe my attraction to C# was really because of my frustration with C++. I think each has its place. -- dave
Mar 31 2004
prev sibling parent reply Ilya Minkov <minkov cs.tum.edu> writes:
Dave Sieber schrieb:

 "Craig Black" <cblack ara.com> wrote:

 Hmm, I can't find anywhere in the docs that says information about the 
 fields is available at runtime. In fact, looking at ClassInfo in object.d 
 in the phobos library, there is information on the base class and the 
 interfaces, which would be used for dynamic casting, and a few other 
 things, but no information about the methods or fields.

Good that you looked. I looked a bit through the compiler source and it really seems to lack it these days. Then i'm looking now at the source of DLI (changes from it should be easy to reintegrate with DMD), which introduced the extended RTTI, which as i now suspect did not make it back into the main distro. It apparently at least has placeholders for such things as FieldInfo, EnumInfo, StructInfo, MethodInfo. Its version of Phobos includes a working module pickle, which saves a tree of classes and arrays and can restore it. I tried to compile it on current DMD, but it apparently requieres more definitions from its object.d, which incude SymbolInfo, huge ClassInfo including FieldInfo[] and MethodInfo[], and all other definitions to support it. It would also all requiere compiler support... Which i think should be reintegrated. I'm sorry i deceived you all. This also makes it more clear why Butron, the author of DLI, has been loosing interest in D. He was probably not finding enough support for his effort from Walter.
As far as the file size and memory overhead, it should be a compiler
option to include metadata.  That way you could include it only if you
need it. 

Yes -- following the C++ philosophy that you don't pay for it if you don't use it.

No, it should probably not. It should be a per-class option. -eye
Mar 31 2004
parent Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
Ilya Minkov <minkov cs.tum.edu> wrote:

 Good that you looked. I looked a bit through the compiler source and
 it really seems to lack it these days.

Yes, I think it's not on the agenda for D 1.0.
 I'm sorry i deceived you all.

Hey, no sweat. It wasn't intentional, now was it? :-) -- dave
Mar 31 2004
prev sibling next sibling parent reply "Julio César Carrascal Urquijo" <jcesar phreaker.net> writes:
"Craig Black" <cblack ara.com> escribió en el mensaje
news:c49mja$2tc4$1 digitaldaemon.com...
(...)
 It sounds like reflection (a.k.a. introspection) is taking a backseat to

 many other features that have been added.  Does anyone realize the power

 class meta-data at run-time?  It's huge!  It's a whole new programming
 paradigm!  I could give you many examples of how I would use it if anyone
 cared to listen to me rant.

Agreed. There's support for RTTI and Serialization already but class meta-data is really handy, especially if you want to build a Form Designer, Web Services or just loading code from a file.
 I am very interested in D and all of its innovate features, but for my
 purposes it would not be a vast improvement over C++ if reflection were
 absent.  Maybe I'm all by myself in saying this, but I would use D if it
 included reflection.  I could not justify porting from C++ otherwise.

Well, I think reflection, as implemented in Java or .NET (I mean creating and executing code at runtime), isn't that important for D right now, but class meta-data is. Also, don't think it's that hard for Walter to do it, but he probably want D as it is right now for the 1.0 release.
Mar 29 2004
parent "Craig Black" <cblack ara.com> writes:
 Well, I think reflection, as implemented in Java or .NET (I mean creating
 and executing code at runtime), isn't that important for D right now, but
 class meta-data is. Also, don't think it's that hard for Walter to do it,
 but he probably want D as it is right now for the 1.0 release.

Yes, that's what I mean. Reflection usually refers to metadata plus the capability to create code at run-time. Introspection usually refers to merely metadata. I use the term reflection loosely because its the popular term to use. Introspection is probably what I should have said. Introspection is the one that makes my mouth water. Full reflection is too much for my little brain to handle at the moment. I would be VERY pleased with merely metadata. Craig
Apr 01 2004
prev sibling next sibling parent "Derek Parnell" <Derek.Parnell psyc.ward> writes:
On Mon, 29 Mar 2004 11:32:26 -0600 (30/Mar/04 03:32:26 AM)
, Craig Black <cblack ara.com> wrote:

 Thanks and praise to Walter and all those who are sacrificing their time  
 to
 make this innovative programming language a reality!

 It sounds like reflection (a.k.a. introspection) is taking a backseat to  
 the
 many other features that have been added.  Does anyone realize the power  
 of
 class meta-data at run-time?  It's huge!  It's a whole new programming
 paradigm!  I could give you many examples of how I would use it if anyone
 cared to listen to me rant.

 I am very interested in D and all of its innovate features, but for my
 purposes it would not be a vast improvement over C++ if reflection were
 absent.  Maybe I'm all by myself in saying this, but I would use D if it
 included reflection.  I could not justify porting from C++ otherwise.

 I would REALLY like to know what Walter intends to do regarding  
 reflection.

You are preaching to the converted here. Reflection deals with situations where the data's meta-data cannot be known at compile-time, but is known at run-time. This can occur in utility libraries that are used by a third party - such as report writers, form builders, data-converters, etc... And ideally we are talking about non-built-in datatypes as well, such as classes and structs. -- Derek
Mar 29 2004
prev sibling next sibling parent reply Richard Krehbiel <rich kastle.com> writes:
Craig Black wrote:
 It sounds like reflection (a.k.a. introspection) is taking a backseat to the
 many other features that have been added.  Does anyone realize the power of
 class meta-data at run-time?  It's huge!

Do you realize the memory space requirements of the full symbol table that it takes to do full introspection at run-time? It's huge! :-) I for one would not support automatic inclusion of full introspection to D. It's a whole new programming
 paradigm!

Yes, and it's not the domain of *compiled* *to* *machine* *code* languages.
 I could give you many examples of how I would use it if anyone
 cared to listen to me rant.
 
 I am very interested in D and all of its innovate features, but for my
 purposes it would not be a vast improvement over C++ if reflection were
 absent.  Maybe I'm all by myself in saying this, but I would use D if it
 included reflection.  I could not justify porting from C++ otherwise.
 
 I would REALLY like to know what Walter intends to do regarding reflection.

Frankly, Java's just sitting there, ready for your use where you need full introspction and reflection. Or perhaps Python. But please don't engorge compiled languages with full run-time symbol tables - at least, not by default. (I'd have no arguments if Walter wanted to offer a class definition modifier saying "this class is fully introspective," at which point it could emit symbol table stuff into the object files. But please, not ALL THE TIME.)
Apr 05 2004
parent Ilya Minkov <minkov cs.tum.edu> writes:
Richard Krehbiel schrieb:
 I for one would not support automatic inclusion of full introspection to D.

Perhaps automatic for fields (because often the whole hierarchy has to be serialized) and non-automatic for methods? Method data would also tend to be much larger than field data, i think. Syntactically, inclusion of such information with classes can be done by inheriting them from a "magic" interface which would also be used to access it. Actually, even field data can be placed optionally this way since it's so simple to use.
   It's a whole new programming paradigm!

Yes, and it's not the domain of *compiled* *to* *machine* *code* languages.

Which leads me to an idea of companion scripting language to D, which should allow easy combination of scripted and native code. But that's a whole new direction of work! And hey, it would need near to full introspection capabilities in the compiled part of D. -eye
Apr 05 2004
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Craig Black" <cblack ara.com> wrote in message
news:c49mja$2tc4$1 digitaldaemon.com...
 I would REALLY like to know what Walter intends to do regarding

I do intend to expand the 'classinfo' so that one could figure out all the fields and methods of a class. However, I spent some time trying to figure out how to make dynamic class loading work with some special compiler magic, but it isn't going to happen. The stumbling block is not knowing at compile time what the type of an expression is. I could wave the magic spec wand and say that all methods in a dynamic class must return void and that data fields are not possible, but such restrictions make it not so useful.
Apr 08 2004
next sibling parent J Anderson <REMOVEanderson badmama.com.au> writes:
Walter wrote:

"Craig Black" <cblack ara.com> wrote in message
news:c49mja$2tc4$1 digitaldaemon.com...
  

I would REALLY like to know what Walter intends to do regarding
    

I do intend to expand the 'classinfo' so that one could figure out all the fields and methods of a class. However, I spent some time trying to figure out how to make dynamic class loading work with some special compiler magic, but it isn't going to happen. The stumbling block is not knowing at compile time what the type of an expression is. I could wave the magic spec wand and say that all methods in a dynamic class must return void and that data fields are not possible, but such restrictions make it not so useful.

section in the wiki. http://www.prowiki.org/wiki4d/wiki.cgi?HowTo/RealtimeTypeInformation -- -Anderson: http://badmama.com.au/~anderson/
Apr 08 2004
prev sibling next sibling parent reply "Craig Black" <cblack ara.com> writes:
 I do intend to expand the 'classinfo' so that one could figure out all the
 fields and methods of a class.

Super-de-duper! What version do you forsee as having these capabilities?
However, I spent some time trying to figure
 out how to make dynamic class loading work with some special compiler

 but it isn't going to happen. The stumbling block is not knowing at

 time what the type of an expression is.

"Dynamic class loading"? You're talking about loading a class from a DLLat run-time, right? I don't really understand your problem. Perhaps I have a different idea about what "dynamic class loading" is.
 I could wave the magic spec wand and say that all methods in a dynamic

 must return void and that data fields are not possible, but such
 restrictions make it not so useful.

Again I don't understand your dilemma. Perhaps, if you have time, you could expound on it. Thanks, Craig
Apr 09 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Craig Black" <cblack ara.com> wrote in message
news:c572ms$j7c$1 digitaldaemon.com...
 I do intend to expand the 'classinfo' so that one could figure out all


 fields and methods of a class.

Super-de-duper! What version do you forsee as having these capabilities?

2.0.
However, I spent some time trying to figure
 out how to make dynamic class loading work with some special compiler

 but it isn't going to happen. The stumbling block is not knowing at

 time what the type of an expression is.


 run-time, right?  I don't really understand your problem.  Perhaps I have

 different idea about what "dynamic class loading" is.

I mean given the name of a class, load it at runtime from some external database (like a DLL), create an instance of that class, and then access its members.
 I could wave the magic spec wand and say that all methods in a dynamic

 must return void and that data fields are not possible, but such
 restrictions make it not so useful.

Again I don't understand your dilemma. Perhaps, if you have time, you

 expound on it.

Object o = classFactory("TheNameOfSomeClassNotKnownAtCompileTime"); int a = o.x + o.y; At runtime we can determine if the fields x and y are present, and what their types are. But we don't know how to compile the + expression, since we don't know the types of x and y at compile time. Think of how this would impact function overloading, for example: foo(o.x);
Aug 09 2004
parent teqDruid <me teqdruid.com> writes:
On Mon, 09 Aug 2004 16:15:03 -0700, Walter wrote:

 
 "Craig Black" <cblack ara.com> wrote in message
 news:c572ms$j7c$1 digitaldaemon.com...
 I do intend to expand the 'classinfo' so that one could figure out all


 fields and methods of a class.

Super-de-duper! What version do you forsee as having these capabilities?

2.0.
However, I spent some time trying to figure
 out how to make dynamic class loading work with some special compiler

 but it isn't going to happen. The stumbling block is not knowing at

 time what the type of an expression is.


 run-time, right?  I don't really understand your problem.  Perhaps I have

 different idea about what "dynamic class loading" is.

I mean given the name of a class, load it at runtime from some external database (like a DLL), create an instance of that class, and then access its members.
 I could wave the magic spec wand and say that all methods in a dynamic

 must return void and that data fields are not possible, but such
 restrictions make it not so useful.

Again I don't understand your dilemma. Perhaps, if you have time, you

 expound on it.

Object o = classFactory("TheNameOfSomeClassNotKnownAtCompileTime"); int a = o.x + o.y;

Something like this isn't a problem with classes, but rather a typing issue, and (obviously) the way to resolve it is using a cast. For instance if you were to provide a classFactory method, one would then do: Object o = classFactory("TheNameOfSomeClassNotKnownAtCompileTime"); MyInterface a = cast(MyInterface)o; if (a is null) throw new Exception("Bad Class"); Casting is probably the most consistent method of doing anything like this- you even have to cast with Reflection in Java- and I really like the way they do it. For those of you who aren't familiar, in Java for each class you write, there is a Class class as well. That is to say, one can get a Class object representing your class. One of that Object's fields is an array of Constructor objects (one can even do a Class.getConstructor(Class[] parameterTypes) which was a HUGE pain before Java got autoboxing in 1.5). Once one has a Constructor, run this on the constructor object: "Object newInstance(Object[] initargs) " <- right out of JDocs. One can also do similar things with Method objects in Java. I'm sure that Walter has thought of simply casting, but has a better method (as he frequently does) although I'm having trouble imagining any other method that is type-safe. So the question is... what'cha got brewing up there, Walter? John
Aug 10 2004
prev sibling parent School <school users.sf.net> writes:
Walter ´£¨ì:

 "Craig Black" <cblack ara.com> wrote in message
 news:c49mja$2tc4$1 digitaldaemon.com...
 
I would REALLY like to know what Walter intends to do regarding

reflection. I do intend to expand the 'classinfo' so that one could figure out all the fields and methods of a class. However, I spent some time trying to figure out how to make dynamic class loading work with some special compiler magic, but it isn't going to happen. The stumbling block is not knowing at compile time what the type of an expression is. I could wave the magic spec wand and say that all methods in a dynamic class must return void and that data fields are not possible, but such restrictions make it not so useful.

Check it http://reflection.sourceforge.net/ -- School, yet another nickname for anonymous. :D ;-D
Apr 11 2004