www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - TypeInfoEx and Variant: suggestions?

reply Christopher Wright <dhasenan gmail.com> writes:
Hi all,

I finally found some time to work on TypeInfoEx. It's usable, in that 
you can view information about a type; but you can't do anything with 
that information. One large barrier is the lack of a usable Variant type.

To invoke a function on an object or struct, you need to pass it as 
either Object or void* or Variant.

If you invoke a function, you want to pass in an array of Variant for 
the arguments. You want to get the return type as a Variant.

If you get a field, you want to get it as a Variant.

You could use templates instead, but that's not usable in many 
situations, so I won't have that as the only option.

Andrei's std.variant has an interesting design, but it has two flaws 
that render it unusable:
  - It cannot be used with structs of arbitrary size. It's parameterized 
on size, but no size will always be sufficient; and due to its 
allocation scheme, choosing to support (for instance) 1KB structs will 
force a bool to take 1KB.
  - It cannot be created with RTTI; it requires compile-time type 
information.

So, my options are:
  - Write my own Variant supporting the necessary operations. This makes 
things annoying for anyone wanting to use my code with Phobos, since now 
there are two Variant structs running around.
  - Use std.boxer, which is deprecated.
  - Pass around an opaque Pair!(TypeInfo, void*).

Any suggestions? I've written such a Variant for D1 and Tango, but I 
relish neither the idea of supporting both with the same class, nor code 
duplication.
Apr 16 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Christopher Wright wrote:

 Hi all,
 
 I finally found some time to work on TypeInfoEx. It's usable, in that
 you can view information about a type; but you can't do anything with
 that information. One large barrier is the lack of a usable Variant type.
 
 To invoke a function on an object or struct, you need to pass it as
 either Object or void* or Variant.
 
 If you invoke a function, you want to pass in an array of Variant for
 the arguments. You want to get the return type as a Variant.
 
 If you get a field, you want to get it as a Variant.
 
 You could use templates instead, but that's not usable in many
 situations, so I won't have that as the only option.
 
 Andrei's std.variant has an interesting design, but it has two flaws
 that render it unusable:
  - It cannot be used with structs of arbitrary size. It's parameterized
 on size, but no size will always be sufficient; and due to its
 allocation scheme, choosing to support (for instance) 1KB structs will
 force a bool to take 1KB.
  - It cannot be created with RTTI; it requires compile-time type
 information.
 
 So, my options are:
  - Write my own Variant supporting the necessary operations. This makes
 things annoying for anyone wanting to use my code with Phobos, since now
 there are two Variant structs running around.
  - Use std.boxer, which is deprecated.
  - Pass around an opaque Pair!(TypeInfo, void*).
 
 Any suggestions? I've written such a Variant for D1 and Tango, but I
 relish neither the idea of supporting both with the same class, nor code
 duplication.


Use the Tango Variant code.  It supports values of any size and doesn't
require knowledge of the compile-time type.  It's in tango.core so the
dependencies are relatively small, so porting should be fairly simple
(the code originally compiled under both Phobos and Tango, but that
version's out of date now.)

If you look at http://dsource.org/projects/tango/ticket/821 there's a
patch to the current Variant that adds support for varargs as well.

  -- Daniel
Apr 16 2009
parent reply Christopher Wright <dhasenan gmail.com> writes:
Daniel Keep wrote:

 Use the Tango Variant code.  It supports values of any size and doesn't
 require knowledge of the compile-time type.


Not so. That was one of three reasons that I wrote my own Variant 
previously (the others being toString and sameness comparison).
Apr 16 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Christopher Wright wrote:

 Daniel Keep wrote:

 Use the Tango Variant code.  It supports values of any size and doesn't
 require knowledge of the compile-time type.


 
 Not so. That was one of three reasons that I wrote my own Variant
 previously (the others being toString and sameness comparison).


What not so?  I know for a fact it supports values of any size, so I
assume you're referring to compile-time.  With the new patch you can
pack and unpack Variants from TypeInfo and a void* so you DON'T need to
know the type ahead of time.  There isn't a way to construct from a
single TypeInfo,void* pair since I wasn't sure if that would actually be
needed.

toString is a design issue, not a technical one; they didn't want stuff
in tango.core depending on stuff outside of that.  If you look at the
history of Variant, there's an old version with "proper" toString code
that should port forward painlessly.

On a side note, I'm working on getting Tango's Layout to understand
Variants and do the "proper" stringification.  Still trying to work out
how to get the changes to compile without having to rebuild Tango...
*grumble*

As for comparisons, it supports opEqual and opCmp between Variants, it
just doesn't attempt to do any automatic conversion between
non-identical but comparable types (like comparing ints to longs.)

Is there something specific you think Tango's Variant should do that it
doesn't?

  -- Daniel
Apr 16 2009
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:


 On a side note, I'm working on getting Tango's Layout to understand
 Variants and do the "proper" stringification.  Still trying to work out
 how to get the changes to compile without having to rebuild Tango...
 *grumble*



Have you tried building your test app with your modified Tango file  
specifically?  That generally works for me:

dmd variantTest.d ~/d/include/d/tango/core/Variant.d

-Steve
Apr 17 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Steven Schveighoffer wrote:

 On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:
 

 On a side note, I'm working on getting Tango's Layout to understand
 Variants and do the "proper" stringification.  Still trying to work out
 how to get the changes to compile without having to rebuild Tango...
 *grumble*


 
 
 Have you tried building your test app with your modified Tango file
 specifically?  That generally works for me:
 
 dmd variantTest.d ~/d/include/d/tango/core/Variant.d
 
 -Steve


That works for Variant, but it doesn't seem to work with Layout.  It
compiles and runs, but it doesn't seem to actually use the code I give it.

  -- Daniel
Apr 17 2009
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 17 Apr 2009 07:11:33 -0400, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:


 Steven Schveighoffer wrote:

 On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:


 On a side note, I'm working on getting Tango's Layout to understand
 Variants and do the "proper" stringification.  Still trying to work out
 how to get the changes to compile without having to rebuild Tango...
 *grumble*



 Have you tried building your test app with your modified Tango file
 specifically?  That generally works for me:

 dmd variantTest.d ~/d/include/d/tango/core/Variant.d

 -Steve


 That works for Variant, but it doesn't seem to work with Layout.  It
 compiles and runs, but it doesn't seem to actually use the code I give  
 it.


I think I ran into this before.  I think it's because Layout is a  
template, and is compiled when it is used, not when the module is  
compiled.  You have to recompile the user of Layout as well (i.e.  
Stdout.d, and probably Print.d).

-Steve
Apr 17 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Steven Schveighoffer wrote:

 On Fri, 17 Apr 2009 07:11:33 -0400, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:
 

 Steven Schveighoffer wrote:

 On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:


 On a side note, I'm working on getting Tango's Layout to understand
 Variants and do the "proper" stringification.  Still trying to work out
 how to get the changes to compile without having to rebuild Tango...
 *grumble*



 Have you tried building your test app with your modified Tango file
 specifically?  That generally works for me:

 dmd variantTest.d ~/d/include/d/tango/core/Variant.d

 -Steve


 That works for Variant, but it doesn't seem to work with Layout.  It
 compiles and runs, but it doesn't seem to actually use the code I give
 it.


 
 I think I ran into this before.  I think it's because Layout is a
 template, and is compiled when it is used, not when the module is
 compiled.  You have to recompile the user of Layout as well (i.e.
 Stdout.d, and probably Print.d).
 
 -Steve


Ah, but I'm using it directly.  AND I've put my main function inside the
modified Layout module.  What more does it want from me?  Cookies?
Chocolates?!

  -- Daniel
Apr 17 2009
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 17 Apr 2009 07:23:48 -0400, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:


 Steven Schveighoffer wrote:

 On Fri, 17 Apr 2009 07:11:33 -0400, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:


 Steven Schveighoffer wrote:

 On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:


 On a side note, I'm working on getting Tango's Layout to understand
 Variants and do the "proper" stringification.  Still trying to work  
 out
 how to get the changes to compile without having to rebuild Tango...
 *grumble*



 Have you tried building your test app with your modified Tango file
 specifically?  That generally works for me:

 dmd variantTest.d ~/d/include/d/tango/core/Variant.d

 -Steve


 That works for Variant, but it doesn't seem to work with Layout.  It
 compiles and runs, but it doesn't seem to actually use the code I give
 it.


 I think I ran into this before.  I think it's because Layout is a
 template, and is compiled when it is used, not when the module is
 compiled.  You have to recompile the user of Layout as well (i.e.
 Stdout.d, and probably Print.d).

 -Steve


 Ah, but I'm using it directly.  AND I've put my main function inside the
 modified Layout module.  What more does it want from me?  Cookies?
 Chocolates?!


You must be doing something wrong, it works for me:

[steves localhost trunk]$ svn diff tango/text/convert/Layout.d
Index: tango/text/convert/Layout.d
===================================================================
--- tango/text/convert/Layout.d (revision 4435)
+++ tango/text/convert/Layout.d (working copy)
   -35,6 +35,8   

  private import  tango.io.model.IConduit : OutputStream;

+private import tango.stdc.stdio;
+
  /*******************************************************************************

          Platform issues ...
   -602,6 +604,7   
                              return integer (result, *cast(ushort*) p,  
format, ushort.max, "u");

                         case TypeCode.INT:
+                            printf("inside INT print!\n");
                              return integer (result, *cast(int*) p,  
format, uint.max);

                         case TypeCode.UINT:
[steves localhost trunk]$ cd ~/dstuff/testing
[steves localhost testing]$ cat teststuff.d
import tango.text.convert.Layout;

void main()
{
     auto layout = new Layout!(char);
     int x = 0;
     auto result = layout("{}", x);
}
[steves localhost testing]$ dmd teststuff.d
[steves localhost testing]$ ./teststuff
inside INT print!
[steves localhost testing]$

-Steve
Apr 17 2009
prev sibling parent Christopher Wright <dhasenan gmail.com> writes:
Daniel Keep wrote:

 
 Christopher Wright wrote:

 Daniel Keep wrote:

 Use the Tango Variant code.  It supports values of any size and doesn't
 require knowledge of the compile-time type.


 Not so. That was one of three reasons that I wrote my own Variant
 previously (the others being toString and sameness comparison).


 
 What not so?  I know for a fact it supports values of any size, so I
 assume you're referring to compile-time.  With the new patch you can
 pack and unpack Variants from TypeInfo and a void* so you DON'T need to
 know the type ahead of time.  There isn't a way to construct from a
 single TypeInfo,void* pair since I wasn't sure if that would actually be
 needed.


Okay, you must have done that very recently, since I can't find it in my 
working copy.


 toString is a design issue, not a technical one; they didn't want stuff
 in tango.core depending on stuff outside of that.  If you look at the
 history of Variant, there's an old version with "proper" toString code
 that should port forward painlessly.


Yes, I'm familiar with that issue. I'm not faulting Tango for lacking a 
proper toString, but it was a problem, one that I couldn't work around 
with Tango's Variant, so I did what I had to do.


 On a side note, I'm working on getting Tango's Layout to understand
 Variants and do the "proper" stringification.  Still trying to work out
 how to get the changes to compile without having to rebuild Tango...
 *grumble*
 
 As for comparisons, it supports opEqual and opCmp between Variants, it
 just doesn't attempt to do any automatic conversion between
 non-identical but comparable types (like comparing ints to longs.)


Ah. Mine does that -- it stores all floating point types as reals, for 
instance, and all integer types as longs (except for ulongs).


 Is there something specific you think Tango's Variant should do that it
 doesn't?


Apparently, nothing that won't be in the next release, aside from 
bitwise comparisons.


   -- Daniel
Apr 17 2009
prev sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Christopher Wright (dhasenan gmail.com)'s article

 Hi all,
 I finally found some time to work on TypeInfoEx. It's usable, in that
 you can view information about a type; but you can't do anything with
 that information. One large barrier is the lack of a usable Variant type.
 To invoke a function on an object or struct, you need to pass it as
 either Object or void* or Variant.
 If you invoke a function, you want to pass in an array of Variant for
 the arguments. You want to get the return type as a Variant.
 If you get a field, you want to get it as a Variant.
 You could use templates instead, but that's not usable in many
 situations, so I won't have that as the only option.
 Andrei's std.variant has an interesting design, but it has two flaws
 that render it unusable:
   - It cannot be used with structs of arbitrary size. It's parameterized
 on size, but no size will always be sufficient; and due to its
 allocation scheme, choosing to support (for instance) 1KB structs will
 force a bool to take 1KB.
   - It cannot be created with RTTI; it requires compile-time type
 information.
 So, my options are:
   - Write my own Variant supporting the necessary operations. This makes
 things annoying for anyone wanting to use my code with Phobos, since now
 there are two Variant structs running around.
   - Use std.boxer, which is deprecated.
   - Pass around an opaque Pair!(TypeInfo, void*).
 Any suggestions? I've written such a Variant for D1 and Tango, but I
 relish neither the idea of supporting both with the same class, nor code
 duplication.


Andrei has said that he would like to fix std.variant to support arbitrary sized
stuff, but just doesn't have time right now.  If you are able and willing,
submit
a patch.  If it's decent, it will probably be accepted.  If you don't have time
or
don't want to, someone who does should submit one since this seems to be
something
a lot of people want.  I've looked at doing this before, and I'd be willing to
step up and submit a patch after some of my coursework dies down and I have more
free time (fairly soon), if noone else gets around to it before then.
Apr 16 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
dsimcha wrote:

 == Quote from Christopher Wright (dhasenan gmail.com)'s article

 Hi all,
 I finally found some time to work on TypeInfoEx. It's usable, in that
 you can view information about a type; but you can't do anything with
 that information. One large barrier is the lack of a usable Variant type.
 To invoke a function on an object or struct, you need to pass it as
 either Object or void* or Variant.
 If you invoke a function, you want to pass in an array of Variant for
 the arguments. You want to get the return type as a Variant.
 If you get a field, you want to get it as a Variant.
 You could use templates instead, but that's not usable in many
 situations, so I won't have that as the only option.
 Andrei's std.variant has an interesting design, but it has two flaws
 that render it unusable:
   - It cannot be used with structs of arbitrary size. It's parameterized
 on size, but no size will always be sufficient; and due to its
 allocation scheme, choosing to support (for instance) 1KB structs will
 force a bool to take 1KB.
   - It cannot be created with RTTI; it requires compile-time type
 information.
 So, my options are:
   - Write my own Variant supporting the necessary operations. This makes
 things annoying for anyone wanting to use my code with Phobos, since now
 there are two Variant structs running around.
   - Use std.boxer, which is deprecated.
   - Pass around an opaque Pair!(TypeInfo, void*).
 Any suggestions? I've written such a Variant for D1 and Tango, but I
 relish neither the idea of supporting both with the same class, nor code
 duplication.


 
 Andrei has said that he would like to fix std.variant to support arbitrary
sized
 stuff, but just doesn't have time right now.  If you are able and willing,
submit
 a patch.  If it's decent, it will probably be accepted.  If you don't have
time or
 don't want to, someone who does should submit one since this seems to be
something
 a lot of people want.  I've looked at doing this before, and I'd be willing to
 step up and submit a patch after some of my coursework dies down and I have
more
 free time (fairly soon), if noone else gets around to it before then.


It would be great if somebody could find the time to implement the pesky 
arbitrarily-sized structs for Variant.

I don't have much time to add to this interesting discussion, but let me 
make a few quick points. First off, I'm very happy that discriminated 
unions are receiving increased interest from the community. I think they 
are an interesting abstraction mechanism with cool applications in 
dynamic code.

Second, I have hacked discriminated unions for years and wrote a fairly 
influential paper on them as far back as 2001 (which has inspired many 
others, including Boost's), and some more articles later on. I can say 
that std.variant is the best implementation of a discriminated I've 
seen, by a mile. std.variant is truly general (at no point in its design 
or implementation appears a list of supported types), well positioned 
for working with full introspection when that will become available, 
tight in memory usage, and extremely fast because it uses only one 
indirect call for dispatching. The latter point is very important in 
heavy-duty usage; hash-based dispatch sounds and looks good on paper, 
until you need to invoke it in a tight loop. Then you'd start trying to 
avoid it, which undermines motivation for using variants.

Lack of construction from RTTI is not a major limitation and can be 
obviated. RTTI objects cannot be forged, so they were created from some 
objects that did have static type information available. This suggests 
that code can be shuffled such that Variant construction is done at that 
point, instead of getting the RTTI alone (and losing static type 
information) and attempting to construct the Variant later.


Andrei
Apr 16 2009