www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - TypeInfo in the library

reply "Adam D. Ruppe" <destructionator gmail.com> writes:
I'd like to see typeinfo moved completely to the library. The 
language would then not depend on it directly. The point of this 
thread is to see how practical it is. Here's the changes I have 
in mind:

1) Stop automatic generation of TypeInfo.

2) typeid(T) in the language now returns a static instance of 
TypeInfoImpl!T. The vtbl entry in a class also points to this. If 
the library doesn't implement TypeInfoImpl(T), these return null.

3) Add __traits(classInitializer, Class) which returns what we 
have today through typeid(Class).init. (This would be used to 
make init in the library implementation)


I think the others are already possible:

string TypeInfoImpl!T.toString() { return T.stringof; }
interface TypeInfo {} /* TypeInfoImpl(T) : TypeInfo */

// ElementType used for illustration only, no need to depend on 
phobos since we can do this with an is expression too. that's 
just harder to read/write lol

TypeInfo TypeInfoImpl!T.next() { return typeid(ElementType!T); }

Interfaces and parent class can also be done with traits today, 
so we can auto-generate the with a template too.


And so on, look at druntime's typeinfos now, most of it is just 
doing a bunch of casts of the same code - easily templated.



If I'm not missing anything, these three changes would let us 
completely define (or not define) type info in the library 
without breaking anything we have now!


Functions that depend on typeinfo can be generated the same way 
as they are now, though of course, I'd also enjoy moving more of 
those things (arrays, etc.) to be templates too :)
Feb 13 2014
next sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
But that would mean that you have to pull the implementation of casts 
and other features that rely on TypeInfo right now into the library too. 
Do you really think this such a good idea? My biggest concern with this 
is, that when TypeInfo is pulled into phobos, the implementation of 
TypeInfo is going to depend on _all_ phobos modules, making at lot 
harder to use D without phobos unless you want to rewrite the entire 
TypeInfo and all features depending on it.
Feb 14 2014
next sibling parent reply "Mike" <none none.com> writes:
On Friday, 14 February 2014 at 09:09:42 UTC, Benjamin Thaut wrote:

 But that would mean that you have to pull the implementation of 
 casts and other features that rely on TypeInfo right now into 
 the library too. Do you really think this such a good idea? My 
 biggest concern with this is, that when TypeInfo is pulled into 
 phobos, the implementation of TypeInfo is going to depend on 
 _all_ phobos modules, making at lot harder to use D without 
 phobos unless you want to rewrite the entire TypeInfo and all 
 features depending on it.


I think Adam would have to clarify what he meant by "the 
library".  I'd like all the TypeInfo stuff in the runtime, not 
phobos.  According to a conversation I had with Iain Buclaw [1], 
the TypeInfo implementations are broken into two parts:  There's 
the TypeInfo implementations in object.d/di and then there's the 
TypeInfo implementations in D runtime's src/rt/typeinfo folder 
[2].

I don't understand why the two are separated.  Iain just said the 
ones in object.d "are required to be known at compile time for 
certain operations."

I'm too much of a D novice to understand the implications of 
Adam's proposal, but if it's feasible, I think it would be be 
quite useful to those porting D to more limited platforms like I 
am.  Currently I'm fooling the compiler with silly things like...

class TypeInfo_Class : TypeInfo
{
     ubyte[68] ignore;
}

... and that's a dirty hack if I ever saw one.

The runtime might be a good place for it given the consequences 
you've just articulated.  The bottom line, I think, is to get it 
out of the compiler.

Mike

[1] 
http://forum.dlang.org/post/mailman.211.1389183225.15871.digitalmars-d puremagic.com
[2] 
https://github.com/D-Programming-Language/druntime/tree/master/src/rt/typeinfo
Feb 14 2014
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 14 February 2014 at 11:26:42 UTC, Mike wrote:

 I think Adam would have to clarify what he meant by "the 
 library".


You got it.
Feb 14 2014
prev sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 14 February 2014 at 09:09:42 UTC, Benjamin Thaut wrote:

 But that would mean that you have to pull the implementation of 
 casts and other features that rely on TypeInfo right now into 
 the library too.


They already are: druntime/src/rt/cast_.d I wouldn't change that, 
I'm more thinking about changing rt/typeinfo/*.d from being 
hand-written to auto generated by a template and moving the 
object size and layout from the compiler (see verifyStructSize in 
dmd) to the druntime code too.

No phobos here, that would definitely be a bad idea.


 making at lot harder to use D without phobos unless you want to 
 rewrite the entire TypeInfo and all features depending on it.


Right now, if you try to use D without druntime, it will complain 
about missing typeinfos and force you to create the classes with 
a particular size. Much of it can be auto generated by templates 
now (which gives a lot of cool flexibility btw), but you are 
still locked into the compiler's fairly strict layout. I want to 
open up some flexibility.
Feb 14 2014
parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 14.02.2014 16:32, schrieb Adam D. Ruppe:

 On Friday, 14 February 2014 at 09:09:42 UTC, Benjamin Thaut wrote:

 But that would mean that you have to pull the implementation of casts
 and other features that rely on TypeInfo right now into the library too.


 They already are: druntime/src/rt/cast_.d I wouldn't change that, I'm
 more thinking about changing rt/typeinfo/*.d from being hand-written to
 auto generated by a template and moving the object size and layout from
 the compiler (see verifyStructSize in dmd) to the druntime code too.

 No phobos here, that would definitely be a bad idea.


 making at lot harder to use D without phobos unless you want to
 rewrite the entire TypeInfo and all features depending on it.


 Right now, if you try to use D without druntime, it will complain about
 missing typeinfos and force you to create the classes with a particular
 size. Much of it can be auto generated by templates now (which gives a
 lot of cool flexibility btw), but you are still locked into the
 compiler's fairly strict layout. I want to open up some flexibility.


Ok, I misunderstood you then. Moving TypeInfo generation to druntime 
would actually be a good idea. Ocasionally I wanted to extend TypeInfo 
for various use cases.

I said using D without phobos, I never said using D without druntime. 
Thats a difference.
Feb 16 2014
prev sibling next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 14.02.2014 01:42, Adam D. Ruppe wrote:

 I'd like to see typeinfo moved completely to the library. The language
 would then not depend on it directly. The point of this thread is to see
 how practical it is. Here's the changes I have in mind:

 1) Stop automatic generation of TypeInfo.

 2) typeid(T) in the language now returns a static instance of
 TypeInfoImpl!T. The vtbl entry in a class also points to this. If the
 library doesn't implement TypeInfoImpl(T), these return null.

 3) Add __traits(classInitializer, Class) which returns what we have
 today through typeid(Class).init. (This would be used to make init in
 the library implementation)


I like this idea (I remember I had this also as a suggestion on my 
slides for dconf2013). This is just taking the RTInfo template 
generation a step further, yet coming with the same problems. See this 
pull request for a (probably incomplete) list of places I found where 
type info generation is triggered from the glue code, and not during 
semantic analysis:

https://github.com/D-Programming-Language/dmd/pull/2480

The only drawback I see is that it makes compilation a bit slower and 
pollutes object files with code just executed during CTFE. Unfortunately 
this often also ends up in the executable because it is referenced by 
some data not in COMDATs.
Feb 14 2014
parent reply "Max Samukha" <maxsamukha gmail.com> writes:
On Friday, 14 February 2014 at 20:05:53 UTC, Rainer Schuetze 
wrote:


 The only drawback I see is that it makes compilation a bit 
 slower and pollutes object files with code just executed during 
 CTFE.


Currently, the bloat can be reduced like this:

void foo()
{
   if (__ctfe)
   {
     // function body
   }
   else
     asm { naked; }
}

But that's an awful dangerous hack. A better solution is needed.
Feb 17 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 17 February 2014 at 09:33:01 UTC, Max Samukha wrote:

 But that's an awful dangerous hack. A better solution is needed.


I have been thinking about addition of something like 
`pragma(nocodegen)` to the language. Should be simple to 
implement and can fix some of most crazy bloat issues in 
libraries until something like whole-program optimization can 
become practical reality.

It will help not only CTFE - for example, all template 
constraints and all template argument list metaprogramming 
facilities can be annotated with it in Phobos.

Walter, would you approve something like this?
Feb 17 2014
next sibling parent reply Kenji Hara <k.hara.pg gmail.com> writes:
If a function is only used for CTFE, compiler can elide its codegen. It's
in the scope of optimization.

Kenji Hara


2014-02-17 22:48 GMT+09:00 Dicebot <public dicebot.lv>:


 On Monday, 17 February 2014 at 09:33:01 UTC, Max Samukha wrote:


 But that's an awful dangerous hack. A better solution is needed.


 I have been thinking about addition of something like `pragma(nocodegen)`
 to the language. Should be simple to implement and can fix some of most
 crazy bloat issues in libraries until something like whole-program
 optimization can become practical reality.

 It will help not only CTFE - for example, all template constraints and all
 template argument list metaprogramming facilities can be annotated with it
 in Phobos.

 Walter, would you approve something like this?
Feb 17 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 17 February 2014 at 14:51:20 UTC, Kenji Hara wrote:

 If a function is only used for CTFE, compiler can elide its 
 codegen. It's
 in the scope of optimization.


It can't know for sure it without some sort of WPO / LTO - and 
those won't happen any time soon. I propose a simple workaround 
that can be done right now (it can be optional compiler-specific 
addition)
Feb 17 2014
parent reply Kenji Hara <k.hara.pg gmail.com> writes:
2014-02-18 0:48 GMT+09:00 Dicebot <public dicebot.lv>:


 On Monday, 17 February 2014 at 14:51:20 UTC, Kenji Hara wrote:


 If a function is only used for CTFE, compiler can elide its codegen. It's
 in the scope of optimization.


 It can't know for sure it without some sort of WPO / LTO - and those won't
 happen any time soon. I propose a simple workaround that can be done right
 now (it can be optional compiler-specific addition)


In some case it would be possible without WPO/LTO:

- Immediately called lambdas

  enum x = (){ return some_sort_ctfe_calculartions; };   // lambda codegen
is unnecessary.

- modue private functions

  module a;
  private string foo() { ... }
  enum x = foo();
  // Compiler can elide codegen for 'foo', if other declarations
  // in module a don't use it for runtime code.

Kenji Hara
Feb 17 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 17 February 2014 at 16:01:50 UTC, Kenji Hara wrote:

 - Immediately called lambdas

   enum x = (){ return some_sort_ctfe_calculartions; };   // 
 lambda codegen
 is unnecessary.


True but this is only one of many cases.


 - modue private functions

   module a;
   private string foo() { ... }
   enum x = foo();
   // Compiler can elide codegen for 'foo', if other declarations
   // in module a don't use it for runtime code.


It is not that simple. Other declarations can also be templates 
or CTFE functions (but public) - compiler can't know if those 
will be used in runtime codegen in other modules. And being 
conservative will make optimization much less useful.

This is exactly why I am referencing WPO/LTO - doing this is 
plain optimization is theoretically possible but is huge effort 
to implement in somewhat reliable way. I'd like to have something 
simple that can help right now and turned into no-op later once 
compiler becomes clever enough.
Feb 17 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Tue, 18 Feb 2014 01:01:18 +0900
schrieb Kenji Hara <k.hara.pg gmail.com>:


 In some case it would be possible without WPO/LTO:
 
 - Immediately called lambdas
 
   enum x = (){ return some_sort_ctfe_calculartions; };   // lambda
 codegen is unnecessary.
 
 - modue private functions
 
   module a;
   private string foo() { ... }
   enum x = foo();
   // Compiler can elide codegen for 'foo', if other declarations
   // in module a don't use it for runtime code.
 
 Kenji Hara
 


- Template instances of any module if the instance is only used in CTFE
Feb 17 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-02-17 17:01, Kenji Hara wrote:


 - modue private functions

    module a;
    private string foo() { ... }
    enum x = foo();
    // Compiler can elide codegen for 'foo', if other declarations
    // in module a don't use it for runtime code.


Private functions can be called from other modules via function pointers.

-- 
/Jacob Carlborg
Feb 17 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Jacob Carlborg"  wrote in message news:ldtrf3$14t0$1 digitalmars.com... 


 On 2014-02-17 17:01, Kenji Hara wrote:

 - modue private functions
    // Compiler can elide codegen for 'foo', if other declarations
    // in module a don't use it for runtime code.





 Private functions can be called from other modules via function pointers.


That would be a runtime use of 'foo', to get the function pointer.
Feb 17 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-02-18 03:16, Daniel Murphy wrote:


 That would be a runtime use of 'foo', to get the function pointer.


Other modules can get the function pointer as well. Or perhaps that's 
what you're saying.

-- 
/Jacob Carlborg
Feb 18 2014
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
17-Feb-2014 17:48, Dicebot пишет:

 On Monday, 17 February 2014 at 09:33:01 UTC, Max Samukha wrote:

 But that's an awful dangerous hack. A better solution is needed.


 I have been thinking about addition of something like
 `pragma(nocodegen)` to the language. Should be simple to implement and
 can fix some of most crazy bloat issues in libraries until something
 like whole-program optimization can become practical reality.


Like it!


-- 
Dmitry Olshansky
Feb 17 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-02-14 01:42, Adam D. Ruppe wrote:


 I think the others are already possible:

 string TypeInfoImpl!T.toString() { return T.stringof; }
 interface TypeInfo {} /* TypeInfoImpl(T) : TypeInfo */


Currently what .stringof evaluates to and the format of it is not 
specified and should not be relied. I assume the current 
typeid(T).toString is reliable.


 // ElementType used for illustration only, no need to depend on phobos
 since we can do this with an is expression too. that's just harder to
 read/write lol

 TypeInfo TypeInfoImpl!T.next() { return typeid(ElementType!T); }

 Interfaces and parent class can also be done with traits today, so we
 can auto-generate the with a template too.


What about the rest of the fields, in ClassInfo, for example? Implement 
with traits? I'm thinking of "classInvariant", "m_flags", "deallocator", 
"defaultConstructor" and so on.

-- 
/Jacob Carlborg
Feb 15 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/13/2014 4:42 PM, Adam D. Ruppe wrote:

 If I'm not missing anything, these three changes would let us completely define
 (or not define) type info in the library without breaking anything we have now!


It's a good idea worth exploring.
Feb 16 2014
next sibling parent "Mike" <none none.com> writes:
On Sunday, 16 February 2014 at 20:19:13 UTC, Walter Bright wrote:

 It's a good idea worth exploring.


Walter, I'm glad you voiced your opinion on this.

In my opinion, this should be a prerequisite to the minimalD 
ideas that are currently being discussed.  While I don't want to 
ask you to step away from more high value tasks that benefit the 
larger community, I'm wondering...would you be willing to mentor 
this change and help the community do the implementation?

I don't have the skills to implement this (yet), so is there 
anyone else out there that I could support (bribe) to make this 
change happen?

Mike
Feb 16 2014
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:


 On 2/13/2014 4:42 PM, Adam D. Ruppe wrote:

 If I'm not missing anything, these three changes would let us 
 completely define
 (or not define) type info in the library without breaking 
 anything we have now!


 It's a good idea worth exploring.


And at this stage of the D life it's acceptable to even have tiny 
breaking changes, if they give back significant good things. A 
purely library implementation of TypeInfo could bring some 
advantages.

Bye,
bearophile
Feb 16 2014
prev sibling next sibling parent reply "Mike" <none none.com> writes:
On Friday, 14 February 2014 at 00:42:08 UTC, Adam D. Ruppe wrote:

 I'd like to see typeinfo moved completely to the library. The 
 language would then not depend on it directly. The point of 
 this thread is to see how practical it is. Here's the changes I 
 have in mind:

 1) Stop automatic generation of TypeInfo.

 2) typeid(T) in the language now returns a static instance of 
 TypeInfoImpl!T. The vtbl entry in a class also points to this. 
 If the library doesn't implement TypeInfoImpl(T), these return 
 null.

 3) Add __traits(classInitializer, Class) which returns what we 
 have today through typeid(Class).init. (This would be used to 
 make init in the library implementation)


 I think the others are already possible:

 string TypeInfoImpl!T.toString() { return T.stringof; }
 interface TypeInfo {} /* TypeInfoImpl(T) : TypeInfo */

 // ElementType used for illustration only, no need to depend on 
 phobos since we can do this with an is expression too. that's 
 just harder to read/write lol

 TypeInfo TypeInfoImpl!T.next() { return typeid(ElementType!T); }

 Interfaces and parent class can also be done with traits today, 
 so we can auto-generate the with a template too.


 And so on, look at druntime's typeinfos now, most of it is just 
 doing a bunch of casts of the same code - easily templated.



 If I'm not missing anything, these three changes would let us 
 completely define (or not define) type info in the library 
 without breaking anything we have now!


 Functions that depend on typeinfo can be generated the same way 
 as they are now, though of course, I'd also enjoy moving more 
 of those things (arrays, etc.) to be templates too :)


Added enhancement bugzilla because it's a great idea: 
https://d.puremagic.com/issues/show_bug.cgi?id=12270
Feb 26 2014
parent reply "Trass3r" <un known.com> writes:
Any news on this?
Jul 13 2014
next sibling parent "Mike" <none none.com> writes:
On Sunday, 13 July 2014 at 15:30:59 UTC, Trass3r wrote:

 Any news on this?


It's waiting on you or I to do the implementation and submit a 
pull request.  If it needs to be me, it'll probably be done 
sometime towards the end of 2016.  And that assumes the current 
desire to fossilize D doesn't prevent such changes from being 
accepted.

Mike
Jul 13 2014
prev sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 13 July 2014 at 15:30:59 UTC, Trass3r wrote:

 Any news on this?


I haven't messed around with it since the winter and probably 
won't be able to for a while. (The current setup works well 
enough for me anyway...)
Jul 15 2014
prev sibling parent "Mike" <none none.com> writes:
On Friday, 14 February 2014 at 00:42:08 UTC, Adam D. Ruppe wrote:

 I'd like to see typeinfo moved completely to the library. The 
 language would then not depend on it directly. The point of 
 this thread is to see how practical it is. Here's the changes I 
 have in mind:


I'm thinking about trying to implementing this.  There's some 
promising work in GDC[1] and DMD[2] related to reigning in 
TypeInfo, but it comes with compromises (e.g. not being able to 
use some of the convenient array-related functions and postblit).

[1] (GDC -fno-typeinfo) - 
https://github.com/D-Programming-GDC/GDC/pull/100
[2] (DMD Resolve TypeInfo variables after semantic) - 
https://github.com/D-Programming-Language/dmd/pull/4654

However, that work has helped me better understand what the 
compiler's doing and, with a little more study, I might be able 
to tackle this implementation.


 1) Stop automatic generation of TypeInfo.

 2) typeid(T) in the language now returns a static instance of 
 TypeInfoImpl!T. The vtbl entry in a class also points to this. 
 If the library doesn't implement TypeInfoImpl(T), these return 
 null.

 3) Add __traits(classInitializer, Class) which returns what we 
 have today through typeid(Class).init. (This would be used to 
 make init in the library implementation)


I'm wondering if (2) and (3) need happen to keep the status quo.  
typeid(T) would need to return TypeInfoImpl!(T), but if 
TypeInfoImpl!(T) is absent from the runtime, we could just throw 
a compiler error.  Currently if no TypeInfo is implemented in the 
runtime, errors happen, so we'd be no worse off.

For (3), is the trait really needed to keep the status quo? 
Again, the compiler would just throw an error if 
TypeInfoImpl!T.init() is not found.

As a result, the runtime would still need to have a 
TypeInfoImpl!(T) to get a build, but at least only what's 
actually used would need to be implemented.

Am I missing something?

Also, I think it solve the TypeInfo code bloat problem [3] when 
compiling with -ffunction-sections/-fdata-sections/--gc-sections. 
  The reason being that the compiler would no longer be "packing" 
the TypeInfo as alluded to in [4].

[3] (TypeInfo.name strings don't get put into separate sections 
when compiling with -fdata-sections) - 
http://bugzilla.gdcproject.org/show_bug.cgi?id=184
[4] (TypeInfo Packing) - 
http://forum.dlang.org/post/lfyehafdfrrktbwqemhu forum.dlang.org

Mike
Jun 04 2015