www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Size of a class instance at compile time

reply Daniel Keep <daniel.keep+lists gmail.com> writes:
Ok, ASSUMING:

1. I'm only going to be compiling with dmd...
2. ...for x86...
3. ...under Windows,

is this an accurate way of getting the size of a class instance at 
compile time?

 template InstanceSize(T)
 {
     const uint InstanceSize = 12 + CountSizes!(typeof(T.tupleof));
 }

 template CountSizes()
 {
     const uint CountSizes = 0;
 }

 template CountSizes(T)
 {
     const uint CountSizes = T.sizeof;
 }

 template CountSizes(T, Tail...)
 {
     const uint CountSizes = T.sizeof + CountSizes!(Tail);
 }

It seems to report the right sizes, but I'm not entirely sure what the '12' is for. Pointer to ClassInfo, ... what else? For those wondering: I'm doing this for some custom mixin memory allocators I'm playing with. -- Daniel "isizeof, isizeof, my kingdom for isizeof!" -- "Rick 3" by Bill Wobblepike.
Jan 13 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Daniel Keep wrote:
 
 Ok, ASSUMING:
 
 1. I'm only going to be compiling with dmd...
 2. ...for x86...
 3. ...under Windows,

(2) is a bit redundant after (1)... Also, AFAIK (3) doesn't matter in this respect.
 is this an accurate way of getting the size of a class instance at 
 compile time?
 
  > template InstanceSize(T)
  > {
  >     const uint InstanceSize = 12 + CountSizes!(typeof(T.tupleof));
  > }
  >
  > template CountSizes()
  > {
  >     const uint CountSizes = 0;
  > }
  >
  > template CountSizes(T)
  > {
  >     const uint CountSizes = T.sizeof;
  > }
  >
  > template CountSizes(T, Tail...)
  > {
  >     const uint CountSizes = T.sizeof + CountSizes!(Tail);
  > }

Sorry, it's not.
 It seems to report the right sizes, but I'm not entirely sure what the 
 '12' is for.  Pointer to ClassInfo, ... what else?

There are only 8 bytes of overhead: pointer to vtable and pointer to monitor. (the vtable contains the classinfo pointer) See http://www.digitalmars.com/d/abi.html, under "Classes". It doesn't specifically mention how non-static members are formatted, but I think I figured it out (see below). You also forgot to take alignment gaps into account. Try this: ----- template InstanceSize(T) { const InstanceSize = CountSizes!(8, typeof(T.tupleof)); } template CountSizes(size_t N) { const CountSizes = N; } template CountSizes(size_t N, T) { const uint CountSizes = AlignUp!(T.alignof, N) + T.sizeof; } template CountSizes(size_t N, T, Tail...) { const uint CountSizes = CountSizes!(AlignUp!(T.alignof, N) + T.sizeof, Tail); } template AlignUp(size_t Align, size_t N) { static assert((Align & (Align - 1)) == 0, "Alignment not power of two"); const AlignUp = (N + Align - 1) & ~(Align - 1); } // // Test code // class Test { /// Change members to test different layouts char c; int i; byte b; } import std.stdio; void main() { writefln("Static size: ", InstanceSize!(Test)); writefln("Actual size: ", Test.classinfo.init.length); } -----
Jan 13 2007
parent reply Sean Kelly <sean f4.ca> writes:
Frits van Bommel wrote:
 
 You also forgot to take alignment gaps into account.

I think this may be a problem. The ABI allows class data to be rearranged, so alignment gaps can't be predicted (I think) from the output of tupleof. I still wish we could get .isizeof for this :-p Sean
Jan 13 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Sean Kelly wrote:
 Frits van Bommel wrote:
 You also forgot to take alignment gaps into account.

I think this may be a problem. The ABI allows class data to be rearranged, so alignment gaps can't be predicted (I think) from the output of tupleof. I still wish we could get .isizeof for this :-p

As I noted earlier in that post, the ABI indeed doesn't specify the format of the non-static member part of an object. It seems to be one of the incomplete parts. But from my experiments it seems the current compiler (v1.0) doesn't reorder fields. I've tested my code with several layouts and AFAICT it consistently provides the same answer as a runtime classinfo.init.length. So assuming no edge cases[1] are triggered it should work until the next ABI break at the very least... Since ABI breaks seem to be indicated by red 'Need to recompile because:' clauses in the changelog, it should be pretty clear when this needs to be changed. [1]: e.g. I haven't tested using anything with explicit align(N) declarations, not sure what happens there (would .alignof be adjusted?).
Jan 13 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Frits van Bommel wrote:
 [1]: e.g. I haven't tested using anything with explicit align(N) 
 declarations, not sure what happens there (would .alignof be adjusted?).

Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------ _Way_ shorter, and always provides the correct answer under two simple assumptions: 1) The last non-static member is also last in the layout 2) There's no padding at the end. Assumption (1) is pretty likely to be broken if and when member reordering is implemented, though.
Jan 13 2007
next sibling parent Daniel Keep <daniel.keep+lists gmail.com> writes:
Frits van Bommel wrote:
 Frits van Bommel wrote:
 
 [1]: e.g. I haven't tested using anything with explicit align(N) 
 declarations, not sure what happens there (would .alignof be adjusted?).

Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------ _Way_ shorter, and always provides the correct answer under two simple assumptions: 1) The last non-static member is also last in the layout 2) There's no padding at the end. Assumption (1) is pretty likely to be broken if and when member reordering is implemented, though.

Oh that's beautiful; I can't believe I forgot about offsetof! Thanks very much for that: this should make the allocators a bit more efficient :) -- Daniel
Jan 13 2007
prev sibling parent reply Daniel Keep <daniel.keep+lists gmail.com> writes:
Frits van Bommel wrote:
 Frits van Bommel wrote:
 
 [1]: e.g. I haven't tested using anything with explicit align(N) 
 declarations, not sure what happens there (would .alignof be adjusted?).

Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------

Just wondering; do we even need to worry about align(N) attributes? AFAIK, that only applies to structures, and we can get the size of those using sizeof since they're value types. http://digitalmars.com/d/attribute.html#align -- says down the bottom of that heading that it only applies to structs and struct members.
 _Way_ shorter, and always provides the correct answer under two simple 
 assumptions:
 1) The last non-static member is also last in the layout
 2) There's no padding at the end.
 
 Assumption (1) is pretty likely to be broken if and when member 
 reordering is implemented, though.

I just knocked up a slightly more robust version that should *theoretically* still work even if the members are moved around. It basically just does the same thing yours does, but it runs over the whole tuple, and passes out the largest value it finds. Obviously, it doesn't help for the above two points, but Walter seems to like stuff at the start of the block, not trailing off the end. Plus, when would a class require padding? -- Daniel template InstanceSize(T) { const InstanceSize = InstanceSizeImpl!(T, 0); } template InstanceSizeImpl(T, size_t i) { static if( i < T.tupleof.length ) const InstanceSizeImpl = Max!( T.tupleof[i].offsetof + T.tupleof[i].sizeof, InstanceSizeImpl!(T, i+1)); else const InstanceSizeImpl = 0u; } template Max(size_t a, size_t b) { static if( a > b ) const Max = a; else const Max = b; }
Jan 14 2007
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Daniel Keep wrote:
 Frits van Bommel wrote:
 Frits van Bommel wrote:

 [1]: e.g. I haven't tested using anything with explicit align(N) 
 declarations, not sure what happens there (would .alignof be adjusted?).

Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------

Just wondering; do we even need to worry about align(N) attributes? AFAIK, that only applies to structures, and we can get the size of those using sizeof since they're value types.

They work for classes too. Not sure if it's documented, but in my test file I have the following: ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } class Test { char c; int i; } class Test2 { char c; align(1) int i; } import std.stdio; void main() { writefln("Test:"); writefln("Static size: ", InstanceSize!(Test)); writefln("Actual size: ", Test.classinfo.init.length); writefln(); writefln("Test2:"); writefln("Static size: ", InstanceSize!(Test2)); writefln("Actual size: ", Test2.classinfo.init.length); } ----- Which gives size 16 and 13 for Test and Test2, respectively. So alignment declarations definitely makes a difference even for classes.
 http://digitalmars.com/d/attribute.html#align -- says down the bottom of 
 that heading that it only applies to structs and struct members.

Then I guess either the spec or the implementation is wrong.
 I just knocked up a slightly more robust version that should 
 *theoretically* still work even if the members are moved around.  It 
 basically just does the same thing yours does, but it runs over the 
 whole tuple, and passes out the largest value it finds.

Ah, that's a good solution to that problem I guess. Unless there's ever any weird need to put overhead bytes at the end that should work. (Where are interface vtable pointers put? [tests] Damn, looks like they're at the end :( )
 Obviously, it doesn't help for the above two points, but Walter seems to 
 like stuff at the start of the block, not trailing off the end.  Plus, 
 when would a class require padding?

Apparently, implementing an interface counts as well...
Jan 14 2007
next sibling parent Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
Frits van Bommel wrote:
 Ah, that's a good solution to that problem I guess.
 Unless there's ever any weird need to put overhead bytes at the end that 
 should work.
 (Where are interface vtable pointers put? [tests] Damn, looks like 
 they're at the end :( )

So I was wondering, could is(T supertuple == super) be of help here? Specifically, how many interface vptrs are there in an object? One per interface? Or are inherited interfaces folded like classes, one vptr per most-derived interface? I think this could be made to work if we figure out the details. It looks to a bit complicated though, since I'm guessing interfaces of base classes come before members of derived classes. That plus double.alignof > (void*).alignof on x86 means that to determine alignment gaps correctly we'd probably need to work from Object up... I don't feel like coding that right now :(.
Jan 14 2007
prev sibling parent reply Daniel Keep <daniel.keep+lists gmail.com> writes:
Frits van Bommel wrote:
 Daniel Keep wrote:
 I just knocked up a slightly more robust version that should 
 *theoretically* still work even if the members are moved around.  It 
 basically just does the same thing yours does, but it runs over the 
 whole tuple, and passes out the largest value it finds.

Ah, that's a good solution to that problem I guess. Unless there's ever any weird need to put overhead bytes at the end that should work. (Where are interface vtable pointers put? [tests] Damn, looks like they're at the end :( )
 Obviously, it doesn't help for the above two points, but Walter seems 
 to like stuff at the start of the block, not trailing off the end.  
 Plus, when would a class require padding?

Apparently, implementing an interface counts as well...

Oh... BUGGER. Well, that's inconvenient. I guess I'm just going to have to keep trying until I either find a way to do this properly, or Walter adds isizeof :P Interface vtable pointers; they're just regular D arrays, right? One for each interface? Here it is: expression.html#IsExpression -- we can use is(T I == super) to get a tuple of the base classes and all the interfaces. If we can accurately predict how large the extra padding is from the number of interfaces, it's sorted. Time to go test the theory, then :) -- Daniel
Jan 14 2007
parent reply Daniel Keep <daniel.keep+lists gmail.com> writes:
Got it!  No, really, this time FOR SURE ^_^

The end solution is utterly evil, but is, AFAIK, correct.

What I had to do in the end was to say that the size of a class instance 
is the maximum of:

1) size of the base class + if(implements interfaces)
            padding + interface count * pointer size,
2) max(class member alignof + sizeof) + if(implements interfaces)
            padding + interface count * pointer size

I also added a minimum size of 2*pointer size to account for classes 
that don't have any member variables.

Getting the padding for interfaces was fun:

 struct Align
 {
     ubyte a;
     void* b;
 }

 const PTR_ALIGN = Align.tupleof[1].alignof;

I *think* that's right. In any case, it's reporting the correct size for every test case I throw at it. Anything else that needs fixing? :P -- Daniel private struct Align { ubyte a; void* b; } private const PTR_ALIGN = Align.tupleof[1].alignof; private template AlignPad(size_t base, size_t aligned) { static if( aligned == 0 ) const AlignPad = base; else const AlignPad = ((base+PTR_ALIGN-1)/PTR_ALIGN)*PTR_ALIGN + aligned; } template InstanceSize(T) { static if( is( T == Object ) ) const InstanceSize = 2*(void*).sizeof; else const InstanceSize = Max!( AlignPad!( InstanceSize!(Super!(T)), InterfaceCount!(T)*(void*).sizeof), AlignPad!( InstanceSizeImpl!(T, 0), + InterfaceCount!(T)*(void*).sizeof)); } private template Super(T) { static if( is( T S == super ) ) alias First!(S) Super; else static assert(false, "Can't get super of "~T.mangleof); } private template First(T) { alias T First; } private template First(T, Ts...) { alias T First; } private template InstanceSizeImpl(T, size_t i) { static if( i < T.tupleof.length ) const InstanceSizeImpl = Max!( T.tupleof[i].offsetof + T.tupleof[i].sizeof, InstanceSizeImpl!(T, i+1)); else // This is necessary to account for classes without member // variables. const InstanceSizeImpl = 2*(void*).sizeof; } private template Max(size_t a, size_t b) { static if( a > b ) const Max = a; else const Max = b; } private template InstanceSizeAligned(T, size_t alignment) { static if( alignment == 0 ) const InstanceSizeAligned = InstanceSize!(T); else const uint InstanceSizeAligned = InstanceSizeAlignImpl!(T, alignment).result; } private template InstanceSizeAlignedImpl(T, size_t alignment) { private const base_size = InstanceSize!(T); const result = ((base_size+alignment-1)/alignment)*alignment; } private template InterfaceCount(T) { static if( is( T == Object ) ) const InterfaceCount = 0u; else static if( is( T S == super ) ) const InterfaceCount = InterfaceCountImpl!(S); } private template InterfaceCountImpl(TBase, TInterfaces...) { const InterfaceCountImpl = TInterfaces.length; }
Jan 14 2007
parent reply Don Clugston <dac nospam.com.au> writes:
Daniel Keep wrote:
 
 Got it!  No, really, this time FOR SURE ^_^
 
 The end solution is utterly evil, but is, AFAIK, correct.
 
 What I had to do in the end was to say that the size of a class instance 
 is the maximum of:
 
 1) size of the base class + if(implements interfaces)
            padding + interface count * pointer size,
 2) max(class member alignof + sizeof) + if(implements interfaces)
            padding + interface count * pointer size
 
 I also added a minimum size of 2*pointer size to account for classes 
 that don't have any member variables.
 
 Getting the padding for interfaces was fun:
 
  > struct Align
  > {
  >     ubyte a;
  >     void* b;
  > }
  >
  > const PTR_ALIGN = Align.tupleof[1].alignof;

Nifty!
 I *think* that's right.  In any case, it's reporting the correct size 
 for every test case I throw at it.  Anything else that needs fixing? :P

One other nasty case is where the last member of a class is an 80-bit real, with padding that is different between Windows, Linux32, and Linux64. Does that work correctly?
 
     -- Daniel
 
 private
 struct Align
 {
     ubyte a;
     void* b;
 }
 
 private
 const PTR_ALIGN = Align.tupleof[1].alignof;
 
 private
 template AlignPad(size_t base, size_t aligned)
 {
     static if( aligned == 0 )
         const AlignPad = base;
     else
         const AlignPad = ((base+PTR_ALIGN-1)/PTR_ALIGN)*PTR_ALIGN
             + aligned;
 }
 
 template InstanceSize(T)
 {
     static if( is( T == Object ) )
         const InstanceSize = 2*(void*).sizeof;
     else
         const InstanceSize = Max!(
             AlignPad!(
                 InstanceSize!(Super!(T)),
                 InterfaceCount!(T)*(void*).sizeof),
 
             AlignPad!(
                 InstanceSizeImpl!(T, 0),
                 + InterfaceCount!(T)*(void*).sizeof));
 }
 
 private
 template Super(T)
 {
     static if( is( T S == super ) )
         alias First!(S) Super;
     else
         static assert(false, "Can't get super of "~T.mangleof);
 }
 
 private
 template First(T)
 {
     alias T First;
 }
 
 private
 template First(T, Ts...)
 {
     alias T First;
 }
 
 private
 template InstanceSizeImpl(T, size_t i)
 {
     static if( i < T.tupleof.length )
         const InstanceSizeImpl = Max!(
             T.tupleof[i].offsetof + T.tupleof[i].sizeof,
             InstanceSizeImpl!(T, i+1));
     else
         // This is necessary to account for classes without member
         // variables.
         const InstanceSizeImpl = 2*(void*).sizeof;
 }
 
 private
 template Max(size_t a, size_t b)
 {
     static if( a > b )
         const Max = a;
     else
         const Max = b;
 }
 
 private
 template InstanceSizeAligned(T, size_t alignment)
 {
     static if( alignment == 0 )
         const InstanceSizeAligned = InstanceSize!(T);
     else
         const uint InstanceSizeAligned
             = InstanceSizeAlignImpl!(T, alignment).result;
 }
 
 private
 template InstanceSizeAlignedImpl(T, size_t alignment)
 {
     private const base_size = InstanceSize!(T);
     const result = ((base_size+alignment-1)/alignment)*alignment;
 }
 
 private
 template InterfaceCount(T)
 {
     static if( is( T == Object ) )
         const InterfaceCount = 0u;
     else static if( is( T S == super ) )
         const InterfaceCount = InterfaceCountImpl!(S);
 }
 
 private
 template InterfaceCountImpl(TBase, TInterfaces...)
 {
     const InterfaceCountImpl = TInterfaces.length;
 }
 

Jan 14 2007
parent reply Daniel Keep <daniel.keep+lists gmail.com> writes:
Don Clugston wrote:
 Daniel Keep wrote:
 I *think* that's right.  In any case, it's reporting the correct size 
 for every test case I throw at it.  Anything else that needs fixing? :P

One other nasty case is where the last member of a class is an 80-bit real, with padding that is different between Windows, Linux32, and Linux64. Does that work correctly?

Well, I tried sticking a real into the test cases in various spots, and it seemed fine. That said, I'm not sure what the potential problem is; it currently determines the size of the member variables by finding the maximum of (member.alignof+member.sizeof) for every member variable. This means that, even if the last member is a real, that it will correctly account for any padding *before* the real, as well as its' full size. Unless the compiler does really weird stuff like, for example, sticking in padding after a real that isn't part of its' ".sizeof" on non Windows platforms, I don't think it should be a problem. That said, if you DO find anything amiss, I'd love to know :) That reminds me. Walter: I know you're probably busy, but if you could just give this the once-over to verify that it's counting the bytes properly, that would be awesome. -- Daniel
Jan 15 2007
parent Don Clugston <dac nospam.com.au> writes:
Daniel Keep wrote:
 Don Clugston wrote:
 Daniel Keep wrote:
 I *think* that's right.  In any case, it's reporting the correct size 
 for every test case I throw at it.  Anything else that needs fixing? :P

One other nasty case is where the last member of a class is an 80-bit real, with padding that is different between Windows, Linux32, and Linux64. Does that work correctly?

Well, I tried sticking a real into the test cases in various spots, and it seemed fine. That said, I'm not sure what the potential problem is; it currently determines the size of the member variables by finding the maximum of (member.alignof+member.sizeof) for every member variable. This means that, even if the last member is a real, that it will correctly account for any padding *before* the real, as well as its' full size. Unless the compiler does really weird stuff like, for example, sticking in padding after a real that isn't part of its' ".sizeof" on non Windows platforms, I don't think it should be a problem.

That's exactly what I was worried about -- reals are followed by 0 bytes of padding on Windows, 2 on Linux32, and 6 on linux64. But it seems it is already included in sizeof; real.sizeof varies between platforms, even though the hardware is identical. real.alignof is constant for all of them.
 That said, if you DO find anything amiss, I'd love to know :)
 
 That reminds me.  Walter: I know you're probably busy, but if you could 
 just give this the once-over to verify that it's counting the bytes 
 properly, that would be awesome.
 
     -- Daniel

Jan 15 2007