digitalmars.D.learn - Size of a class instance at compile time
- Daniel Keep (13/29) Jan 13 2007 Ok, ASSUMING:
- Frits van Bommel (48/78) Jan 13 2007 (2) is a bit redundant after (1)...
- Sean Kelly (5/7) Jan 13 2007 I think this may be a problem. The ABI allows class data to be
- Frits van Bommel (15/22) Jan 13 2007 As I noted earlier in that post, the ABI indeed doesn't specify the
- Frits van Bommel (16/18) Jan 13 2007 Yep, definitely breaks there. But I did find a much more elegant
- Daniel Keep (4/28) Jan 13 2007 Oh that's beautiful; I can't believe I forgot about offsetof! Thanks
- Daniel Keep (34/57) Jan 14 2007 Just wondering; do we even need to worry about align(N) attributes?
- Frits van Bommel (37/68) Jan 14 2007 They work for classes too. Not sure if it's documented, but in my test
- Frits van Bommel (11/16) Jan 14 2007 So I was wondering, could is(T supertuple == super) be of help here?
- Daniel Keep (12/31) Jan 14 2007 Oh... BUGGER. Well, that's inconvenient. I guess I'm just going to
- Daniel Keep (110/116) Jan 14 2007 Got it! No, really, this time FOR SURE ^_^
- Don Clugston (5/145) Jan 14 2007 One other nasty case is where the last member of a class is an 80-bit
- Daniel Keep (16/24) Jan 15 2007 Well, I tried sticking a real into the test cases in various spots, and
- Don Clugston (6/34) Jan 15 2007 That's exactly what I was worried about -- reals are followed by 0 bytes...
Ok, ASSUMING: 1. I'm only going to be compiling with dmd... 2. ...for x86... 3. ...under Windows, is this an accurate way of getting the size of a class instance at compile time?template InstanceSize(T) { const uint InstanceSize = 12 + CountSizes!(typeof(T.tupleof)); } template CountSizes() { const uint CountSizes = 0; } template CountSizes(T) { const uint CountSizes = T.sizeof; } template CountSizes(T, Tail...) { const uint CountSizes = T.sizeof + CountSizes!(Tail); }It seems to report the right sizes, but I'm not entirely sure what the '12' is for. Pointer to ClassInfo, ... what else? For those wondering: I'm doing this for some custom mixin memory allocators I'm playing with. -- Daniel "isizeof, isizeof, my kingdom for isizeof!" -- "Rick 3" by Bill Wobblepike.
Jan 13 2007
Daniel Keep wrote:Ok, ASSUMING: 1. I'm only going to be compiling with dmd... 2. ...for x86... 3. ...under Windows,(2) is a bit redundant after (1)... Also, AFAIK (3) doesn't matter in this respect.is this an accurate way of getting the size of a class instance at compile time? > template InstanceSize(T) > { > const uint InstanceSize = 12 + CountSizes!(typeof(T.tupleof)); > } > > template CountSizes() > { > const uint CountSizes = 0; > } > > template CountSizes(T) > { > const uint CountSizes = T.sizeof; > } > > template CountSizes(T, Tail...) > { > const uint CountSizes = T.sizeof + CountSizes!(Tail); > }Sorry, it's not.It seems to report the right sizes, but I'm not entirely sure what the '12' is for. Pointer to ClassInfo, ... what else?There are only 8 bytes of overhead: pointer to vtable and pointer to monitor. (the vtable contains the classinfo pointer) See http://www.digitalmars.com/d/abi.html, under "Classes". It doesn't specifically mention how non-static members are formatted, but I think I figured it out (see below). You also forgot to take alignment gaps into account. Try this: ----- template InstanceSize(T) { const InstanceSize = CountSizes!(8, typeof(T.tupleof)); } template CountSizes(size_t N) { const CountSizes = N; } template CountSizes(size_t N, T) { const uint CountSizes = AlignUp!(T.alignof, N) + T.sizeof; } template CountSizes(size_t N, T, Tail...) { const uint CountSizes = CountSizes!(AlignUp!(T.alignof, N) + T.sizeof, Tail); } template AlignUp(size_t Align, size_t N) { static assert((Align & (Align - 1)) == 0, "Alignment not power of two"); const AlignUp = (N + Align - 1) & ~(Align - 1); } // // Test code // class Test { /// Change members to test different layouts char c; int i; byte b; } import std.stdio; void main() { writefln("Static size: ", InstanceSize!(Test)); writefln("Actual size: ", Test.classinfo.init.length); } -----
Jan 13 2007
Frits van Bommel wrote:You also forgot to take alignment gaps into account.I think this may be a problem. The ABI allows class data to be rearranged, so alignment gaps can't be predicted (I think) from the output of tupleof. I still wish we could get .isizeof for this :-p Sean
Jan 13 2007
Sean Kelly wrote:Frits van Bommel wrote:As I noted earlier in that post, the ABI indeed doesn't specify the format of the non-static member part of an object. It seems to be one of the incomplete parts. But from my experiments it seems the current compiler (v1.0) doesn't reorder fields. I've tested my code with several layouts and AFAICT it consistently provides the same answer as a runtime classinfo.init.length. So assuming no edge cases[1] are triggered it should work until the next ABI break at the very least... Since ABI breaks seem to be indicated by red 'Need to recompile because:' clauses in the changelog, it should be pretty clear when this needs to be changed. [1]: e.g. I haven't tested using anything with explicit align(N) declarations, not sure what happens there (would .alignof be adjusted?).You also forgot to take alignment gaps into account.I think this may be a problem. The ABI allows class data to be rearranged, so alignment gaps can't be predicted (I think) from the output of tupleof. I still wish we could get .isizeof for this :-p
Jan 13 2007
Frits van Bommel wrote:[1]: e.g. I haven't tested using anything with explicit align(N) declarations, not sure what happens there (would .alignof be adjusted?).Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------ _Way_ shorter, and always provides the correct answer under two simple assumptions: 1) The last non-static member is also last in the layout 2) There's no padding at the end. Assumption (1) is pretty likely to be broken if and when member reordering is implemented, though.
Jan 13 2007
Frits van Bommel wrote:Frits van Bommel wrote:Oh that's beautiful; I can't believe I forgot about offsetof! Thanks very much for that: this should make the allocators a bit more efficient :) -- Daniel[1]: e.g. I haven't tested using anything with explicit align(N) declarations, not sure what happens there (would .alignof be adjusted?).Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------ _Way_ shorter, and always provides the correct answer under two simple assumptions: 1) The last non-static member is also last in the layout 2) There's no padding at the end. Assumption (1) is pretty likely to be broken if and when member reordering is implemented, though.
Jan 13 2007
Frits van Bommel wrote:Frits van Bommel wrote:Just wondering; do we even need to worry about align(N) attributes? AFAIK, that only applies to structures, and we can get the size of those using sizeof since they're value types. http://digitalmars.com/d/attribute.html#align -- says down the bottom of that heading that it only applies to structs and struct members.[1]: e.g. I haven't tested using anything with explicit align(N) declarations, not sure what happens there (would .alignof be adjusted?).Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------_Way_ shorter, and always provides the correct answer under two simple assumptions: 1) The last non-static member is also last in the layout 2) There's no padding at the end. Assumption (1) is pretty likely to be broken if and when member reordering is implemented, though.I just knocked up a slightly more robust version that should *theoretically* still work even if the members are moved around. It basically just does the same thing yours does, but it runs over the whole tuple, and passes out the largest value it finds. Obviously, it doesn't help for the above two points, but Walter seems to like stuff at the start of the block, not trailing off the end. Plus, when would a class require padding? -- Daniel template InstanceSize(T) { const InstanceSize = InstanceSizeImpl!(T, 0); } template InstanceSizeImpl(T, size_t i) { static if( i < T.tupleof.length ) const InstanceSizeImpl = Max!( T.tupleof[i].offsetof + T.tupleof[i].sizeof, InstanceSizeImpl!(T, i+1)); else const InstanceSizeImpl = 0u; } template Max(size_t a, size_t b) { static if( a > b ) const Max = a; else const Max = b; }
Jan 14 2007
Daniel Keep wrote:Frits van Bommel wrote:They work for classes too. Not sure if it's documented, but in my test file I have the following: ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } class Test { char c; int i; } class Test2 { char c; align(1) int i; } import std.stdio; void main() { writefln("Test:"); writefln("Static size: ", InstanceSize!(Test)); writefln("Actual size: ", Test.classinfo.init.length); writefln(); writefln("Test2:"); writefln("Static size: ", InstanceSize!(Test2)); writefln("Actual size: ", Test2.classinfo.init.length); } ----- Which gives size 16 and 13 for Test and Test2, respectively. So alignment declarations definitely makes a difference even for classes.Frits van Bommel wrote:Just wondering; do we even need to worry about align(N) attributes? AFAIK, that only applies to structures, and we can get the size of those using sizeof since they're value types.[1]: e.g. I haven't tested using anything with explicit align(N) declarations, not sure what happens there (would .alignof be adjusted?).Yep, definitely breaks there. But I did find a much more elegant implementation that *does* work in that case (as well as in normal cases): ----- template InstanceSize(T) { const InstanceSize = T.tupleof[$-1].offsetof + T.tupleof[$-1].sizeof; } ------http://digitalmars.com/d/attribute.html#align -- says down the bottom of that heading that it only applies to structs and struct members.Then I guess either the spec or the implementation is wrong.I just knocked up a slightly more robust version that should *theoretically* still work even if the members are moved around. It basically just does the same thing yours does, but it runs over the whole tuple, and passes out the largest value it finds.Ah, that's a good solution to that problem I guess. Unless there's ever any weird need to put overhead bytes at the end that should work. (Where are interface vtable pointers put? [tests] Damn, looks like they're at the end :( )Obviously, it doesn't help for the above two points, but Walter seems to like stuff at the start of the block, not trailing off the end. Plus, when would a class require padding?Apparently, implementing an interface counts as well...
Jan 14 2007
Frits van Bommel wrote:Ah, that's a good solution to that problem I guess. Unless there's ever any weird need to put overhead bytes at the end that should work. (Where are interface vtable pointers put? [tests] Damn, looks like they're at the end :( )So I was wondering, could is(T supertuple == super) be of help here? Specifically, how many interface vptrs are there in an object? One per interface? Or are inherited interfaces folded like classes, one vptr per most-derived interface? I think this could be made to work if we figure out the details. It looks to a bit complicated though, since I'm guessing interfaces of base classes come before members of derived classes. That plus double.alignof > (void*).alignof on x86 means that to determine alignment gaps correctly we'd probably need to work from Object up... I don't feel like coding that right now :(.
Jan 14 2007
Frits van Bommel wrote:Daniel Keep wrote:Oh... BUGGER. Well, that's inconvenient. I guess I'm just going to have to keep trying until I either find a way to do this properly, or Walter adds isizeof :P Interface vtable pointers; they're just regular D arrays, right? One for each interface? Here it is: expression.html#IsExpression -- we can use is(T I == super) to get a tuple of the base classes and all the interfaces. If we can accurately predict how large the extra padding is from the number of interfaces, it's sorted. Time to go test the theory, then :) -- DanielI just knocked up a slightly more robust version that should *theoretically* still work even if the members are moved around. It basically just does the same thing yours does, but it runs over the whole tuple, and passes out the largest value it finds.Ah, that's a good solution to that problem I guess. Unless there's ever any weird need to put overhead bytes at the end that should work. (Where are interface vtable pointers put? [tests] Damn, looks like they're at the end :( )Obviously, it doesn't help for the above two points, but Walter seems to like stuff at the start of the block, not trailing off the end. Plus, when would a class require padding?Apparently, implementing an interface counts as well...
Jan 14 2007
Got it! No, really, this time FOR SURE ^_^ The end solution is utterly evil, but is, AFAIK, correct. What I had to do in the end was to say that the size of a class instance is the maximum of: 1) size of the base class + if(implements interfaces) padding + interface count * pointer size, 2) max(class member alignof + sizeof) + if(implements interfaces) padding + interface count * pointer size I also added a minimum size of 2*pointer size to account for classes that don't have any member variables. Getting the padding for interfaces was fun:struct Align { ubyte a; void* b; } const PTR_ALIGN = Align.tupleof[1].alignof;I *think* that's right. In any case, it's reporting the correct size for every test case I throw at it. Anything else that needs fixing? :P -- Daniel private struct Align { ubyte a; void* b; } private const PTR_ALIGN = Align.tupleof[1].alignof; private template AlignPad(size_t base, size_t aligned) { static if( aligned == 0 ) const AlignPad = base; else const AlignPad = ((base+PTR_ALIGN-1)/PTR_ALIGN)*PTR_ALIGN + aligned; } template InstanceSize(T) { static if( is( T == Object ) ) const InstanceSize = 2*(void*).sizeof; else const InstanceSize = Max!( AlignPad!( InstanceSize!(Super!(T)), InterfaceCount!(T)*(void*).sizeof), AlignPad!( InstanceSizeImpl!(T, 0), + InterfaceCount!(T)*(void*).sizeof)); } private template Super(T) { static if( is( T S == super ) ) alias First!(S) Super; else static assert(false, "Can't get super of "~T.mangleof); } private template First(T) { alias T First; } private template First(T, Ts...) { alias T First; } private template InstanceSizeImpl(T, size_t i) { static if( i < T.tupleof.length ) const InstanceSizeImpl = Max!( T.tupleof[i].offsetof + T.tupleof[i].sizeof, InstanceSizeImpl!(T, i+1)); else // This is necessary to account for classes without member // variables. const InstanceSizeImpl = 2*(void*).sizeof; } private template Max(size_t a, size_t b) { static if( a > b ) const Max = a; else const Max = b; } private template InstanceSizeAligned(T, size_t alignment) { static if( alignment == 0 ) const InstanceSizeAligned = InstanceSize!(T); else const uint InstanceSizeAligned = InstanceSizeAlignImpl!(T, alignment).result; } private template InstanceSizeAlignedImpl(T, size_t alignment) { private const base_size = InstanceSize!(T); const result = ((base_size+alignment-1)/alignment)*alignment; } private template InterfaceCount(T) { static if( is( T == Object ) ) const InterfaceCount = 0u; else static if( is( T S == super ) ) const InterfaceCount = InterfaceCountImpl!(S); } private template InterfaceCountImpl(TBase, TInterfaces...) { const InterfaceCountImpl = TInterfaces.length; }
Jan 14 2007
Daniel Keep wrote:Got it! No, really, this time FOR SURE ^_^ The end solution is utterly evil, but is, AFAIK, correct. What I had to do in the end was to say that the size of a class instance is the maximum of: 1) size of the base class + if(implements interfaces) padding + interface count * pointer size, 2) max(class member alignof + sizeof) + if(implements interfaces) padding + interface count * pointer size I also added a minimum size of 2*pointer size to account for classes that don't have any member variables. Getting the padding for interfaces was fun: > struct Align > { > ubyte a; > void* b; > } > > const PTR_ALIGN = Align.tupleof[1].alignof;Nifty!I *think* that's right. In any case, it's reporting the correct size for every test case I throw at it. Anything else that needs fixing? :POne other nasty case is where the last member of a class is an 80-bit real, with padding that is different between Windows, Linux32, and Linux64. Does that work correctly?-- Daniel private struct Align { ubyte a; void* b; } private const PTR_ALIGN = Align.tupleof[1].alignof; private template AlignPad(size_t base, size_t aligned) { static if( aligned == 0 ) const AlignPad = base; else const AlignPad = ((base+PTR_ALIGN-1)/PTR_ALIGN)*PTR_ALIGN + aligned; } template InstanceSize(T) { static if( is( T == Object ) ) const InstanceSize = 2*(void*).sizeof; else const InstanceSize = Max!( AlignPad!( InstanceSize!(Super!(T)), InterfaceCount!(T)*(void*).sizeof), AlignPad!( InstanceSizeImpl!(T, 0), + InterfaceCount!(T)*(void*).sizeof)); } private template Super(T) { static if( is( T S == super ) ) alias First!(S) Super; else static assert(false, "Can't get super of "~T.mangleof); } private template First(T) { alias T First; } private template First(T, Ts...) { alias T First; } private template InstanceSizeImpl(T, size_t i) { static if( i < T.tupleof.length ) const InstanceSizeImpl = Max!( T.tupleof[i].offsetof + T.tupleof[i].sizeof, InstanceSizeImpl!(T, i+1)); else // This is necessary to account for classes without member // variables. const InstanceSizeImpl = 2*(void*).sizeof; } private template Max(size_t a, size_t b) { static if( a > b ) const Max = a; else const Max = b; } private template InstanceSizeAligned(T, size_t alignment) { static if( alignment == 0 ) const InstanceSizeAligned = InstanceSize!(T); else const uint InstanceSizeAligned = InstanceSizeAlignImpl!(T, alignment).result; } private template InstanceSizeAlignedImpl(T, size_t alignment) { private const base_size = InstanceSize!(T); const result = ((base_size+alignment-1)/alignment)*alignment; } private template InterfaceCount(T) { static if( is( T == Object ) ) const InterfaceCount = 0u; else static if( is( T S == super ) ) const InterfaceCount = InterfaceCountImpl!(S); } private template InterfaceCountImpl(TBase, TInterfaces...) { const InterfaceCountImpl = TInterfaces.length; }
Jan 14 2007
Don Clugston wrote:Daniel Keep wrote:Well, I tried sticking a real into the test cases in various spots, and it seemed fine. That said, I'm not sure what the potential problem is; it currently determines the size of the member variables by finding the maximum of (member.alignof+member.sizeof) for every member variable. This means that, even if the last member is a real, that it will correctly account for any padding *before* the real, as well as its' full size. Unless the compiler does really weird stuff like, for example, sticking in padding after a real that isn't part of its' ".sizeof" on non Windows platforms, I don't think it should be a problem. That said, if you DO find anything amiss, I'd love to know :) That reminds me. Walter: I know you're probably busy, but if you could just give this the once-over to verify that it's counting the bytes properly, that would be awesome. -- DanielI *think* that's right. In any case, it's reporting the correct size for every test case I throw at it. Anything else that needs fixing? :POne other nasty case is where the last member of a class is an 80-bit real, with padding that is different between Windows, Linux32, and Linux64. Does that work correctly?
Jan 15 2007
Daniel Keep wrote:Don Clugston wrote:That's exactly what I was worried about -- reals are followed by 0 bytes of padding on Windows, 2 on Linux32, and 6 on linux64. But it seems it is already included in sizeof; real.sizeof varies between platforms, even though the hardware is identical. real.alignof is constant for all of them.Daniel Keep wrote:Well, I tried sticking a real into the test cases in various spots, and it seemed fine. That said, I'm not sure what the potential problem is; it currently determines the size of the member variables by finding the maximum of (member.alignof+member.sizeof) for every member variable. This means that, even if the last member is a real, that it will correctly account for any padding *before* the real, as well as its' full size. Unless the compiler does really weird stuff like, for example, sticking in padding after a real that isn't part of its' ".sizeof" on non Windows platforms, I don't think it should be a problem.I *think* that's right. In any case, it's reporting the correct size for every test case I throw at it. Anything else that needs fixing? :POne other nasty case is where the last member of a class is an 80-bit real, with padding that is different between Windows, Linux32, and Linux64. Does that work correctly?That said, if you DO find anything amiss, I'd love to know :) That reminds me. Walter: I know you're probably busy, but if you could just give this the once-over to verify that it's counting the bytes properly, that would be awesome. -- Daniel
Jan 15 2007