www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to dynamically alias a struct to a byte array?

reply Charles Hixson <charleshixsn earthlink.net> writes:
The situation is I have a C library routine that returns a pointer to a 
malloc_ed byte array.  The handler routine that I've written immediately 
copies the unmanaged byte array to a byte[], and then calls 
std.stdlib.free(barray).  Then it returns the byte array.  So I've 
already copied it once.

Now other code looks at the byte array, and decides what kind of struct 
it represents.  When it figures this out, is there any way I can declare 
the array to be of the desired struct type without copying it over 
again?  I don't want to cast it each time I use it, since this is data 
read in from a file.  Better would be to copy it over and over.

The only thing I've though of so far is to declare a class that holds a 
bunch of abstract methods, and then have multiple descendants that each 
have a special variant of the struct.  And even then I've either got to 
copy it again, or to cast it each time I use it.  (Though at least with 
this approach all the casts are located in one area of the code.)
Feb 17 2013
parent reply "jerro" <a a.com> writes:
 The only thing I've though of so far is to declare a class that 
 holds a bunch of abstract methods, and then have multiple 
 descendants that each have a special variant of the struct.  
 And even then I've either got to copy it again, or to cast it 
 each time I use it.  (Though at least with this approach all 
 the casts are located in one area of the code.)


You don't need to cast it each time, you can do something like:

auto structPointer = cast(MyStruct*) byteArray.ptr;

and use structPointer after that (in D structPointer.foo will get 
the member foo, same as (*structPointer).foo). So if you go with 
the common parent approach , you can just store that pointer as a 
field. This will require one extra pointer dereference each time 
you access the struct, though. If you need to access it a lot it 
may be more efficient to make a field of type MyStrtuct and just 
copy the data to it. The time it takes to copy the struct 
shouldn't really be significant anyway, since you are reading it 
from a file and file IO is typically much slower than copying 
memory.

By the way, if you use the common parent approach, the common 
parent should probably be an interface 
(http://dlang.org/interface.html), not a class.
Feb 17 2013
next sibling parent Charles Hixson <charleshixsn earthlink.net> writes:
On 02/17/2013 12:12 PM, jerro wrote:

 The only thing I've though of so far is to declare a class that holds
 a bunch of abstract methods, and then have multiple descendants that
 each have a special variant of the struct. And even then I've either
 got to copy it again, or to cast it each time I use it. (Though at
 least with this approach all the casts are located in one area of the
 code.)


 You don't need to cast it each time, you can do something like:

 auto structPointer = cast(MyStruct*) byteArray.ptr;

 and use structPointer after that (in D structPointer.foo will get the
 member foo, same as (*structPointer).foo). So if you go with the common
 parent approach , you can just store that pointer as a field. This will
 require one extra pointer dereference each time you access the struct,
 though. If you need to access it a lot it may be more efficient to make
 a field of type MyStrtuct and just copy the data to it. The time it
 takes to copy the struct shouldn't really be significant anyway, since
 you are reading it from a file and file IO is typically much slower than
 copying memory.

 By the way, if you use the common parent approach, the common parent
 should probably be an interface (http://dlang.org/interface.html), not a
 class.


I don't think the common parent *can* be an interface, as it needs to 
hold the byte array.  But since the forms of struct that are used by the 
byte array are limited, and so are the access methods, I think that a 
common "abstract" class would work. (Not really abstract, just with 
methods defined unusably.)

..... rambling stuff beyond this point: musing about design issues .....

OTOH, every class access also involves a dereference, so it may well be 
better to just use that "auto structPointer = cast(MyStruct*) 
byteArray.ptr;" that you mentioned.  I'll need to think about that.  I 
would mean that I wouldn't need to have carefully unimplemented methods 
for structs that didn't allow them.  But it may really be a logically 
better answer, as to get the other to work I'd need to either copy the 
byte array, or to create an instance of the root class (so I couldn't 
really be abstract, but would just need to have methods implemented as
bool example(params) { assert(false, "This method shouldn't be used"); }

Still, as you mentioned copying the data isn't usually too high a price 
when it's done near alongside an IO request.

Than again, I always worry about mixing pointers and garbage collection. 
  I'll need to read that section of the documentation again.  Two or 
three times.  Perhaps five.  (Pointers are the MAIN reason I was happy 
to abandon C for Python/Ruby/Java/Eiffel/Smalltalk/... a decade or so 
ago.  I'm really an old FORTRAN IV programmer, who was happy to move to 
PL/I because it made EBCDIC character handling easier.  These days read 
unicode instead of EBCDIC.  But pointers still make me uneasy.  The feel 
too much like dropping into assembler.)

So what I'll probably end up doing is defining lots of separate structs 
that each take a byte array for a constructor argument.  If I'm going to 
use them often enough, that might even be faster, and it will almost 
certainly be more intelligible a few months after I write it.  And I may 
end up nesting them within a class anyway so that I can call them 
without needing to know exactly which kind I'm calling.  Well, actually 
if I'm going to build a class, then I won't need that kind of 
complexity, because a lot of the "different kinds of struct" is just so 
that the data can easily be read in from and written out to a file. 
Which means static array sizes.  So I was thinking of having a few 
custom sizes of struct based around different amounts of data in various 
parts, but if I'm going to be using a class as the access, then I can 
just use dynamic arrays, and build (or parse) the byte array each time I 
do IO.  Which means multiple dereferences, but which is enough simpler 
than the other approach that it may pay for itself.
Feb 17 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/17/2013 12:12 PM, jerro wrote:


 You don't need to cast it each time, you can do something like:

 auto structPointer = cast(MyStruct*) byteArray.ptr;


And that pointer can be converted to a D slice:

   MyStruct[] slice = structPointer[0..number_of_struct_objects];

Use 'slice' as a regular D slice from that point on.

Ali
Feb 17 2013
parent Charles Hixson <charleshixsn earthlink.net> writes:
On 02/17/2013 09:32 PM, Ali Çehreli wrote:

 On 02/17/2013 12:12 PM, jerro wrote:

  > You don't need to cast it each time, you can do something like:
  >
  > auto structPointer = cast(MyStruct*) byteArray.ptr;

 And that pointer can be converted to a D slice:

 MyStruct[] slice = structPointer[0..number_of_struct_objects];

 Use 'slice' as a regular D slice from that point on.

 Ali


Unfortunately, it's a struct because it isn't an array.  It's several 
arrays, not all of the same type.  (one int array, one uint array, one 
long array, a few counters and flags, and occasionally also a char array 
[actually a utf-8 string], but since it's coming in from C it doesn't 
start out as immutable, and it contains an extra terminal \0.)  So a 
slice wouldn't work.

The structPointer would work, but since I really dislike working with 
pointers, I'll probably convert it into a class with dynamic arrays 
almost immediately.  And on the way out, convert it back into a byte 
array at the very last moment.  This will be a bit less efficient, but 
as a trade-off, I won't get confused about what I'm doing (or not as 
much so).
(OTOH, do note that the structPointer approach would require that I 
define lots of different structs, whereas if I convert it into a class, 
I can get by with one definition, and a more complicated construct/emit 
pair.  So it's probably a reasonable tradeoff...or perhaps even a bit 
favorable towards the class, even though every reference to a dynamic 
array involves an extra dereference.)
Feb 19 2013