www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Any way to set len and ptr in dyn arrays?

reply nobody <nobody mailinator.com> writes:
The only way I have found is to use a struct/union kludge:

// redefine the dyn array struct
struct sArrayKludge
{
   int len;
   void* ptr;
}

// use a union to allow reaching inside
union uArrayKludge
{
   byte[] myArray;
   sArrayKludge myArrayInternals;
}

byte someFunc()
{
   // declare the union where I would declare myArray
   uArrayKludge theArray;

   int[] buf = cast(byte[]) read("file.ext");

   // use the union where I would use the array
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];

   // now I can use the ith byte
   return theArray.myArray[buf[0]+buf[1]];
}



Thanks in advance.
Aug 07 2006
next sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
nobody wrote:
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];

erm ... foo = (&buf[2])[0 .. buf[0] * buf[1]];
Aug 07 2006
parent reply nobody <nobody mailinator.com> writes:
Tom S wrote:
 nobody wrote:
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];

erm ... foo = (&buf[2])[0 .. buf[0] * buf[1]];

Thanks for your answer. I understand that would ultimately have the same result. My question is about whether that directly sets the len and ptr fields or are does more stuff happen. If so what is it? If not then why is there no direct access to the ptr and len fields?
Aug 08 2006
parent reply Frank Benoit <keinfarbton nospam.xyz> writes:
nobody schrieb:
 Tom S wrote:
 nobody wrote:
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];

erm ... foo = (&buf[2])[0 .. buf[0] * buf[1]];

Thanks for your answer. I understand that would ultimately have the same result. My question is about whether that directly sets the len and ptr fields or are does more stuff happen. If so what is it? If not then why is there no direct access to the ptr and len fields?

This builds a slice, which is like setting ptr/length of an array. This is called slicing. Instead of Toms example, I think you can also write this: foo = buf[ 2 .. buf[0] * buf[1] + 2 ]; Refering to http://www.digitalmars.com/d/arrays.html slicing: b = a[1..3]; array copy: s[] = t[]; s[1..2] = t[0..1];
Aug 08 2006
parent reply nobody <nobody mailinator.com> writes:
Frank Benoit wrote:
 nobody schrieb:
 Tom S wrote:
 nobody wrote:
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];

foo = (&buf[2])[0 .. buf[0] * buf[1]];

result. My question is about whether that directly sets the len and ptr fields or are does more stuff happen. If so what is it? If not then why is there no direct access to the ptr and len fields?

This builds a slice, which is like setting ptr/length of an array. This is called slicing. Instead of Toms example, I think you can also write this: foo = buf[ 2 .. buf[0] * buf[1] + 2 ];

Thanks to you also for your response. I was trying to suggest before that I understand how slicing ends up -- ie with ptr and len as I desired. I am worried about what happens while the slicing happens. I am worried about any gotchas.
Aug 08 2006
parent reply Frank Benoit <keinfarbton nospam.xyz> writes:
 This builds a slice, which is like setting ptr/length of an array. This
 is called slicing. Instead of Toms example, I think you can also write
 this:
 foo = buf[ 2 .. buf[0] * buf[1] + 2 ];

Thanks to you also for your response. I was trying to suggest before that I understand how slicing ends up -- ie with ptr and len as I desired. I am worried about what happens while the slicing happens. I am worried about any gotchas.

I my understanding this is exactly like setting the ptr and length. One side effect could be, that the compiler perhaps inserts array boundary check. But I don't know that.
Aug 08 2006
parent reply BCS <BCS pathlink.com> writes:
Frank Benoit wrote:
This builds a slice, which is like setting ptr/length of an array. This
is called slicing. Instead of Toms example, I think you can also write
this:
foo = buf[ 2 .. buf[0] * buf[1] + 2 ];

Thanks to you also for your response. I was trying to suggest before that I understand how slicing ends up -- ie with ptr and len as I desired. I am worried about what happens while the slicing happens. I am worried about any gotchas.

I my understanding this is exactly like setting the ptr and length. One side effect could be, that the compiler perhaps inserts array boundary check. But I don't know that.

I don't think that any extra overhead is involved in slicing. D doesn't even check if a pointer is valid. (That might be a gotcha come to think of it) void main() { char[] foo = (cast(char*)null)[0..10]; }
Aug 08 2006
parent nobody <nobody mailinator.com> writes:
BCS wrote:
 Frank Benoit wrote:
 This builds a slice, which is like setting ptr/length of an array. This
 is called slicing. Instead of Toms example, I think you can also write
 this:
 foo = buf[ 2 .. buf[0] * buf[1] + 2 ];

Thanks to you also for your response. I was trying to suggest before that I understand how slicing ends up -- ie with ptr and len as I desired. I am worried about what happens while the slicing happens. I am worried about any gotchas.

I my understanding this is exactly like setting the ptr and length. One side effect could be, that the compiler perhaps inserts array boundary check. But I don't know that.

I don't think that any extra overhead is involved in slicing. D doesn't even check if a pointer is valid. (That might be a gotcha come to think of it) void main() { char[] foo = (cast(char*)null)[0..10]; }

Great example! I am now convinced there is nothing to worry about. Thanks everyone for the help!
Aug 09 2006
prev sibling parent reply James Dunne <james.jdunne gmail.com> writes:
nobody wrote:
 The only way I have found is to use a struct/union kludge:
 
 // redefine the dyn array struct
 struct sArrayKludge
 {
   int len;
   void* ptr;
 }
 
 // use a union to allow reaching inside
 union uArrayKludge
 {
   byte[] myArray;
   sArrayKludge myArrayInternals;
 }
 
 byte someFunc()
 {
   // declare the union where I would declare myArray
   uArrayKludge theArray;
 
   int[] buf = cast(byte[]) read("file.ext");
 
   // use the union where I would use the array
   theArray.myArrayInternals.len = buf[0] * buf[1];
   theArray.myArrayInternals.ptr = &buf[2];
 
   // now I can use the ith byte
   return theArray.myArray[buf[0]+buf[1]];
 }
 
 
 
 Thanks in advance.

This is very interesting and all, but why would you do such a thing? Is the .length property on the dynamic array just not cuttin' it for you? -- -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O M-- V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e h>--->++ r+++ y+++ ------END GEEK CODE BLOCK------ James Dunne
Aug 07 2006
parent reply nobody <nobody mailinator.com> writes:
James Dunne wrote:
 This is very interesting and all, but why would you do such a thing?
 
 Is the .length property on the dynamic array just not cuttin' it for you?
 

No it is not. The whole raison d'etre for that array is to give me access to the data in buf. From my reading of the docs it looks like any case in which the length increases then this array can be copied and might not point to the data in buf anymore. So now if I tried to save changes to buf by passing it to the std.file.write method it would look like nothing happened.
Aug 08 2006
parent reply James Dunne <james.jdunne gmail.com> writes:
nobody wrote:
 James Dunne wrote:
 
 This is very interesting and all, but why would you do such a thing?

 Is the .length property on the dynamic array just not cuttin' it for you?

No it is not. The whole raison d'etre for that array is to give me access to the data in buf. From my reading of the docs it looks like any case in which the length increases then this array can be copied and might not point to the data in buf anymore. So now if I tried to save changes to buf by passing it to the std.file.write method it would look like nothing happened.

I'm still not following you. You want to access, on the byte level, the compiler-dependent implementation details of how dynamic arrays work. I get that. But why? Here: // Sample program (with v0.163): import std.file; int main(char[][] args) { char[] myFileContents = cast(char[]) std.file.read("hello.txt"); size_t origLength = myFileContents.length; myFileContents.length = origLength + 20; myFileContents[origLength .. $] = "20 characters here!!"; std.file.write("hello2.txt", cast(void[]) myFileContents); return 0; } Show me pseudo code of the larger problem you're trying to solve. I think your assumptions on the way D's dynamic arrays work are getting in the way here. -- -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O M-- V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e h>--->++ r+++ y+++ ------END GEEK CODE BLOCK------ James Dunne
Aug 08 2006
parent reply nobody <nobody mailinator.com> writes:
James Dunne wrote:
 nobody wrote:
 James Dunne wrote:

 This is very interesting and all, but why would you do such a thing?

 Is the .length property on the dynamic array just not cuttin' it for 
 you?

No it is not. The whole raison d'etre for that array is to give me access to the data in buf. From my reading of the docs it looks like any case in which the length increases then this array can be copied and might not point to the data in buf anymore. So now if I tried to save changes to buf by passing it to the std.file.write method it would look like nothing happened.

I'm still not following you. You want to access, on the byte level, the compiler-dependent implementation details of how dynamic arrays work. I get that. But why? Here: // Sample program (with v0.163): import std.file; int main(char[][] args) { char[] myFileContents = cast(char[]) std.file.read("hello.txt"); size_t origLength = myFileContents.length; myFileContents.length = origLength + 20; myFileContents[origLength .. $] = "20 characters here!!"; std.file.write("hello2.txt", cast(void[]) myFileContents); return 0; } Show me pseudo code of the larger problem you're trying to solve. I think your assumptions on the way D's dynamic arrays work are getting in the way here.

minimal example of my problem and it got way too big. Instead I left it for this morning hoping I might find a smaller way to illustrate what I meant. My original example was crap (sorry for that!)so here is an example that is close: int main(char[][] args) { char[] buf = "parse words"; char[] ex = buf[0..1]; while(ex[$-1] != ' ') ex.length = ex.length + 1; dout.writeLine(uex.ex); return 0; } Instead of writing into buf using the ex array this example instead tries to read from buf via the ex array. However as I mentioned before if you increase ex's length via .length then it no longer points into buf. As a result the above example never returns. The way forward is changing the length by some other means. This is what my original example was poorly trying to illustrate. This version in fact works just fine: struct sArrayKludge { int len; void* ptr; } union uArrayKludge { char[] ex; sArrayKludge meta; } int main(char[][] args) { char[] buf = "parse words"; uArrayKludge uex; uex.meta.ptr = &buf[0]; uex.meta.len = 1; while(uex.ex[$-1] != ' ') uex.meta.len = uex.meta.len + 1; dout.writeLine(uex.ex); return 0; } Tom S, Frank Benoit and BCS helped me to see that slicing will probably not have strange side effects so I should be using the following version which also works fine: int main(char[][] args) { char[] buf = "parse words"; char[] ex = buf[0..1]; while(ex[$-1] != ' ') ex = buf[0..ex.length+1]; dout.writeLine(ex); return 0; } So now with that said I am curious what incorrect assumptions you thought I had about dynamic arrays? I always try to pay attention when people say I might be missing something and even if what you were going to say is not applicable to me it might help someone else.
Aug 10 2006
parent James Dunne <james.jdunne gmail.com> writes:
nobody wrote:
 James Dunne wrote:
 
 nobody wrote:

 James Dunne wrote:

 This is very interesting and all, but why would you do such a thing?

 Is the .length property on the dynamic array just not cuttin' it for 
 you?

No it is not. The whole raison d'etre for that array is to give me access to the data in buf. From my reading of the docs it looks like any case in which the length increases then this array can be copied and might not point to the data in buf anymore. So now if I tried to save changes to buf by passing it to the std.file.write method it would look like nothing happened.

I'm still not following you. You want to access, on the byte level, the compiler-dependent implementation details of how dynamic arrays work. I get that. But why? Here: // Sample program (with v0.163): import std.file; int main(char[][] args) { char[] myFileContents = cast(char[]) std.file.read("hello.txt"); size_t origLength = myFileContents.length; myFileContents.length = origLength + 20; myFileContents[origLength .. $] = "20 characters here!!"; std.file.write("hello2.txt", cast(void[]) myFileContents); return 0; } Show me pseudo code of the larger problem you're trying to solve. I think your assumptions on the way D's dynamic arrays work are getting in the way here.

example of my problem and it got way too big. Instead I left it for this morning hoping I might find a smaller way to illustrate what I meant. My original example was crap (sorry for that!)so here is an example that is close: int main(char[][] args) { char[] buf = "parse words"; char[] ex = buf[0..1]; while(ex[$-1] != ' ') ex.length = ex.length + 1; dout.writeLine(uex.ex); return 0; } Instead of writing into buf using the ex array this example instead tries to read from buf via the ex array. However as I mentioned before if you increase ex's length via .length then it no longer points into buf. As a result the above example never returns. The way forward is changing the length by some other means. This is what my original example was poorly trying to illustrate. This version in fact works just fine: struct sArrayKludge { int len; void* ptr; } union uArrayKludge { char[] ex; sArrayKludge meta; } int main(char[][] args) { char[] buf = "parse words"; uArrayKludge uex; uex.meta.ptr = &buf[0]; uex.meta.len = 1; while(uex.ex[$-1] != ' ') uex.meta.len = uex.meta.len + 1; dout.writeLine(uex.ex); return 0; } Tom S, Frank Benoit and BCS helped me to see that slicing will probably not have strange side effects so I should be using the following version which also works fine: int main(char[][] args) { char[] buf = "parse words"; char[] ex = buf[0..1]; while(ex[$-1] != ' ') ex = buf[0..ex.length+1]; dout.writeLine(ex); return 0; } So now with that said I am curious what incorrect assumptions you thought I had about dynamic arrays? I always try to pay attention when people say I might be missing something and even if what you were going to say is not applicable to me it might help someone else.

Okay I think your incorrect assumption at first was that increasing the length field of a sliced array would extend that slice forward through the original array. Ideally, you should never depend on compiler-dependent details of implementation unless that behavior is rigorously defined in the ABI. With D's basic set of primitive array operations and slicing, you can do just about anything you need to. -- -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O M-- V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e h>--->++ r+++ y+++ ------END GEEK CODE BLOCK------ James Dunne
Aug 10 2006