www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - convert ubyte[k..k + 1] to int

reply "ref2401" <refactor24 gmail.com> writes:
i have an array of ubytes. how can i convert two adjacent ubytes 
from the array to an integer?

pseudocode example:
ubyte[5] array = createArray();
int value = array[2..3];

is there any 'memcpy' method or something else to do this?
May 16 2012
next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:

 i have an array of ubytes. how can i convert two adjacent ubytes from  
 the array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g. import std.stdio; void main() { ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ]; int value = *cast(short*)array[2..3].ptr; writefln("Result = %s", value); } The line: int value = *cast(short*)array[2..3].ptr; 1. slices 2 bytes from the array. 2. obtains the ptr to them 3. casts the ptr to short* 4. copies the value pointed at by the short* ptr to an int You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/Endianness The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine. R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
May 16 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 17:03:44 Regan Heath wrote:
 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:
 i have an array of ubytes. how can i convert two adjacent ubytes from
 the array to an integer?
 
 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];
 
 is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g. import std.stdio; void main() { ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ]; int value = *cast(short*)array[2..3].ptr; writefln("Result = %s", value); } The line: int value = *cast(short*)array[2..3].ptr; 1. slices 2 bytes from the array. 2. obtains the ptr to them 3. casts the ptr to short* 4. copies the value pointed at by the short* ptr to an int You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/Endianness The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine.

As long as you're going from big endian to little endian, std.bitmanip.bigEndianToNative will do the conversion fairly easily, but if they're in little endian, then the nasty casting is the way to go. - Jonathan M Davis
May 16 2012
parent "Robert DaSilva" <spunit262 yahoo.com> writes:
On Wednesday, 16 May 2012 at 18:47:55 UTC, Jonathan M Davis wrote:
 As long as you're going from big endian to little endian,
 std.bitmanip.bigEndianToNative will do the conversion fairly 
 easily, but if
 they're in little endian, then the nasty casting is the way to 
 go.

 - Jonathan M Davis

Except they don't take slices. You need a helper function. ubyte[2] _2(ubyte[] a){ubyte[2] b; assert(a.length==2); b[]=a; return b;}
May 16 2012
prev sibling next sibling parent Andrew Wiley <wiley.andrew.j gmail.com> writes:
On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan netmail.co.nz> wrote:

 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:

  i have an array of ubytes. how can i convert two adjacent ubytes from the
 array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g. import std.stdio; void main() { ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ]; int value = *cast(short*)array[2..3].ptr; writefln("Result = %s", value); } The line: int value = *cast(short*)array[2..3].ptr; 1. slices 2 bytes from the array. 2. obtains the ptr to them 3. casts the ptr to short* 4. copies the value pointed at by the short* ptr to an int You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/**Endianness<http://en.wikipedia.org/wiki/Endianness> The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine. R

Unfortunately, this is undefined behavior because you're breaking alignment rules. On x86, this will just cause a slow load from memory. On ARM, this will either crash your program with a bus error on newer hardware or give you a gibberish value on ARMv6 and older. Declaring a short, getting a pointer to it, and casting that pointer to a ubyte* to copy into it is fine, but casting a ubyte* to a short* will cause a 2-byte load from a 1-byte aligned address, which leads down the yellow brick road to pain.
May 16 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, May 16, 2012 at 10:25:51PM -0500, Andrew Wiley wrote:
 On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan netmail.co.nz> wrote:
 
 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com> wrote:

  i have an array of ubytes. how can i convert two adjacent ubytes from the
 array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to an int. e.g. import std.stdio; void main() { ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ]; int value = *cast(short*)array[2..3].ptr; writefln("Result = %s", value); } The line: int value = *cast(short*)array[2..3].ptr; 1. slices 2 bytes from the array. 2. obtains the ptr to them 3. casts the ptr to short* 4. copies the value pointed at by the short* ptr to an int You may need to worry about little/big endian issues, see: http://en.wikipedia.org/wiki/**Endianness<http://en.wikipedia.org/wiki/Endianness> The above code outputs "Result = 1" on my little-endian x86 desktop machine but would output "Result = 256" on a big-endian machine. R

Unfortunately, this is undefined behavior because you're breaking alignment rules. On x86, this will just cause a slow load from memory. On ARM, this will either crash your program with a bus error on newer hardware or give you a gibberish value on ARMv6 and older. Declaring a short, getting a pointer to it, and casting that pointer to a ubyte* to copy into it is fine, but casting a ubyte* to a short* will cause a 2-byte load from a 1-byte aligned address, which leads down the yellow brick road to pain.

Do unions suffer from this problem? Could this prevent alignment problems: short bytesToShort(ubyte[] b) in { assert(b.length==2); } body { union U { short val; ubyte[2] b; } U u; u.b[] = b[]; return u.val; } ? T -- Береги платье снову, а здоровье смолоду.
May 16 2012
prev sibling parent reply Andrew Wiley <wiley.andrew.j gmail.com> writes:
On Wed, May 16, 2012 at 11:07 PM, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:

 On Wed, May 16, 2012 at 10:25:51PM -0500, Andrew Wiley wrote:
 On Wed, May 16, 2012 at 11:03 AM, Regan Heath <regan netmail.co.nz>

wrote:
 On Wed, 16 May 2012 15:24:33 +0100, ref2401 <refactor24 gmail.com>


wrote:
  i have an array of ubytes. how can i convert two adjacent ubytes from


the
 array to an integer?

 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];

 is there any 'memcpy' method or something else to do this?

You don't need to "copy" the data, just tell the compiler to "pretend" it's a short (in this case, for 2 bytes) then copy the value/assign to


an
 int. e.g.

 import std.stdio;

 void main()
 {
        ubyte[5] array = [ 0xFF, 0xFF, 0x01, 0x00, 0xFF ];
        int value = *cast(short*)array[2..3].ptr;
        writefln("Result = %s", value);
 }

 The line:
  int value = *cast(short*)array[2..3].ptr;

 1. slices 2 bytes from the array.
 2. obtains the ptr to them
 3. casts the ptr to short*
 4. copies the value pointed at by the short* ptr to an int

 You may need to worry about little/big endian issues, see:
 http://en.wikipedia.org/wiki/**Endianness<


http://en.wikipedia.org/wiki/Endianness>
 The above code outputs "Result = 1" on my little-endian x86 desktop
 machine but would output "Result = 256" on a big-endian machine.

 R

Unfortunately, this is undefined behavior because you're breaking alignment rules. On x86, this will just cause a slow load from memory. On ARM, this will either crash your program with a bus error on newer hardware or give you a gibberish value on ARMv6 and older. Declaring a short, getting a pointer to it, and casting that pointer to a ubyte* to copy into it is fine, but casting a ubyte* to a short* will cause a 2-byte load from a 1-byte aligned address, which leads down the yellow brick road to pain.

Do unions suffer from this problem? Could this prevent alignment problems: short bytesToShort(ubyte[] b) in { assert(b.length==2); } body { union U { short val; ubyte[2] b; } U u; u.b[] = b[]; return u.val; } ?

As I understand it, this should be fine because the compiler will guarantee that the union is aligned to the maximum alignment required by one of its members, which is the short. This is probably the safest solution.
May 16 2012
parent reply "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 17 May 2012 at 04:16:10 UTC, Andrew Wiley wrote:
 On Wed, May 16, 2012 at 11:07 PM, H. S. Teoh 
 <hsteoh quickfur.ath.cx> wrote:
 Do unions suffer from this problem? Could this prevent 
 alignment
 problems:

        short bytesToShort(ubyte[] b)
        in { assert(b.length==2); }
        body {
                union U {
                        short val;
                        ubyte[2] b;
                }
                U u;

                u.b[] = b[];
                return u.val;
        }

 ?

As I understand it, this should be fine because the compiler will guarantee that the union is aligned to the maximum alignment required by one of its members, which is the short. This is probably the safest solution.

And what about the following code: // This implementation is optimized for speed via swapping endianness in-place pure immutable(C)[] fixEndian(C, Endian blobEndian = endian)(ubyte[] blob) if(is(CharTypeOf!C)) { import std.bitmanip, std.system; auto data = cast(C[]) blob; static if(blobEndian != endian) { static assert(!is(typeof(C) == char)); // UTF-8 doesn't have endianness foreach(ref ch; data) ch = swapEndian(ch); } return cast(immutable) data; }
May 17 2012
parent reply "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 17 May 2012 at 07:07:58 UTC, Roman D. Boiko wrote:
 And what about the following code:

 // This implementation is optimized for speed via swapping
 endianness in-place
 pure immutable(C)[] fixEndian(C, Endian blobEndian =
 endian)(ubyte[] blob) if(is(CharTypeOf!C))
 {
      import std.bitmanip, std.system;
      auto data = cast(C[]) blob;
      static if(blobEndian != endian)
      {
          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
 have endianness
          foreach(ref ch; data) ch = swapEndian(ch);
      }
      return cast(immutable) data;
 }

I mean, is it safe (assuming that we are allowed to mutate blob, and its length is a multiple of C.sizeof)? I do casting from ubyte[] to C[].
May 17 2012
parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 05/17/12 10:15, Roman D. Boiko wrote:
 On Thursday, 17 May 2012 at 07:07:58 UTC, Roman D. Boiko wrote:
 And what about the following code:

 // This implementation is optimized for speed via swapping
 endianness in-place
 pure immutable(C)[] fixEndian(C, Endian blobEndian =
 endian)(ubyte[] blob) if(is(CharTypeOf!C))
 {
      import std.bitmanip, std.system;
      auto data = cast(C[]) blob;
      static if(blobEndian != endian)
      {
          static assert(!is(typeof(C) == char)); // UTF-8 doesn't
 have endianness
          foreach(ref ch; data) ch = swapEndian(ch);
      }
      return cast(immutable) data;
 }

I mean, is it safe (assuming that we are allowed to mutate blob, and its length is a multiple of C.sizeof)? I do casting from ubyte[] to C[].

Only if C.ptr ends up properly aligned. There are also aliasing issues, which i don't think are sufficiently defined for D (for C, it would be legal only because char* is allowed to alias anything). artur
May 17 2012
parent "Roman D. Boiko" <rb d-coding.com> writes:
On Thursday, 17 May 2012 at 08:39:21 UTC, Artur Skawina wrote:
 On 05/17/12 10:15, Roman D. Boiko wrote:
 I mean, is it safe (assuming that we are allowed to mutate 
 blob, and its length is a multiple of C.sizeof)?
 
 I do casting from ubyte[] to C[].

Only if C.ptr ends up properly aligned. There are also aliasing issues, which i don't think are sufficiently defined for D (for C, it would be legal only because char* is allowed to alias anything). artur

Is it possible to ensure? In my case blob is created as auto blob = cast(ubyte[]) read(fileName); I assume that alignment is safe. But what should I do to be safe in a general case?
May 17 2012
prev sibling parent Luis Panadero =?UTF-8?B?R3VhcmRlw7Fv?= <luis.panadero gmail.com> writes:
ref2401 wrote:

 i have an array of ubytes. how can i convert two adjacent ubytes
 from the array to an integer?
 
 pseudocode example:
 ubyte[5] array = createArray();
 int value = array[2..3];
 
 is there any 'memcpy' method or something else to do this?

Try to use littleEndianToNative or bigEndianToNative, but you should check the endianes of the data that you desire convert to a int. For example: ubyte[5] array = createArray(); int value = littleEndianToNative!int(array); // or: // int value = bigEndianToNative!int(array); -- I'm afraid that I have a blog: http://zardoz.es
May 21 2012