## digitalmars.D.learn - ubytes to ulong problem

• Charles Hixson (36/36) Dec 21 2013 I was planning to ask if there were a better way to do this, but instead...
• ponce (2/37) Dec 21 2013
• ponce (2/40) Dec 21 2013 Use the exponentiation operator which is spelled: ^^
• Charles Hixson (5/43) Dec 21 2013 Thanks. I was *sure* it was something stupid on my part. It's 'good'
• John Colvin (38/73) Dec 21 2013 As pointed out before, you're using '^' which is xor, instead of
• =?UTF-8?B?QWxpIMOHZWhyZWxp?= (22/24) Dec 21 2013 Assuming that the program needs to support only big endian and little
• John Colvin (3/30) Dec 21 2013 Nevermind equally fast, that will be much faster. -10 brain
• Charles Hixson (6/31) Dec 21 2013 Will that work even when the alignment is to odd bytes? Because that's
• =?UTF-8?B?QWxpIMOHZWhyZWxp?= (21/59) Dec 21 2013 No, it is not guaranteed to work unless the alignment is right.
• Charles Hixson (24/85) Dec 22 2013 Nice, but the block is longer than 8 bytes, so I should use a "for (i =
• John Colvin (4/51) Dec 22 2013 It's just an unaligned load. If your target cpu architecture
```I was planning to ask if there were a better way to do this, but instead
I need to ask what's my mistake?
For some reason, if called with an uninitialized ubyte array, and an
index of 0, it returns a value of 8, even though all the values in the
array are 0.
The error has to be somewhere in the "ret = " statement, but I sure
don't see it.

/**    Convert 8 consecutive bytes sliced from a ubyte[] into a ulong
*  param    block    The array from which to slice.
*  param    n    The starting index within the block    */
ulong    ubytesToUlong(ubyte[] block, int n)
{    ulong    ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret    =    cast(ulong)block[n] * 2^21
+    cast(ulong)block[n+1] * 2^18
+    cast(ulong)block[n+2] * 2^15
+    cast(ulong)block[n+3] * 2^12
+    cast(ulong)block[n+4] * 2^9
+    cast(ulong)block[n+5] * 2^6
+    cast(ulong)block[n+6] * 2^3
+    cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return    ret;
}

--
Charles Hixson
```
Dec 21 2013
"ponce" <contact g3mesfrommars.fr> writes:
```On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all the
values in the array are 0.
The error has to be somewhere in the "ret = " statement, but I
sure don't see it.

/**    Convert 8 consecutive bytes sliced from a ubyte[] into a
ulong
*  param    block    The array from which to slice.
*  param    n    The starting index within the block    */
ulong    ubytesToUlong(ubyte[] block, int n)
{    ulong    ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret    =    cast(ulong)block[n] * 2^21
+    cast(ulong)block[n+1] * 2^18
+    cast(ulong)block[n+2] * 2^15
+    cast(ulong)block[n+3] * 2^12
+    cast(ulong)block[n+4] * 2^9
+    cast(ulong)block[n+5] * 2^6
+    cast(ulong)block[n+6] * 2^3
+    cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return    ret;
}

```
Dec 21 2013
"ponce" <contact g3mesfrommars.fr> writes:
```On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote:
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all
the values in the array are 0.
The error has to be somewhere in the "ret = " statement, but I
sure don't see it.

/**    Convert 8 consecutive bytes sliced from a ubyte[] into
a ulong
*  param    block    The array from which to slice.
*  param    n    The starting index within the block    */
ulong    ubytesToUlong(ubyte[] block, int n)
{    ulong    ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret    =    cast(ulong)block[n] * 2^21
+    cast(ulong)block[n+1] * 2^18
+    cast(ulong)block[n+2] * 2^15
+    cast(ulong)block[n+3] * 2^12
+    cast(ulong)block[n+4] * 2^9
+    cast(ulong)block[n+5] * 2^6
+    cast(ulong)block[n+6] * 2^3
+    cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return    ret;
}

Use the exponentiation operator which is spelled: ^^
```
Dec 21 2013
```On 12/21/2013 02:31 PM, ponce wrote:
On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote:
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote:
I was planning to ask if there were a better way to do this, but
For some reason, if called with an uninitialized ubyte array, and an
index of 0, it returns a value of 8, even though all the values in
the array are 0.
The error has to be somewhere in the "ret = " statement, but I sure
don't see it.

/**    Convert 8 consecutive bytes sliced from a ubyte[] into a ulong
*  param    block    The array from which to slice.
*  param    n    The starting index within the block    */
ulong    ubytesToUlong(ubyte[] block, int n)
{    ulong    ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret    =    cast(ulong)block[n] * 2^21
+    cast(ulong)block[n+1] * 2^18
+    cast(ulong)block[n+2] * 2^15
+    cast(ulong)block[n+3] * 2^12
+    cast(ulong)block[n+4] * 2^9
+    cast(ulong)block[n+5] * 2^6
+    cast(ulong)block[n+6] * 2^3
+    cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return    ret;
}

Use the exponentiation operator which is spelled: ^^

Thanks.  I was *sure* it was something stupid on my part.  It's 'good'
to know that I was right about *that*.

--
Charles Hixson
```
Dec 21 2013
"John Colvin" <john.loughran.colvin gmail.com> writes:
```On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson
wrote:
I was planning to ask if there were a better way to do this,
For some reason, if called with an uninitialized ubyte array,
and an index of 0, it returns a value of 8, even though all the
values in the array are 0.
The error has to be somewhere in the "ret = " statement, but I
sure don't see it.

/**    Convert 8 consecutive bytes sliced from a ubyte[] into a
ulong
*  param    block    The array from which to slice.
*  param    n    The starting index within the block    */
ulong    ubytesToUlong(ubyte[] block, int n)
{    ulong    ret;
assert (n >= 0);
assert (n + 8 <= block.length);
writefln ("n = %s", n);
writefln ("block[0] = %s", cast(ulong)block[0]);
writefln ("block[1] = %s", cast(ulong)block[1]);
writefln ("block[2] = %s", cast(ulong)block[2]);
writefln ("block[3] = %s", cast(ulong)block[3]);
writefln ("block[4] = %s", cast(ulong)block[4]);
writefln ("block[5] = %s", cast(ulong)block[5]);
writefln ("block[6] = %s", cast(ulong)block[6]);
writefln ("block[7] = %s", cast(ulong)block[7]);
ret    =    cast(ulong)block[n] * 2^21
+    cast(ulong)block[n+1] * 2^18
+    cast(ulong)block[n+2] * 2^15
+    cast(ulong)block[n+3] * 2^12
+    cast(ulong)block[n+4] * 2^9
+    cast(ulong)block[n+5] * 2^6
+    cast(ulong)block[n+6] * 2^3
+    cast(ulong)block[n+7] * 2^0;
writefln ("ret = %s", ret);
return    ret;
}

As pointed out before, you're using '^' which is xor, instead of
^^. Ideally the compiler will optimise your version to be fast,
but you may find you get better performance by doing the bit
manipulations eplicitly:

/**Convert 8 consecutive bytes sliced from a ubyte[]
* to a ulong using bigendian byte ordering.
*  param    block    The array from which to slice.
*  param    n    The starting index within the block, default 0*/
ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
return ((cast(ulong)block[n  ]) << 56)
| ((cast(ulong)block[n+1]) << 48)
| ((cast(ulong)block[n+2]) << 40)
| ((cast(ulong)block[n+3]) << 32)
| ((cast(ulong)block[n+4]) << 24)
| ((cast(ulong)block[n+5]) << 16)
| ((cast(ulong)block[n+6]) <<  8)
|  (cast(ulong)block[n+7]);
}

unittest
{
ubyte[8] a = [0,0,0,0,0,0,0,0];

assert(a[].ubytesToUlong() == 0);

a[7] = 3;
a[6] = 1;

assert(a[].ubytesToUlong() == 259);
}
```
Dec 21 2013
=?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
```On 12/21/2013 03:13 PM, John Colvin wrote:

Ideally the compiler will optimise your version to be fast, but you may
find you get better performance by doing the bit manipulations eplicitly:

Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:

import std.bitmanip;
import std.system;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);

if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);

} else {
return value;
}
}

Ali
```
Dec 21 2013
"John Colvin" <john.loughran.colvin gmail.com> writes:
```On Saturday, 21 December 2013 at 23:52:05 UTC, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:

Ideally the compiler will optimise your version to be fast,

but you may
find you get better performance by doing the bit

manipulations eplicitly:

Assuming that the program needs to support only big endian and
little endian systems (i.e. excluding systems where no D
compiler exists :)), the following is less wordy and should be
equally fast:

import std.bitmanip;
import std.system;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);

if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);

} else {
return value;
}
}

Ali

Nevermind equally fast, that will be much faster. -10 brain
points for me tonight...
```
Dec 21 2013
```On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:

Ideally the compiler will optimise your version to be fast, but you may
find you get better performance by doing the bit manipulations

eplicitly:

Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:

import std.bitmanip;
import std.system;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);

if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);

} else {
return value;
}
}

Ali

Will that work even when the alignment is to odd bytes?  Because that's
the case I was really worried about.  The ubyte array is a packed
mixture of types, some of which are isolated bytes.

--
Charles Hixson
```
Dec 21 2013
=?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
```On 12/21/2013 05:44 PM, Charles Hixson wrote:
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:

Ideally the compiler will optimise your version to be fast, but you may
find you get better performance by doing the bit manipulations

eplicitly:

Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:

import std.bitmanip;
import std.system;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);

if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);

} else {
return value;
}
}

Ali

Will that work even when the alignment is to odd bytes?  Because that's
the case I was really worried about.  The ubyte array is a packed
mixture of types, some of which are isolated bytes.

No, it is not guaranteed to work unless the alignment is right.

import std.array;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = block.front;
block.popFront();

foreach (ub; block) {
value <<= 8;
value |= ub;
}

return value;
}

Ali
```
Dec 21 2013
```On 12/21/2013 07:57 PM, Ali Çehreli wrote:
On 12/21/2013 05:44 PM, Charles Hixson wrote:
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:

Ideally the compiler will optimise your version to be fast, but

you may
find you get better performance by doing the bit manipulations

eplicitly:

Assuming that the program needs to support only big endian and little
endian systems (i.e. excluding systems where no D compiler exists :)),
the following is less wordy and should be equally fast:

import std.bitmanip;
import std.system;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);

if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);

} else {
return value;
}
}

Ali

Will that work even when the alignment is to odd bytes?  Because that's
the case I was really worried about.  The ubyte array is a packed
mixture of types, some of which are isolated bytes.

No, it is not guaranteed to work unless the alignment is right.

import std.array;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = block.front;
block.popFront();

foreach (ub; block) {
value <<= 8;
value |= ub;
}

return value;
}

Ali

Nice, but the block is longer than 8 bytes, so I should use a "for (i =
n; i < n + 8; i++)" rather than a foreach, and index off of i.  I
clearly need to redo the documentation a bit (even though it's form me
of a few months from now).  It needs to say something like "Convert a 8
byte slice from a ubyte array starting at index n into a ulong."  n
should always be required to be specified, so I don't want a default
value.  (0 was used as a test case, because I'd made a really stupid
mistake and used "^" for exponentiation, and then couldn't see what was
going on, so I was simplifying everything...and I still couldn't see
it.  Actually the array starts with a ushort, which specifies the number
of ulongs to follow before a bunch of bytes that are unintelligible data
to the class that's using this function.  (OTOH, it seems like something
generally useful, so I'll probably put it in a utils.d file, with some
other generally useful routines.)

OTOH, if I'm going to consider this to be a general utility function,
then I really don't want to make assumptions about where things start,
etc.  Perhaps I should throw an exception (other than assertion error)
if the index is bad or the array is to short for the given index.  I
need to think about that a bit more.  The alternative is to use enforce
rather than assertions...though as long as I'm the only user assertions
suffice.  (It's not going to be separately compiled.)

--
Charles Hixson
```
Dec 22 2013
=?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
```On 12/22/2013 01:04 AM, Charles Hixson wrote:

Nice, but the block is longer than 8 bytes, so I should use a "for (i =
n; i < n + 8; i++)" rather than a foreach, and index off of i.

Makes sense. That reminded me of the Phobos function that does exactly
what you want. Have you considered std.bitmanip.read?

Ali
```
Dec 22 2013
```On 12/22/2013 02:22 AM, Ali Çehreli wrote:
On 12/22/2013 01:04 AM, Charles Hixson wrote:

Nice, but the block is longer than 8 bytes, so I should use a "for (i =
n; i < n + 8; i++)" rather than a foreach, and index off of i.

Makes sense. That reminded me of the Phobos function that does exactly
what you want. Have you considered std.bitmanip.read?

Ali

No, thanks.  That's precisely what I was looking for.

--
Charles Hixson
```
Dec 22 2013
"John Colvin" <john.loughran.colvin gmail.com> writes:
```On Sunday, 22 December 2013 at 03:57:38 UTC, Ali Çehreli wrote:
On 12/21/2013 05:44 PM, Charles Hixson wrote:
On 12/21/2013 03:52 PM, Ali Çehreli wrote:
On 12/21/2013 03:13 PM, John Colvin wrote:

Ideally the compiler will optimise your version to be fast,
but you may
find you get better performance by doing the bit
manipulations

eplicitly:

Assuming that the program needs to support only big endian
and little
endian systems (i.e. excluding systems where no D compiler
exists :)),
the following is less wordy and should be equally fast:

import std.bitmanip;
import std.system;

ulong ubytesToUlong(ubyte[] block, size_t n = 0)
in
{
assert (n >= 0);
assert (n + 8 <= block.length);
}
body
{
ulong value = *cast(ulong*)(block.ptr + n);

if (std.system.endian == Endian.littleEndian) {
return *cast(ulong*)(value.nativeToBigEndian.ptr);

} else {
return value;
}
}

Ali

Will that work even when the alignment is to odd bytes?
Because that's
the case I was really worried about.  The ubyte array is a
packed
mixture of types, some of which are isolated bytes.

No, it is not guaranteed to work unless the alignment is right.