www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Boy, std.bitmanip.bigEndianToNative is annoying to use

reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
In the spirit of forum bickering, ;-) I stumbled upon this D wart today:
I'm reading some data from a file into a ubyte[] buffer, and I want to
use bigEndianToNative to convert ushort values in the file data into
native byte order (whatever the native order might be).

Sounds simple, right? Unfortunately, bigEndianToNative asks for ubyte[n]
as input. Meaning, this doesn't work:

	ubyte[] buf = ... /* allocate buffer here */;
	file.rawRead(buf);	// Read the data

	ushort myValue = bigEndianToNative!ushort(buf[4 .. 8]); // NG

The last line doesn't compile, 'cos you can't convert a slice of ubyte[]
into ubyte[4].

I can think of no easy way to declare a temporary ubyte[4] to make
bigEndianToNative happy, other than this silly verbosity:

	ubyte[4] tmp;
	tmp[] = buf[4 .. 8];
	ushort myValue = bigEndianToNative!ushort(tmp);

and I have to do this for every single numerical field in the data
buffer that I need to convert. :-(  Why should I copy data around that's
already sitting in a ubyte[] buffer intended precisely for the purpose
of doing such conversions in the first place??

The docs for bigEndianToNative claims that this is to help "prevent
accidentally using a swapped value as a regular one". But I say, "Why,
oh why???" :-(

This is a very anti-user kind of API. How did we think such a
straitjacketed API was a good idea in the first place?!


T

-- 
Curiosity kills the cat. Moral: don't be the cat.
May 22 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 22 May 2015 at 19:10:41 UTC, H. S. Teoh wrote:

 In the spirit of forum bickering, ;-) I stumbled upon this D 
 wart today:
 I'm reading some data from a file into a ubyte[] buffer, and I 
 want to
 use bigEndianToNative to convert ushort values in the file data 
 into
 native byte order (whatever the native order might be).

 Sounds simple, right? Unfortunately, bigEndianToNative asks for 
 ubyte[n]
 as input. Meaning, this doesn't work:

 	ubyte[] buf = ... /* allocate buffer here */;
 	file.rawRead(buf);	// Read the data

 	ushort myValue = bigEndianToNative!ushort(buf[4 .. 8]); // NG

 The last line doesn't compile, 'cos you can't convert a slice 
 of ubyte[]
 into ubyte[4].

 I can think of no easy way to declare a temporary ubyte[4] to 
 make
 bigEndianToNative happy, other than this silly verbosity:

 	ubyte[4] tmp;
 	tmp[] = buf[4 .. 8];
 	ushort myValue = bigEndianToNative!ushort(tmp);

 and I have to do this for every single numerical field in the 
 data
 buffer that I need to convert. :-(  Why should I copy data 
 around that's
 already sitting in a ubyte[] buffer intended precisely for the 
 purpose
 of doing such conversions in the first place??

 The docs for bigEndianToNative claims that this is to help 
 "prevent
 accidentally using a swapped value as a regular one". But I 
 say, "Why,
 oh why???" :-(

 This is a very anti-user kind of API. How did we think such a
 straitjacketed API was a good idea in the first place?!


Isn't the problem that you're trying to convert to a ushort, and 
a ushort is _2_ bytes, not 4? If you sliced it correctly, it 
would compile. For instance, this code compiles just fine for me 
with dmd master:

void main()
{
     import std.bitmanip;
     auto buf = new ubyte[](32);
     auto result = bigEndianToNative!ushort(buf[0 .. 2]);
}

But if I change it to buf[0 .. 4], then it fails to compile. So, 
the fact that bigEndianToNative is taking a static array is 
actually catching a bug for you.

- Jonathan M Davis
May 22 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/22/15 12:21 PM, Jonathan M Davis wrote:

 But if I change it to buf[0 .. 4], then it fails to compile. So, the
 fact that bigEndianToNative is taking a static array is actually
 catching a bug for you.


Profession of undying love for D language in 3, 2, 1, ... :o) -- Andrei
May 22 2015
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, May 22, 2015 at 07:21:40PM +0000, Jonathan M Davis via Digitalmars-d
wrote:
[...]

 Isn't the problem that you're trying to convert to a ushort, and a
 ushort is _2_ bytes, not 4? If you sliced it correctly, it would
 compile. For instance, this code compiles just fine for me with dmd
 master:
 
 void main()
 {
     import std.bitmanip;
     auto buf = new ubyte[](32);
     auto result = bigEndianToNative!ushort(buf[0 .. 2]);
 }
 
 But if I change it to buf[0 .. 4], then it fails to compile. So, the
 fact that bigEndianToNative is taking a static array is actually
 catching a bug for you.


[...]

	face.palm();
	head.hang!inShame();
	self.crawlBackTo(hole);


T

-- 
Perhaps the most widespread illusion is that if we were in power we would
behave very differently from those who now hold it---when, in truth, in order
to get power we would have to become very much like them. -- Unknown
May 22 2015
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, May 22, 2015 at 12:39:12PM -0700, H. S. Teoh via Digitalmars-d wrote:
[...]

 	face.palm();
 	head.hang!inShame();
 	self.crawlBackTo(hole);


[...]

Truth be told, though, the error message was very unhelpful about it. It
just says no templates were matched, no indication about the fact that
the problem was caused by mismatched type lengths.


T

-- 
Verbing weirds language. -- Calvin (& Hobbes)
May 22 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 22 May 2015 at 19:44:59 UTC, H. S. Teoh wrote:

 On Fri, May 22, 2015 at 12:39:12PM -0700, H. S. Teoh via 
 Digitalmars-d wrote:
 [...]

 	face.palm();
 	head.hang!inShame();
 	self.crawlBackTo(hole);


 [...]

 Truth be told, though, the error message was very unhelpful 
 about it. It
 just says no templates were matched, no indication about the 
 fact that
 the problem was caused by mismatched type lengths.


Well, I'm not sure how we could fix that. That's a general 
problem with template constraints. You know that they aren't 
passing, but you don't know _why_ they aren't passing. Though 
maybe in this case, the docs could be improved to make it clearer 
that if you're getting compilation errors, there's a decent 
chance that you got the type sizes wrong.

- Jonathan M Davis
May 22 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/22/15 12:48 PM, Jonathan M Davis wrote:

 On Friday, 22 May 2015 at 19:44:59 UTC, H. S. Teoh wrote:

 On Fri, May 22, 2015 at 12:39:12PM -0700, H. S. Teoh via Digitalmars-d
 wrote:
 [...]

     face.palm();
     head.hang!inShame();
     self.crawlBackTo(hole);


 [...]

 Truth be told, though, the error message was very unhelpful about it. It
 just says no templates were matched, no indication about the fact that
 the problem was caused by mismatched type lengths.


 Well, I'm not sure how we could fix that. That's a general problem with
 template constraints. You know that they aren't passing, but you don't
 know _why_ they aren't passing. Though maybe in this case, the docs
 could be improved to make it clearer that if you're getting compilation
 errors, there's a decent chance that you got the type sizes wrong.


Yah, improving the documentation has maximum impact here. Navigating to 
github.com/pulls?user=D-Programming-Language... F5... F5... F5... F5... 
-- Andrei
May 22 2015
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Friday, 22 May 2015 at 19:48:21 UTC, Jonathan M Davis wrote:

 Well, I'm not sure how we could fix that.


Remove filter n == T.sizeof ?
May 22 2015
parent reply "Kagamin" <spam here.lot> writes:
Hmm, now that I look at it, the declaration of bigEndianToNative 
is weird: http://dpaste.dzfl.pl/6fad7c9ef22d
May 22 2015
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 22 May 2015 at 20:21:59 UTC, Kagamin wrote:

 Hmm, now that I look at it, the declaration of 
 bigEndianToNative is weird: http://dpaste.dzfl.pl/6fad7c9ef22d


What's weird about it?

- Jonathan M Davis
May 22 2015
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 22 May 2015 at 21:03:43 UTC, Jonathan M Davis wrote:

 On Friday, 22 May 2015 at 20:21:59 UTC, Kagamin wrote:

 Hmm, now that I look at it, the declaration of 
 bigEndianToNative is weird: http://dpaste.dzfl.pl/6fad7c9ef22d


 What's weird about it?


Why it has a separate n parameter? It makes no sense. If it was 
like in the paste, it would give proper error message.
May 23 2015
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, May 23, 2015 at 09:54:35AM +0000, Kagamin via Digitalmars-d wrote:

 On Friday, 22 May 2015 at 21:03:43 UTC, Jonathan M Davis wrote:

On Friday, 22 May 2015 at 20:21:59 UTC, Kagamin wrote:

Hmm, now that I look at it, the declaration of bigEndianToNative is
weird: http://dpaste.dzfl.pl/6fad7c9ef22d


What's weird about it?


 
 Why it has a separate n parameter? It makes no sense. If it was like
 in the paste, it would give proper error message.


Submit a PR?


T

-- 
MAS = Mana Ada Sistem?
May 23 2015
prev sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 23/05/2015 7:48 a.m., Jonathan M Davis wrote:

 On Friday, 22 May 2015 at 19:44:59 UTC, H. S. Teoh wrote:

 On Fri, May 22, 2015 at 12:39:12PM -0700, H. S. Teoh via Digitalmars-d
 wrote:
 [...]

     face.palm();
     head.hang!inShame();
     self.crawlBackTo(hole);


 [...]

 Truth be told, though, the error message was very unhelpful about it. It
 just says no templates were matched, no indication about the fact that
 the problem was caused by mismatched type lengths.


 Well, I'm not sure how we could fix that. That's a general problem with
 template constraints. You know that they aren't passing, but you don't
 know _why_ they aren't passing. Though maybe in this case, the docs
 could be improved to make it clearer that if you're getting compilation
 errors, there's a decent chance that you got the type sizes wrong.

 - Jonathan M Davis



template X(T, string v) if (is(T == class) && v.length == 2) {

} else {
	pragma(msg, "X must have two template arguments, where the first is a 
class symbol and the second is a string with a length of two characters.");
}

Or

template X() if (false) {
	pragma(msg, "X must have two template arguments, where the first is a 
class symbol and the second is a string with a length of two characters.");
}

Humm, the first would be abused a lot. The second would be highly 
limiting. The first, but all overloads must be in the style of if, else 
template, else. But how to limit it to one name..

Anyway just some random thoughts.
May 22 2015