www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Comparing floating point: == vs is

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
Under what circumstances would a comparison of floating point values return 
true for is but not ==? If I understand corcectly, is does a bit-by-bit 
comparison. So, I would expect that if is returns true, == would return true. 
However, that does not seem to always be the case. Does anyone know why that 
would be?

In particular, I'm trying to write a function for swapping the endianness of 
floating point values, and while it seems to be generally be returning true for 
is, == keeps failing. So, it sounds like I have two floating point values which 
have the exact same bits (since is succeeds) but which aren't equal according 
to ==. I don't understand what's going on at all. Does anyone have a clue what 
might be happening?

- Jonathan M Davis
Jul 16 2011
next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1687.1310805329.14074.digitalmars-d-learn puremagic.com...
 Under what circumstances would a comparison of floating point values 
 return
 true for is but not ==? If I understand corcectly, is does a bit-by-bit
 comparison. So, I would expect that if is returns true, == would return 
 true.
 However, that does not seem to always be the case. Does anyone know why 
 that
 would be?

 In particular, I'm trying to write a function for swapping the endianness 
 of
 floating point values, and while it seems to be generally be returning 
 true for
 is, == keeps failing. So, it sounds like I have two floating point values 
 which
 have the exact same bits (since is succeeds) but which aren't equal 
 according
 to ==. I don't understand what's going on at all. Does anyone have a clue 
 what
 might be happening?

 - Jonathan M Davis

As of the last release, 'is' for floating point types is broken. (before it just did the same as ==) At compile time, a is b will be true if a and b have the same bit pattern, or if a and b are both nans. At run time, a is b will return true if they have the same bit pattern only, but may sometimes return false on platforms other than windows as it includes the padding bytes in the comparison.
Jul 16 2011
prev sibling next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
I'd recommend using Don's isIdentical (or something like that) in std.math 
until it's fixed. 
Jul 16 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 16 July 2011 19:03:51 Daniel Murphy wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1687.1310805329.14074.digitalmars-d-learn puremagic.com...
 
 Under what circumstances would a comparison of floating point values
 return
 true for is but not ==? If I understand corcectly, is does a bit-by-bit
 comparison. So, I would expect that if is returns true, == would return
 true.
 However, that does not seem to always be the case. Does anyone know why
 that
 would be?
 
 In particular, I'm trying to write a function for swapping the
 endianness
 of
 floating point values, and while it seems to be generally be returning
 true for
 is, == keeps failing. So, it sounds like I have two floating point
 values
 which
 have the exact same bits (since is succeeds) but which aren't equal
 according
 to ==. I don't understand what's going on at all. Does anyone have a
 clue
 what
 might be happening?
 
 - Jonathan M Davis

As of the last release, 'is' for floating point types is broken. (before it just did the same as ==) At compile time, a is b will be true if a and b have the same bit pattern, or if a and b are both nans. At run time, a is b will return true if they have the same bit pattern only, but may sometimes return false on platforms other than windows as it includes the padding bytes in the comparison.

Hmm. Good to know. But assuming that two floating point values have the same bit pattern, shouldn't == return true for them? So, if == is failing, then two floating point values aren't identical, correct? - Jonathan M Davis
Jul 16 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1689.1310808023.14074.digitalmars-d-learn puremagic.com...
 Hmm. Good to know. But assuming that two floating point values have the 
 same
 bit pattern, shouldn't == return true for them? So, if == is failing, then 
 two
 floating point values aren't identical, correct?

 - Jonathan M Davis

Unless one or both of the values are nans, yes. If I had to guess I'd say you've been swapping the padding bytes too.
Jul 16 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 16 July 2011 19:23:12 Daniel Murphy wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1689.1310808023.14074.digitalmars-d-learn puremagic.com...
 
 Hmm. Good to know. But assuming that two floating point values have the
 same
 bit pattern, shouldn't == return true for them? So, if == is failing,
 then two
 floating point values aren't identical, correct?
 
 - Jonathan M Davis

Unless one or both of the values are nans, yes. If I had to guess I'd say you've been swapping the padding bytes too.

I tried both casting to an integer of the appropriate size and swapping that and doing this: private T swapEndianImpl(T)(T val) if(isFloatingPoint!T) { import std.algorithm; union Union { Unqual!T _floating; ubyte[T.sizeof] _array; } Union u; u._floating = val; std.algorithm.reverse(u._array[]); return u._floating; } I was testing the function by reversing the values twice, and casting seemed to fry NaNs, whereas the union/array trick seems to usually result in the correct values when I print them (at least for the ones I've looked at) and is was returning true for most of them, but == has been failing. I don't know anything about padding bytes in floating point though, so maybe that's part of the problem. http://stackoverflow.com/questions/2782725/converting-float-values-from-big- endian-to-little-endian/2782742#2782742 seemed to indicate that I could just swap the bytes using the union/array trick, but I'm not understanding something here and/or something is off with either my implementation or the compiler. - Jonathan M Davis
Jul 16 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1690.1310809801.14074.digitalmars-d-learn puremagic.com...
 I tried both casting to an integer of the appropriate size and swapping 
 that
 and doing this:

 private T swapEndianImpl(T)(T val)
    if(isFloatingPoint!T)
 {
    import std.algorithm;

    union Union
    {
        Unqual!T        _floating;
        ubyte[T.sizeof] _array;
    }

    Union u;
    u._floating = val;
    std.algorithm.reverse(u._array[]);

    return u._floating;
 }

 I was testing the function by reversing the values twice, and casting 
 seemed
 to fry NaNs, whereas the union/array trick seems to usually result in the
 correct values when I print them (at least for the ones I've looked at) 
 and is
 was returning true for most of them, but == has been failing.

 I don't know anything about padding bytes in floating point though, so 
 maybe
 that's part of the problem.
 http://stackoverflow.com/questions/2782725/converting-float-values-from-big-
 endian-to-little-endian/2782742#2782742 seemed to indicate that I could 
 just
 swap the bytes using the union/array trick, but I'm not understanding
 something here and/or something is off with either my implementation or 
 the
 compiler.

 - Jonathan M Davis

The padding issue only applies to reals. On x86 they're 10 bytes with 0 bytes padding (windows) or 2 bytes padding (linux) or 6 bytes padding (osx). I don't know why it would be failing for floats or doubles though. Do you have some tests that fail?
Jul 16 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 16 July 2011 21:19:27 Daniel Murphy wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1690.1310809801.14074.digitalmars-d-learn puremagic.com...
 
 I tried both casting to an integer of the appropriate size and swapping
 that
 and doing this:
 
 private T swapEndianImpl(T)(T val)
 
    if(isFloatingPoint!T)
 
 {
 
    import std.algorithm;
    
    union Union
    {
    
        Unqual!T        _floating;
        ubyte[T.sizeof] _array;
    
    }
    
    Union u;
    u._floating = val;
    std.algorithm.reverse(u._array[]);
    
    return u._floating;
 
 }
 
 I was testing the function by reversing the values twice, and casting
 seemed
 to fry NaNs, whereas the union/array trick seems to usually result in
 the
 correct values when I print them (at least for the ones I've looked at)
 and is
 was returning true for most of them, but == has been failing.
 
 I don't know anything about padding bytes in floating point though, so
 maybe
 that's part of the problem.
 http://stackoverflow.com/questions/2782725/converting-float-values-from-
 big- endian-to-little-endian/2782742#2782742 seemed to indicate that I
 could just
 swap the bytes using the union/array trick, but I'm not understanding
 something here and/or something is off with either my implementation or
 the
 compiler.
 
 - Jonathan M Davis

The padding issue only applies to reals. On x86 they're 10 bytes with 0 bytes padding (windows) or 2 bytes padding (linux) or 6 bytes padding (osx). I don't know why it would be failing for floats or doubles though. Do you have some tests that fail?

This is also being discussed on the main list in the "Byte Order Swapping" thread, so you may want to discuss it there, but this is what I have at the moment: import core.bitop; /++ Swaps the endianness of the given value. Any integral value, character, or floating point value is accepted. +/ T swapEndian(T)(T val) if(isNumeric!T || isSomeChar!T) { static if(val.sizeof == 1) return val; else static if(isUnsigned!T || isFloatingPoint!T) return swapEndianImpl(val); else static if(isIntegral!T) return swapEndianImpl(cast(Unsigned!T) val); else static if(is(Unqual!T == wchar)) return cast(T)swapEndian(cast(ushort)val); else static if(is(Unqual!T == dchar)) return cast(T)swapEndian(cast(uint)val); else static assert(0, T.stringof ~ " unsupported by swapEndian."); } private T swapEndianImpl(T)(T val) if(is(Unqual!T == ushort)) { return ((val & 0xff00U) >> 8) | ((val & 0x00ffU) << 8); } private T swapEndianImpl(T)(T val) if(is(Unqual!T == uint)) { return bswap(val); } private T swapEndianImpl(T)(T val) if(is(Unqual!T == ulong)) { return ((val & 0xff00000000000000UL) >> 56) | ((val & 0x00ff000000000000UL) >> 40) | ((val & 0x0000ff0000000000UL) >> 24) | ((val & 0x000000ff00000000UL) >> 8) | ((val & 0x00000000ff000000UL) << 8) | ((val & 0x0000000000ff0000UL) << 24) | ((val & 0x000000000000ff00UL) << 40) | ((val & 0x00000000000000ffUL) << 56); } private T swapEndianImpl(T)(T val) if(isFloatingPoint!T) { import std.algorithm; union Union { Unqual!T _floating; ubyte[T.sizeof] _array; } Union u; u._floating = val; std.algorithm.reverse(u._array[]); return u._floating; } unittest { import std.stdio; import std.typetuple; foreach(T; TypeTuple!(byte, ubyte, short, ushort, int, uint, long, ulong, char, wchar, dchar, float, double, real)) { scope(failure) writefln("Failed type: %s", T.stringof); T val; const T cval; immutable T ival; assert(swapEndian(swapEndian(val)) == val); assert(swapEndian(swapEndian(cval)) == cval); assert(swapEndian(swapEndian(ival)) == ival); assert(swapEndian(swapEndian(T.min)) == T.min); assert(swapEndian(swapEndian(T.max)) == T.max); static if(isSigned!T) assert(swapEndian(swapEndian(cast(T)0)) == 0); } } I suspect that the problem is a combination of issues with NaN and real. If I change the first three tests (which test init which is NaN) to is, then they pass, but the test for real.max fails regardless of whether is or == is used. The other tests seem to pass regardless of whether is or == is used. So, I may need to do something special for NaN, and I may have to do something to deal with the padding in real. I don't know. - Jonathan M Davis
Jul 16 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1705.1310853321.14074.digitalmars-d-learn puremagic.com...
 I suspect that the problem is a combination of issues with NaN and real. 
 If I
 change the first three tests (which test init which is NaN) to is, then 
 they
 pass, but the test for real.max fails regardless of whether is or == is 
 used.
 The other tests seem to pass regardless of whether is or == is used. So, I 
 may
 need to do something special for NaN, and I may have to do something to 
 deal
 with the padding in real. I don't know.

 - Jonathan M Davis

They all pass for me when using 'is' on win32, so assuming you're on linux or osx the padding issue is what you're running into. The workaround is probably using isIdentical from std.math for floating point types instead of 'is' and only swapping the first 10 bytes in endianswap (for reals only, the other types should work). The 'is' problem will be fixed, but swapping the full 12 or 16 bytes of real will always result in the wrong value.
Jul 16 2011
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 17 July 2011 14:19:12 Daniel Murphy wrote:
 "Jonathan M Davis" <jmdavisProg gmx.com> wrote in message
 news:mailman.1705.1310853321.14074.digitalmars-d-learn puremagic.com...
 
 I suspect that the problem is a combination of issues with NaN and real.
 If I
 change the first three tests (which test init which is NaN) to is, then
 they
 pass, but the test for real.max fails regardless of whether is or == is
 used.
 The other tests seem to pass regardless of whether is or == is used. So,
 I may
 need to do something special for NaN, and I may have to do something to
 deal
 with the padding in real. I don't know.
 
 - Jonathan M Davis

They all pass for me when using 'is' on win32, so assuming you're on linux or osx the padding issue is what you're running into. The workaround is probably using isIdentical from std.math for floating point types instead of 'is' and only swapping the first 10 bytes in endianswap (for reals only, the other types should work). The 'is' problem will be fixed, but swapping the full 12 or 16 bytes of real will always result in the wrong value.

I'm on Linux. I need to do whatever will result in the correct value with == if that is at all possible. The question the is twofold: 1. How to make it so that swapping the endianness twice doesn't butcher NaNs. 2. How to properly swap reals if they have padding. It sounds like the solution for reals is to only swap part of it. So, I assume that the first 10 bytes are a different ten bytes on each platform? If so, are they the ones on the left in little endian or the ones on the right (making big endian the other)? i.e. value[0 .. 10] or value[$ - 10 .. $]? And that still leaves the question of NaN. Why would reversing the bytes twice make NaN's not equal with == anymore? - Jonathan M Davis
Jul 16 2011
parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1708.1310882400.14074.digitalmars-d-learn puremagic.com...
 I'm on Linux.

 I need to do whatever will result in the correct value with == if that is 
 at
 all possible. The question the is twofold:

 1. How to make it so that swapping the endianness twice doesn't butcher 
 NaNs.

 2. How to properly swap reals if they have padding.

 It sounds like the solution for reals is to only swap part of it. So, I 
 assume
 that the first 10 bytes are a different ten bytes on each platform? If so, 
 are
 they the ones on the left in little endian or the ones on the right 
 (making
 big endian the other)? i.e. value[0 .. 10] or value[$ - 10 .. $]?

They're just padding bytes, not included in the structure, so they always come afterwards. eg. union { real f; ubyte[10] b; } should give you them in little or big endian.
 And that still leaves the question of NaN. Why would reversing the bytes 
 twice
 make NaN's not equal with == anymore?

If you were reversing the full 12 bytes, then comparing, some of the data would have ended up in the padding bytes and could have got lost when moving to/from the stack or the floating point registers. Don't forget that is will do a bitwise comparison including the padding bytes on linux, so isIdentical will likely give you the results you want.
Jul 17 2011