www.digitalmars.com         C & C++   DMDScript  

D - Bit Fields and Endianness

reply Russell Lewis <spamhole-2001-07-16 deming-os.org> writes:
I'm currently implementing a standard network protocol in C++.  (Sorry, 
they don't let me use D for work projects yet.)  I make extensive use of 
bit fields, and they are a Good Thing (assuming that you can depend on 
their endianness and physical arrangement).  Yeah, I know that you could 
write a D class with gettors and settors to emulate bit fields, but that 
is an ugly hack.

I vote that we should be able to have bit fields.  We should also be 
able to set endianness as a storage property.  Something like:

    bigEndian u8;

If the machine uses the opposite endianness than what is specified, then 
it implements the byte swap automatically.

While we're on this, a superior solution to bitfields would be the 
ability to set any bit size integers, like this:

struct protocol_message_foo
{
   u1 field1;
   u2 field2;
   u5 reserved;
   bigEndian u24 length;
};

u1, of course, is just an alias of 'bit'.

Just my 2 cents.



For any who are interested, my personal time project (which builds D 
code automatically) is getting along well.  I have almost 5,000 lines of 
D code written, which turns a 337 line definition into a 25,000 line D 
source.  Compiling that makes my poor little old Linux box (with only 
32Mb RAM) thrash for a LONG time.....
Jan 06 2003
next sibling parent "Mike Wynn" <mike.wynn l8night.co.uk> writes:
infact 4 endians might be needed
little, big, host (the default) and network

something to think over is how to have bit packed structs
imagine a big endian struct with a network order bit field arrrgh!

as an aside one thing that I have used on cross platfrom comms protocols as
palendromic and non palendromic (can't spell for toffee) numbers.
you send a magic 0x12343412 as yet is the right comms protocol followed by
0x12345678 (I'm this endian)
ONLY one end has to switch the endianess if the 0x12345678 comes out as
0x78563412
... as long as the c compilers align the structs the same :)




"Russell Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:3E19CDD5.7020508 deming-os.org...
 I'm currently implementing a standard network protocol in C++.  (Sorry,
 they don't let me use D for work projects yet.)  I make extensive use of
 bit fields, and they are a Good Thing (assuming that you can depend on
 their endianness and physical arrangement).  Yeah, I know that you could
 write a D class with gettors and settors to emulate bit fields, but that
 is an ugly hack.

 I vote that we should be able to have bit fields.  We should also be
 able to set endianness as a storage property.  Something like:

     bigEndian u8;

 If the machine uses the opposite endianness than what is specified, then
 it implements the byte swap automatically.

 While we're on this, a superior solution to bitfields would be the
 ability to set any bit size integers, like this:

 struct protocol_message_foo
 {
    u1 field1;
    u2 field2;
    u5 reserved;
    bigEndian u24 length;
 };

 u1, of course, is just an alias of 'bit'.

 Just my 2 cents.



 For any who are interested, my personal time project (which builds D
 code automatically) is getting along well.  I have almost 5,000 lines of
 D code written, which turns a 337 line definition into a 25,000 line D
 source.  Compiling that makes my poor little old Linux box (with only
 32Mb RAM) thrash for a LONG time.....

Jan 06 2003
prev sibling next sibling parent reply Theodore Reed <rizen surreality.us> writes:
On Mon, 6 Jan 2003 22:54:24 -0000
"Mike Wynn" <mike.wynn l8night.co.uk> wrote:

 infact 4 endians might be needed
 little, big, host (the default) and network

What's the difference between big and network? -- Theodore Reed (rizen/bancus) -==- http://www.surreality.us/ ~OpenPGP Signed/Encrypted Mail Preferred; Finger me for my public key!~ "I hold it to be the inalienable right of anybody to go to hell in his own way." -- Robert Frost
Jan 06 2003
parent "Mike Wynn" <mike.wynn l8night.co.uk> writes:
"Theodore Reed" <rizen surreality.us> wrote in message
news:20030106193738.65197a34.rizen surreality.us...
 On Mon, 6 Jan 2003 22:54:24 -0000
 "Mike Wynn" <mike.wynn l8night.co.uk> wrote:

 infact 4 endians might be needed
 little, big, host (the default) and network

What's the difference between big and network?

depends on the hardware I guess.
Jan 07 2003
prev sibling parent reply Chris <cl nopressedmeat.tinfoilhat.ca> writes:
Russell Lewis wrote:
 I'm currently implementing a standard network protocol in C++.  (Sorry, 
 they don't let me use D for work projects yet.)  I make extensive use of 
 bit fields, and they are a Good Thing (assuming that you can depend on 
 their endianness and physical arrangement).  Yeah, I know that you could 
 write a D class with gettors and settors to emulate bit fields, but that 
 is an ugly hack.
 
 I vote that we should be able to have bit fields.  We should also be 
 able to set endianness as a storage property.  Something like:
 
    bigEndian u8;
 
 If the machine uses the opposite endianness than what is specified, then 
 it implements the byte swap automatically.

Are there arguments against this? Is adding this worth the extra effort required of compiler implementors? Having the 4 types seems like a really good idea to me. Chris
Jan 15 2003
parent reply Steve <Steve_member pathlink.com> writes:
It does not seem a good idea to explicitly specify the byte order the compiler
must store scalar types to memory in because you may not know at compile time
.. It is conceivable that a cpu could switch its endian-ness part way through
execution ... though I can't off the top of my head thing of why you'd want to
do this ...


In article <3E25CF35.8070206 nopressedmeat.tinfoilhat.ca>, Chris says...
Russell Lewis wrote:
 I'm currently implementing a standard network protocol in C++.  (Sorry, 
 they don't let me use D for work projects yet.)  I make extensive use of 
 bit fields, and they are a Good Thing (assuming that you can depend on 
 their endianness and physical arrangement).  Yeah, I know that you could 
 write a D class with gettors and settors to emulate bit fields, but that 
 is an ugly hack.
 
 I vote that we should be able to have bit fields.  We should also be 
 able to set endianness as a storage property.  Something like:
 
    bigEndian u8;
 
 If the machine uses the opposite endianness than what is specified, then 
 it implements the byte swap automatically.

Are there arguments against this? Is adding this worth the extra effort required of compiler implementors? Having the 4 types seems like a really good idea to me. Chris

Jan 15 2003
parent reply Brad Beveridge <b.beveridge clear.net.nz> writes:
Steve wrote:
 It does not seem a good idea to explicitly specify the byte order the compiler
 must store scalar types to memory in because you may not know at compile time
 .. It is conceivable that a cpu could switch its endian-ness part way through
 execution ... though I can't off the top of my head thing of why you'd want to
 do this ...

I think that the compiler would know the endian-ness at compile time, otherwise it would arrange constants wrong. Also, would a CPU change endian-ness in real life partway though a program? What would that do to the data already in memory - ouch! I can see the use of bigEndian/littleEndian keywords when writing code that must run on multiple architectures, but assemble data packets in a specific way. If you could do something like bigEndian struct { .. } ethernetPacket and have the compiler do all the byte swizzling, that would be hugely beneficial. Since this is also my first post, I thought I'd just say hi & say that D is a very nice sounding language, can't wait to play with it. Just have to wait (or help, gasp!) until the linux port/gcc for D is better :) Cheers Brad
 
 
 In article <3E25CF35.8070206 nopressedmeat.tinfoilhat.ca>, Chris says...
 
Russell Lewis wrote:

I'm currently implementing a standard network protocol in C++.  (Sorry, 
they don't let me use D for work projects yet.)  I make extensive use of 
bit fields, and they are a Good Thing (assuming that you can depend on 
their endianness and physical arrangement).  Yeah, I know that you could 
write a D class with gettors and settors to emulate bit fields, but that 
is an ugly hack.

I vote that we should be able to have bit fields.  We should also be 
able to set endianness as a storage property.  Something like:

   bigEndian u8;

If the machine uses the opposite endianness than what is specified, then 
it implements the byte swap automatically.

Are there arguments against this? Is adding this worth the extra effort required of compiler implementors? Having the 4 types seems like a really good idea to me. Chris


Jan 15 2003
next sibling parent Russell Lewis <spamhole-2001-07-16 deming-os.org> writes:
Brad Beveridge wrote:
 Steve wrote:
 
 It does not seem a good idea to explicitly specify the byte order the 
 compiler
 must store scalar types to memory in because you may not know at 
 compile time
 .. It is conceivable that a cpu could switch its endian-ness part way 
 through
 execution ... though I can't off the top of my head thing of why you'd 
 want to
 do this ...

I think that the compiler would know the endian-ness at compile time, otherwise it would arrange constants wrong. Also, would a CPU change endian-ness in real life partway though a program? What would that do to the data already in memory - ouch! I can see the use of bigEndian/littleEndian keywords when writing code that must run on multiple architectures, but assemble data packets in a specific way. If you could do something like bigEndian struct { .. } ethernetPacket and have the compiler do all the byte swizzling, that would be hugely beneficial. Since this is also my first post, I thought I'd just say hi & say that D is a very nice sounding language, can't wait to play with it. Just have to wait (or help, gasp!) until the linux port/gcc for D is better :) Cheers Brad
 In article <3E25CF35.8070206 nopressedmeat.tinfoilhat.ca>, Chris says...

 Russell Lewis wrote:

 I'm currently implementing a standard network protocol in C++.  
 (Sorry, they don't let me use D for work projects yet.)  I make 
 extensive use of bit fields, and they are a Good Thing (assuming 
 that you can depend on their endianness and physical arrangement).  
 Yeah, I know that you could write a D class with gettors and settors 
 to emulate bit fields, but that is an ugly hack.

 I vote that we should be able to have bit fields.  We should also be 
 able to set endianness as a storage property.  Something like:

   bigEndian u8;

 If the machine uses the opposite endianness than what is specified, 
 then it implements the byte swap automatically.

Are there arguments against this? Is adding this worth the extra effort required of compiler implementors? Having the 4 types seems like a really good idea to me. Chris



Try Burton Radon's DLI port. It works well (darn near perfect, in fact), provided that your code is valid. It is available from Walter's main page: http://digitalmars.com/d. Look for the "Linux port" link. I'm thinking that maybe I should write a quick FAQ about the setup, installation, and known bugs in DLI...
Jan 15 2003
prev sibling parent reply Steve <Steve_member pathlink.com> writes:
 I think that the compiler would know the endian-ness at compile time, 

endian-ness in real life partway though a program? What would that do to the data already in memory - ouch! Sure, to transition endianness, you'd have to clear out all dynamic data to some form of storage device, then reload following the state change. Like I said, can't think of why you'd want to, but it could be done ...
I think that the compiler would know the endian-ness at compile time, 
otherwise it would arrange constants wrong.  

file should be well defined ... Going off on a bit of a tangent here ... Sorry ... In article <b04kgr$bv7$1 digitaldaemon.com>, Brad Beveridge says...
Steve wrote:
 It does not seem a good idea to explicitly specify the byte order the compiler
 must store scalar types to memory in because you may not know at compile time
 .. It is conceivable that a cpu could switch its endian-ness part way through
 execution ... though I can't off the top of my head thing of why you'd want to
 do this ...

I think that the compiler would know the endian-ness at compile time, otherwise it would arrange constants wrong. Also, would a CPU change endian-ness in real life partway though a program? What would that do to the data already in memory - ouch! I can see the use of bigEndian/littleEndian keywords when writing code that must run on multiple architectures, but assemble data packets in a specific way. If you could do something like bigEndian struct { .. } ethernetPacket and have the compiler do all the byte swizzling, that would be hugely beneficial. Since this is also my first post, I thought I'd just say hi & say that D is a very nice sounding language, can't wait to play with it. Just have to wait (or help, gasp!) until the linux port/gcc for D is better :) Cheers Brad
 
 
 In article <3E25CF35.8070206 nopressedmeat.tinfoilhat.ca>, Chris says...
 
Russell Lewis wrote:

I'm currently implementing a standard network protocol in C++.  (Sorry, 
they don't let me use D for work projects yet.)  I make extensive use of 
bit fields, and they are a Good Thing (assuming that you can depend on 
their endianness and physical arrangement).  Yeah, I know that you could 
write a D class with gettors and settors to emulate bit fields, but that 
is an ugly hack.

I vote that we should be able to have bit fields.  We should also be 
able to set endianness as a storage property.  Something like:

   bigEndian u8;

If the machine uses the opposite endianness than what is specified, then 
it implements the byte swap automatically.

Are there arguments against this? Is adding this worth the extra effort required of compiler implementors? Having the 4 types seems like a really good idea to me. Chris



Jan 15 2003
parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Steve wrote:

 I think that the compiler would know the endian-ness at compile time,

endian-ness in real life partway though a program? What would that do to the data already in memory - ouch! Sure, to transition endianness, you'd have to clear out all dynamic data to some form of storage device, then reload following the state change. Like I said, can't think of why you'd want to, but it could be done ...

The only way to "swap endianness" is to know the size & location of all variables. The hardware can't tell the difference between a u32 (endianness matters) and a vector of 4 u8s (endianness doesn't matter, but location definitely does!) Any machine (or OS) with enough information to know how to swap your endianness automagically would/could certainly have hooks to define certain variables as "endianness sensitive". -- The Villagers are Online! http://villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
Jan 15 2003