www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - interfacing C bitfield structs

reply Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
I'm currently writing D includes for several game and graphics
related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
Freetype2).

Now with SDL I've encountered a problem: SDL makes use of serval
bitfields and I've no idea, how to interface them to D.

Has somebody an idea how I best interface with it elegantly?

Greetings

Wolfgang Draxinger
Mar 17 2006
next sibling parent reply Kyle Furlong <kylefurlong gmail.com> writes:
Wolfgang Draxinger wrote:
 I'm currently writing D includes for several game and graphics
 related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
 Freetype2).
 
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.
 
 Has somebody an idea how I best interface with it elegantly?
 
 Greetings
 
 Wolfgang Draxinger
 

I would just like to make you aware of the fact that all these have been interfaced to D already. You can find many of these bindings (and more) on www.dsource.org under a few projects: bindings, terra, and derelict.
Mar 17 2006
parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Kyle Furlong wrote:

 I would just like to make you aware of the fact that all these
 have been interfaced to D already. You can find many of these
 bindings (and more) on www.dsource.org under a few projects:
 bindings, terra, and derelict.

I know, but they failed to work for me, last time I checked. Don't know how it's unter Windows but I need it for Linux. Additionally I made bindings to GLEW so that you can use OpenGL extensions with a breeze; due to some heavy use of macros I haven't yet added interfaces to the GLEW multi context function. This requires for every extension function either a template wrapper or intermediary function built around, that calls the function pointer from a context struct. -- Wolfgang Draxinger
Mar 17 2006
prev sibling next sibling parent reply Lucas Goss <lgoss007 gmail.com> writes:
Wolfgang Draxinger wrote:
 I'm currently writing D includes for several game and graphics
 related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
 Freetype2).

Me too :)
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.
 
 Has somebody an idea how I best interface with it elegantly?

Not sure what problem you are having. Kyle Furlong wrote:
 ...You can find many of these
 bindings (and more) on www.dsource.org under a few projects:
 bindings, terra, and derelict.


 I know, but they failed to work for me, last time I checked.

Well I know derelict works on linux. But if you don't want to take that approach (the dynamic loading), Anders has sdl bindings here: http://www.algonet.se/~afb/d/ Which you can use or might help point you in the right direction. Lucas
Mar 17 2006
parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Lucas Goss wrote:

 But if you don't want to 
 take that approach (the dynamic loading), Anders has sdl
 bindings here: http://www.algonet.se/~afb/d/

Nice, eventually I even use his bindings. Eventually I put my GLEW bindings over his. My GLEW bindings are up and running, without multicontext support though. I still must write a small script the generates the D template functions for this. If compiled with multitexture supprort GLEW expects the user to supply a function GLEWContext *glewGetContext(). This is used in macros like #define glActiveTextureUnitARB GLEW_GET_FUN(__glewActiveTextureUnitARB) And GLEW_GET_FUN is a macro #ifdef GLEW_MX #define GLEW_GET_FUN(x) glewGetContext()->x #else #define GLEW_GET_FUN(x) x #endif So far the only way I know to bring this to D is to do something like this version(GLEW_MX) { template glActiveTextureUnitARB(unit){ GLvoid glActiveTextureUnitARB(unit) { return __glewActiveTextureUnitARB(unit); } } } else { alias __glewActiveTextureUnitARB glActiveTextureUnitARB; } I'm using templates since using normal functions would eventually be compiled not as inline, which is preferred. This got me to an idea for a new D feature, that would allow to use aliases as a replacement for macros, if I could have written template glActiveTextureUnitARB(...) function { return glewGetContext().__glewActiveTextureUnitARB(...) ; } It would be much more readable. I mean, that by adding the keyword 'function' after the template definition you don't have to repeat the template name to make it a function template and the ellipsis '...' as a placeholder for an arbitrary long parameter list, eventually supporting some scheme to extract a certain parameter. But so I have to extract the parameter list and the return type from the extension functions definition and generate template worms for it. -- Wolfgang Draxinger
Mar 18 2006
prev sibling next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 17 Mar 2006 23:50:39 +0100, Wolfgang Draxinger  
<wdraxinger darkstargames.de> wrote:
 I'm currently writing D includes for several game and graphics
 related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
 Freetype2).

 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.

 Has somebody an idea how I best interface with it elegantly?

I don't know if you'd call this elegant but here is what I did. The important thing is using a type which is the correct size for the bitfields and then creating some way to interface the bits you need. struct DCB { DWORD DCBlength; /* sizeof(DCB) */ DWORD BaudRate; /* Baudrate at which running */ uint BITS; //bit fBinary; /* Binary Mode (skip EOF check) */ //bit fParity; /* Enable parity checking */ //bit fOutxCtsFlow; /* CTS handshaking on output */ //bit fOutxDsrFlow; /* DSR handshaking on output */ //bit[2] fDtrControl; /* DTR Flow control */ //bit fDsrSensitivity; /* DSR Sensitivity */ //bit fTXContinueOnXoff; /* Continue TX when Xoff sent */ //bit fOutX; /* Enable output X-ON/X-OFF */ //bit fInX; /* Enable input X-ON/X-OFF */ //bit fErrorChar; /* Enable Err Replacement */ //bit fNull; /* Enable Null stripping */ //bit[2] fRtsControl; /* Rts Flow control */ //bit fAbortOnError; /* Abort all reads and writes on Error */ //bit[17] fDummy2; /* Reserved */ WORD wReserved; /* Not currently used */ WORD XonLim; /* Transmit X-ON threshold */ WORD XoffLim; /* Transmit X-OFF threshold */ BYTE ByteSize; /* Number of bits/byte, 4-8 */ BYTE Parity; /* 0-4=None,Odd,Even,Mark,Space */ BYTE StopBits; /* 0,1,2 = 1, 1.5, 2 */ char XonChar; /* Tx and Rx X-ON character */ char XoffChar; /* Tx and Rx X-OFF character */ char ErrorChar; /* Error replacement char */ char EofChar; /* End of Input character */ char EvtChar; /* Received Event character */ WORD wReserved1; /* Fill for now. */ } alias DCB* LPDCB; private import std.intrinsic; void fBinary(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,0) : btr(&dcb.BITS,0); } void fParity(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,1) : btr(&dcb.BITS,1); } void fOutxCtsFlow(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,2) : btr(&dcb.BITS,2); } void fOutxDsrFlow(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,3) : btr(&dcb.BITS,3); } void fDtrControl(DCB* dcb, uint b) { (bt(&b,0)) ? bts(&dcb.BITS,4) : btr(&dcb.BITS,4); (bt(&b,1)) ? bts(&dcb.BITS,5) : btr(&dcb.BITS,5); } void fDsrSensitivity(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,6) : btr(&dcb.BITS,6); } void fTXContinueOnXoff(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,7) : btr(&dcb.BITS,7); } void fOutX(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,8) : btr(&dcb.BITS,8); } void fInX(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,9) : btr(&dcb.BITS,9); } void fErrorChar(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,10) : btr(&dcb.BITS,10); } void fNull(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,11) : btr(&dcb.BITS,11); } void fRtsControl(DCB* dcb, uint b) { (bt(&b,0)) ? bts(&dcb.BITS,12) : btr(&dcb.BITS,12); (bt(&b,1)) ? bts(&dcb.BITS,13) : btr(&dcb.BITS,13); } void fAbortOnError(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,14) : btr(&dcb.BITS,14); } Note: The commented types shown in the struct were an attempt to use D's bit type. They were originally C bitfields. Regan
Mar 18 2006
next sibling parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Regan Heath wrote:

 I don't know if you'd call this elegant but here is what I did.
 The important thing is using a type which is the correct size
 for the bitfields and then creating some way to interface the
 bits you need.

I had the same idea and solved it this way. Doesn't seem elegant to me though.
 Note: The commented types shown in the struct were an attempt
 to use D's bit type. They were originally C bitfields.

The same crossed my mind, too, but I was unsure how data alignment would interfere with this. -- Wolfgang Draxinger
Mar 18 2006
prev sibling parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:ops6lu2fsx23k2f5 nrage.netwin.co.nz...
 I don't know if you'd call this elegant but here is what I did. The 
 important thing is using a type which is the correct size for the 
 bitfields and then creating some way to interface the bits you need.

 struct DCB {
     DWORD DCBlength;       /* sizeof(DCB)                     */
     DWORD BaudRate;        /* Baudrate at which running       */
     uint BITS;
     //bit fBinary;           /* Binary Mode (skip EOF check)    */
     //bit fParity;           /* Enable parity checking          */

Add two member functions for each: bool fBinary() { return BITS & 1; } bool fBinary(bool b) { BITS |= b; return b; } bool fParity() { return (BITS & 2) != 0; } bool fParity(bool b) { BITS |= b << 1; return b; } and then, because functions work like properties, you can use them as if they were properties.
Mar 18 2006
next sibling parent reply Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Walter Bright wrote:

 Add two member functions for each:
 
     bool fBinary() { return BITS & 1; }
     bool fBinary(bool b) { BITS |= b; return b; }
 
     bool fParity() { return (BITS & 2) != 0; }
     bool fParity(bool b) { BITS |= b << 1; return b; }
 
 and then, because functions work like properties, you can use
 them as if they were properties.

These set functions won't work if b==0, since it's just an OR. What's needed is BITS = (BITS & ~(1<<n)) | b ? 1<<n : 0 ; Anyways that's a cool idea to do it. Just looks a bit strange. -- Wolfgang Draxinger
Mar 18 2006
next sibling parent "Walter Bright" <newshound digitalmars.com> writes:
"Wolfgang Draxinger" <wdraxinger darkstargames.de> wrote in message 
news:dvi5gv$qtg$1 digitaldaemon.com...
 These set functions won't work if b==0, since it's just an OR.
 What's needed is

 BITS = (BITS & ~(1<<n)) | b ? 1<<n : 0 ;

You're right.
Mar 19 2006
prev sibling parent reply Don Clugston <dac nospam.com.au> writes:
Wolfgang Draxinger wrote:
 Walter Bright wrote:
 
 Add two member functions for each:

     bool fBinary() { return BITS & 1; }
     bool fBinary(bool b) { BITS |= b; return b; }

     bool fParity() { return (BITS & 2) != 0; }
     bool fParity(bool b) { BITS |= b << 1; return b; }

 and then, because functions work like properties, you can use
 them as if they were properties.

These set functions won't work if b==0, since it's just an OR. What's needed is BITS = (BITS & ~(1<<n)) | b ? 1<<n : 0 ;

Or you could do BITS = (BITS & ~(1<<n)) | (b << n) ; to remove the need for a branch instruction. Works just as well for multiple bits, eg for b = 0..7: BITS = (BITS & ~(7<<n)) | (b << n) ; I really think it would be worth adding some functions to Phobos for this sort of thing, and it should also duplicate the bit functions from std.intrinsic. (I don't think the fact that bsr is an intrinsic should be exposed in application code. It might not be true on all processors). I have a few functions for it in my MathExtra library. Anyone have any ideas of what it should be called? "std.bitops"? "std.bitoperations"? "std.bitmanip" ? "std.bittwiddle" ?
Mar 20 2006
next sibling parent "Chris Miller" <chris dprogramming.com> writes:
------------z0K1GXvjPKdY9jWKEI5xba
Content-Type: text/plain; format=flowed; delsp=yes; charset=utf-8
Content-Transfer-Encoding: 8bit

On Mon, 20 Mar 2006 04:14:02 -0500, Don Clugston <dac nospam.com.au> wrote:

 Or you could do

 BITS = (BITS & ~(1<<n)) | (b << n) ;
 to remove the need for a branch instruction.
 Works just as well for multiple bits, eg for b = 0..7:
 BITS = (BITS & ~(7<<n)) | (b << n) ;


 I really think it would be worth adding some functions to Phobos for  
 this sort of thing, and it should also duplicate the bit functions from
 std.intrinsic. (I don't think the fact that bsr is an intrinsic should  
 be exposed in application code. It might not be true on all processors).

I've attached code to simplify this. Here's example usage: struct FlagTest { uint bits; mixin .BitFlag!(uint, bits, 0x1) foo; mixin .BitFlag!(uint, bits, 0x2) bar; mixin .BitFlag!(uint, bits, 0x4) baz; } FlagTest ftest; ftest.foo.flag = true; ftest.bar.flag = false; ftest.baz.flag = true; assert(ftest.bits == 0b101); assert(ftest.foo.flag == true); assert(ftest.bar.flag == false); assert(ftest.baz.flag == true); If only the name in the mixin (if matching mixin name) could be promoted to the name of the mixin identifier, it could elegantly become: ftest.foo = true; e.g.: template Foo() { int Foo = 33; } mixin Foo bar; // The int Foo in the template could become named bar and allow: assert(bar == 33); bar = 22; // etc.. Note: the attached code would look a bit nicer if it wasn't for the digitalmars.D.bugs I just posted. ------------z0K1GXvjPKdY9jWKEI5xba Content-Disposition: attachment; filename=bits.d Content-Type: application/octet-stream; name=bits.d Content-Transfer-Encoding: Base64 Ly8gV3JpdHRlbiBieSBDaHJpc3RvcGhlciBFLiBNaWxsZXINCi8vIFBsYWNlZCBp bnRvIHB1YmxpYyBkb21haW4uDQoNCg0KLy8vIE1peGluIGZvciBib29sZWFuIGJp dCBmbGFncy4NCi8vLyBQYXJhbXM6DQovLy8gCUludFR5cGUgPSB0aGUgdHlwZSBv ZiB0aGUgYml0cy4NCi8vLyAJYml0cyA9IHRoZSB2YXJpYWJsZSBvZiB0aGUgX2Jp dHMuDQovLy8gCUJJVF9GTEFHID0gd2hpY2ggX2JpdHMgdG8gdXNlIGluIGJpdHMu DQp0ZW1wbGF0ZSBCaXRGbGFnKEludFR5cGUsIGFsaWFzIGJpdHMsIHVsb25nIEJJ VF9GTEFHKSAvLyBJbnRUeXBlIEJJVF9GTEFHDQp7DQoJYm9vbCBmbGFnKCkNCgl7 DQoJCXJldHVybiAoYml0cyAmIGNhc3QoSW50VHlwZSlCSVRfRkxBRykgPT0gY2Fz dChJbnRUeXBlKUJJVF9GTEFHOw0KCX0NCgkNCgkNCgl2b2lkIGZsYWcoYm9vbCBv cHQpDQoJew0KCQliaXRzID0gb3B0ID8gKGJpdHMgfCBjYXN0KEludFR5cGUpQklU X0ZMQUcpIDogKGJpdHMgJiB+Y2FzdChJbnRUeXBlKUJJVF9GTEFHKTsNCgl9DQp9 DQoNCg0KLy8vIE1peGluIGZvciBtYXNraW5nIGJpdHMuDQovLy8gUGFyYW1zOg0K Ly8vIAlJbnRUeXBlID0gdGhlIHR5cGUgb2YgdGhlIGJpdHMuDQovLy8gCWJpdHMg PSB0aGUgdmFyaWFibGUgb2YgdGhlIF9iaXRzLg0KLy8vIAlCSVRfTUFTSyA9IHdo aWNoIF9iaXRzIHRvIHVzZSBpbiBiaXRzLCBzdWJqZWN0IHRvIHNoaWZ0aW5nIGJ5 IFNISUZUX0JZLg0KLy8vIAlTSElGVF9CWSA9IG51bWJlciBvZiBiaXRzIHRvIHNo aWZ0IHRoZSBtYXNrIHZhbHVlLg0KdGVtcGxhdGUgQml0TWFzayhJbnRUeXBlLCBh bGlhcyBiaXRzLCB1bG9uZyBCSVRfTUFTSywgdWxvbmcgU0hJRlRfQlkgPSAwKSAv LyBJbnRUeXBlIEJJVF9NQVNLLCBJbnRUeXBlIFNISUZUX0JZID0gMA0Kew0KCUlu dFR5cGUgbWFzaygpDQoJew0KCQlyZXR1cm4gKGJpdHMgPj4gY2FzdChJbnRUeXBl KVNISUZUX0JZKSAmIGNhc3QoSW50VHlwZSlCSVRfTUFTSzsNCgl9DQoJDQoJDQoJ dm9pZCBtYXNrKEludFR5cGUgb3B0KQ0KCXsNCgkJYml0cyB8PSAob3B0ICYgY2Fz dChJbnRUeXBlKUJJVF9NQVNLKSA8PCBjYXN0KEludFR5cGUpU0hJRlRfQlk7DQoJ fQ0KfQ0KDQoNCnVuaXR0ZXN0DQp7DQoJc3RydWN0IEZsYWdUZXN0DQoJew0KCQl1 aW50IGJpdHM7DQoJCQ0KCQltaXhpbiAuQml0RmxhZyEodWludCwgYml0cywgMHgx KSBmb287DQoJCW1peGluIC5CaXRGbGFnISh1aW50LCBiaXRzLCAweDIpIGJhcjsN CgkJbWl4aW4gLkJpdEZsYWchKHVpbnQsIGJpdHMsIDB4NCkgYmF6Ow0KCX0NCgkN CglGbGFnVGVzdCBmdGVzdDsNCgkNCglmdGVzdC5mb28uZmxhZyA9IHRydWU7DQoJ ZnRlc3QuYmFyLmZsYWcgPSBmYWxzZTsNCglmdGVzdC5iYXouZmxhZyA9IHRydWU7 DQoJYXNzZXJ0KGZ0ZXN0LmJpdHMgPT0gMGIxMDEpOw0KCWFzc2VydChmdGVzdC5m b28uZmxhZyA9PSB0cnVlKTsNCglhc3NlcnQoZnRlc3QuYmFyLmZsYWcgPT0gZmFs c2UpOw0KCWFzc2VydChmdGVzdC5iYXouZmxhZyA9PSB0cnVlKTsNCgkNCgkNCglz dHJ1Y3QgTWFza1Rlc3QNCgl7DQoJCXVpbnQgYml0czsNCgkJDQoJCW1peGluIC5C aXRNYXNrISh1aW50LCBiaXRzLCAwYjExLCAwKSBmb287DQoJCW1peGluIC5CaXRN YXNrISh1aW50LCBiaXRzLCAwYjEsIDIpIGJhcjsNCgkJbWl4aW4gLkJpdE1hc2sh KHVpbnQsIGJpdHMsIDBiMTEsIDMpIGJhejsNCgl9DQoJDQoJTWFza1Rlc3QgbXRl c3Q7DQoJDQoJbXRlc3QuZm9vLm1hc2sgPSAwYjEwMTsgLy8gQmVjb21lcyAwYjAx IGluIG10ZXN0LmJpdHMNCgltdGVzdC5iYXIubWFzayA9IDBiMTsgLy8gQmVjb21l cyAwYjEwMCBpbiBtdGVzdC5iaXRzDQoJbXRlc3QuYmF6Lm1hc2sgPSAwYjExMTA7 IC8vIEJlY29tZXMgMGIxMDAwMCBpbiBtdGVzdC5iaXRzDQoJYXNzZXJ0KG10ZXN0 LmJpdHMgPT0gMGIxMDEwMSk7DQoJYXNzZXJ0KG10ZXN0LmZvby5tYXNrID09IDBi MDEpOw0KCWFzc2VydChtdGVzdC5iYXIubWFzayA9PSAwYjEpOw0KCWFzc2VydCht dGVzdC5iYXoubWFzayA9PSAwYjEwKTsNCn0NCg0K ------------z0K1GXvjPKdY9jWKEI5xba--
Mar 20 2006
prev sibling parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Don Clugston wrote:

 BITS = (BITS & ~(1<<n)) | (b << n) ;
 to remove the need for a branch instruction.
 Works just as well for multiple bits, eg for b = 0..7:
 BITS = (BITS & ~(7<<n)) | (b << n) ;

Yes, after posting I had the same idea. You see, I'm still thinking a lot in terms of C/C++ where it isn't gauranteed, that a boolean value of "true" also is always the LSB==1. D is just so cool. -- Wolfgang Draxinger
Mar 20 2006
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Walter Bright wrote:

 Add two member functions for each:
 
     bool fBinary() { return BITS & 1; }
     bool fBinary(bool b) { BITS |= b; return b; }
 
     bool fParity() { return (BITS & 2) != 0; }
     bool fParity(bool b) { BITS |= b << 1; return b; }
 
 and then, because functions work like properties, you can use them as if 
 they were properties. 

You'll need to add some casts, though, for GDC: "cannot implicitly convert expression (BITS & 1) of type uint to bit" Or just change it to: return ((BITS & 1) != 0); --anders
Mar 20 2006
prev sibling next sibling parent reply Miles <_______ _______.____> writes:
Wolfgang Draxinger wrote:
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.

I would like to make use of this thread and ask Walter why it was chosen to keep bitfields out of D. Since D has C ABI support, inline assembly, structs and unions, it seems natural to have bitfields also, to complement low-level programming capabilities and simplify interfacing with C code. Recently I needed to parse and construct float numbers, and I had to do all this funny shift stuff in order to translate ieee754.h to a D module. The final result was ugly and also more error-prone compared to C (yes, abusing bit shifts made D code look worse than C). Something I think that D could improve over C in this field is to take bitfields to the next step and make them endian-aware. If you look into ieee754.h, you will find something like: union ieee754_float { float f; struct { #if __BYTE_ORDER == __BIG_ENDIAN unsigned int negative:1; unsigned int exponent:8; unsigned int mantissa:23; #endif #if __BYTE_ORDER == __LITTLE_ENDIAN unsigned int mantissa:23; unsigned int exponent:8; unsigned int negative:1; #endif } ieee; }; This is the raw definition of an IEEE 32-bit float number. Things gets worse for 64-bit and 80-bit floats. The byte-order dependency could be abstracted by a possible D implementation of bitfields by specifying an attribute that defines if bits should be in register order or in memory order. For example: union ieee754_float { float f; pragma(bitfield_order, register, 4) struct { uint negative:1; // most significant bits of a 32-bit register uint exponent:8; uint mantissa:23; // least significant bits of a 32-bit register } ieee; }; Or, for example, when the position in memory is important (communicating with microcontrolled devices via a serial line): union my_float { byte raw[4]; pragma(bitfield_order, memory) struct { uint negative:1; // first bits in memory uint exponent:8; uint mantissa:23; // last bits in memory }; }; Another suggestion, not related to bitfields but related to floats, would be to add these properties (.negative, .mantissa, .exponent and .quiet_nan) to the floating-point types of D. Best regards, Miles.
Mar 20 2006
next sibling parent reply Don Clugston <dac nospam.com.au> writes:
Miles wrote:
 Wolfgang Draxinger wrote:
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.

I would like to make use of this thread and ask Walter why it was chosen to keep bitfields out of D. Since D has C ABI support, inline assembly, structs and unions, it seems natural to have bitfields also, to complement low-level programming capabilities and simplify interfacing with C code. Recently I needed to parse and construct float numbers, and I had to do all this funny shift stuff in order to translate ieee754.h to a D module. The final result was ugly and also more error-prone compared to C (yes, abusing bit shifts made D code look worse than C). Something I think that D could improve over C in this field is to take bitfields to the next step and make them endian-aware. If you look into ieee754.h, you will find something like: union ieee754_float { float f; struct { #if __BYTE_ORDER == __BIG_ENDIAN unsigned int negative:1; unsigned int exponent:8; unsigned int mantissa:23; #endif #if __BYTE_ORDER == __LITTLE_ENDIAN unsigned int mantissa:23; unsigned int exponent:8; unsigned int negative:1; #endif } ieee; }; This is the raw definition of an IEEE 32-bit float number. Things gets worse for 64-bit and 80-bit floats. The byte-order dependency could be abstracted by a possible D implementation of bitfields by specifying an attribute that defines if bits should be in register order or in memory order. For example: union ieee754_float { float f; pragma(bitfield_order, register, 4) struct { uint negative:1; // most significant bits of a 32-bit register uint exponent:8; uint mantissa:23; // least significant bits of a 32-bit register } ieee; }; Or, for example, when the position in memory is important (communicating with microcontrolled devices via a serial line): union my_float { byte raw[4]; pragma(bitfield_order, memory) struct { uint negative:1; // first bits in memory uint exponent:8; uint mantissa:23; // last bits in memory }; }; Another suggestion, not related to bitfields but related to floats, would be to add these properties (.negative, .mantissa, .exponent and .quiet_nan) to the floating-point types of D.

I agree, that would be great, it would allow us to remove the ugly cast(char [])cast(void *).. hacks from std.math. (It would also allow them to be accessed at compile time, which I would find very useful). Maybe only the mantissa and exponent are required, the position of the negative bit is fixed in the exponent, isn't it? (provided that mantissa is an integral type). Immediate interesting application: efficiently converting random integers into a random real in the range 0..1, just by storing the int into the mantissa of a real.
Mar 21 2006
parent reply Miles <_______ _______.____> writes:
Don Clugston wrote:
 Maybe only the mantissa and exponent are required, the position of the
 negative bit is fixed in the exponent, isn't it?
 (provided that mantissa is an integral type).

I don't think so. The negative bit is not part of the exponent, it is really a separate field. Perhaps you are thinking about the signalling NaN indicator, that is the highest bit of the mantissa. But sure would be appropriate, with a little overhead, to have better cooked .exponent and .mantissa properties. Only that the user wouldn't be able to detect a negative zero.
 Immediate interesting application: efficiently converting random
 integers into a random real in the range 0..1, just by storing the int
 into the mantissa of a real.

Sure! Too bad Walter didn't seem to notice my previous post. Best regards.
Mar 21 2006
parent reply Don Clugston <dac nospam.com.au> writes:
Miles wrote:
 Don Clugston wrote:
 Maybe only the mantissa and exponent are required, the position of the
 negative bit is fixed in the exponent, isn't it?
 (provided that mantissa is an integral type).

I don't think so. The negative bit is not part of the exponent, it is really a separate field. Perhaps you are thinking about the signalling NaN indicator, that is the highest bit of the mantissa.

Well, the sign bit is always adjacent to the highest bit of the exponent. I was thinking that the exponent and sign bit could be combined to make a standard-sized D type -- but this only works for 80 bit reals, not float, double, or quadruple.
 But sure would be appropriate, with a little overhead, to have better
 cooked .exponent and .mantissa properties. Only that the user wouldn't
 be able to detect a negative zero.
 
 Immediate interesting application: efficiently converting random
 integers into a random real in the range 0..1, just by storing the int
 into the mantissa of a real.

Sure! Too bad Walter didn't seem to notice my previous post.

It's not a priority, it's something that could be added later without breaking anything -- just syntactic sugar, really. We can try again later <g>.
Mar 22 2006
parent reply Richard Bradley <darthalias yahoo.com> writes:
 It's not a priority, it's something that could be added later without 
 breaking anything -- just syntactic sugar, really. We can try again 
 later <g>.

Curious, Has there been any movement on this issue? Is there a less error-prone way to do this other then manual shifts and masks? I am looking to write an open-source generic library for hardware design, I thought D would make a much better base then C++. I'm a big fan of what I've seen so far. Sorry to say, but D can't be a serious low-level contender without bitfield support. I'm not talking about using it once or twice in a project, but thousands of structures for each project, and each structure having dozens of bitfields. Having to do masks and shifts just seems like a nightmare of bad code. I can't imagine my friends on the firmware side (or any low-level developer) wouldn't agree. Miles' idea of making D somehow order aware would be a huge win for this close-to-the-hardware stuff. Or even non-order aware bitfields at least isn't worse then what's allready out there. Comments, ideas welcome. Richard
Nov 22 2007
next sibling parent reply "dominik" <aha aha.com> writes:
"Richard Bradley" <darthalias yahoo.com> wrote in message 
news:fi5arc$6mu$1 digitalmars.com...
 Comments, ideas welcome.
 Richard

I shall open a bottle of champagne the day we get bitfields in D!
Nov 22 2007
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
D originally had a 'bit' type in place of 'bool' which could be packed
in structs.  However, people didn't think very much of it, and fairly
loudly complained that they wanted a "real" bool type.  Hence, 'bit' is
now aliased to 'bool'.

The other problem was that 'bit' introduced various problems.  For
instance, you can take the address of anything *except* a bitfield,
since addresses only have byte-level granularity.  Bitfields also cannot
have sensible sizeof properties, which could break generic code.

Off the top of my head, maybe it would suffice to give types a special
'bits' slice type.

int some_float;
int negative = some_float.bits[0..1];
int exponent = some_float.bits[1..9];
int mantissa = some_float.bits[9..$];

Basically just tarting up manual shifts and masking.  Just a thought :)

	-- Daniel
Nov 22 2007
parent Regan Heath <regan netmail.co.nz> writes:
Daniel Keep wrote:
 D originally had a 'bit' type in place of 'bool' which could be packed
 in structs.  However, people didn't think very much of it, and fairly
 loudly complained that they wanted a "real" bool type.  Hence, 'bit' is
 now aliased to 'bool'.
 
 The other problem was that 'bit' introduced various problems.  For
 instance, you can take the address of anything *except* a bitfield,
 since addresses only have byte-level granularity.  Bitfields also cannot
 have sensible sizeof properties, which could break generic code.
 
 Off the top of my head, maybe it would suffice to give types a special
 'bits' slice type.
 
 int some_float;
 int negative = some_float.bits[0..1];
 int exponent = some_float.bits[1..9];
 int mantissa = some_float.bits[9..$];
 
 Basically just tarting up manual shifts and masking.  Just a thought :)

You're not the first to think so :) In fact, why not just slice the int directly: int one; int two; one[0..4] = two[4..8]; one[4..8] = two[0..4]; In the meantime, in order to avoid having to re-code all the bit shifts in every struct with bitfields isn't there a mixin which could implement properties for accessing specific bits. Regan
Nov 23 2007
prev sibling next sibling parent reply Richard Bradley <darthalias yahoo.com> writes:
Yes, You are right Janice. We want to access the bitfields in a structure as if
they were the integers.

Like this:

packet.ip.type_of_service.reliability = 1;

(See http://www.protocols.com/pbook/tcpip2.htm#IP for ip protocol.)

I whole heartedly disagree that your workaround is sufficient. It is a bit like
saying C doesn't need native string support because you can just make a
character array, or that you can just make a C++ class to handle strings. (We
all know how well that works, everyone and their grandmother has a different
string class.)

If you look at your example you are replacing the four parameters in the C++
version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield
positions, and 8 masks.) not to mention the shifting and bitwise operations.
And the need of the user to keep track of the “bits” members size. Plus, what
if you wanted a structure 512 bytes long? Your ushort “bits” would have to be
an array, and you would need to be very careful then of how your bitfields
mapped into that array. This is adding a lot of accidental complexity to the
problem. It's is okay if you are doing this once or twice. But I am literally
talking about doing this on thousands of classes per project, on structures
that can be quite large.

The D workaround adds quite a bit of error-prone complexity.

I would also like to suggest that this isn't a rarely-used feature. You run
into this issue anytime you want to read or write to hardware, (or link to a
library that talks to hardware) or access any sort of network protocol stack,
or any type of communication protocol. 

As an outsider looking in that works in these domains, this becomes a glaring
hole in an otherwise wonderful language. One of my fundamental coding tasks is
to talk to hardware, and I don't see that task as at all being rare.

I would agree that the way C++ does bitfields is “inefficient.” I know of no
general-purpose language that does this well. The issues of portability (with
respect to byte ordering) is valid, but kinda beside the point. If you are
hacking hardware, it is by definition tied to the hardware. Just like the
windows library is not portable outside windows. Plus, if we fixed that whole
byte-order thing it would be a big justification to using the language, as the
way C does it is just ugly.

There seems to be a lot of support for a feature along these lines. There has
been a good discussion going on for quite some time. The underlying question is
do we want D to live in this “low level bare metal “ environment? And if so,
what do we (yes, I am including myself) need to do?

~Richard
Nov 23 2007
next sibling parent Robert Fraser <fraserofthenight gmail.com> writes:
Richard Bradley wrote:
 I would also like to suggest that this isn't a rarely-used feature. You run
into this issue anytime you want to read or write to hardware, (or link to a
library that talks to hardware) or access any sort of network protocol stack,
or any type of communication protocol.

It's also used to reduce database throughput and in serialization of large structures to files, among other things. Although I'm not at all a "low-level"/"close-to-the-metal" programmer, I would definitely appreciate such a feature.
Nov 23 2007
prev sibling parent reply Nick B <nick.barbalich gmail.com> writes:
Richard Bradley wrote:
There seems to be a lot of support for a feature along these lines. 
There has been a good discussion going on for quite some time. The 
underlying question is do we want D to live in this “low level bare 
metal “ environment? And if so, what do we (yes, I am including 

Richard As no one else if volunteering at present, I would like to suggest the following. You create one document that covers the following: 1. Detail exactly the Bitfield functionality what you require (you seem to know this better than anyone on this list); 2. Address why the language _needs_ this functionality; 2. Describe the target audience of the said feature(s); 3. Make a pitch as to the _number_ of additional users/projects who might join the "D" community as a result of said features. When completed, ask for feedback from the "D" community, incorporate any feedback if required, and then, finally send it to Walter directly, asking for comment/feedback. Good Luck Nick B
Nov 23 2007
parent "David B. Held" <dheld codelogicconsulting.com> writes:
Nick B wrote:
 [...]
 As no one else if volunteering at present,
 I would like to suggest the following.
 
 You create one document that covers the following:
 
 1. Detail exactly the Bitfield functionality what you require (you
    seem to know this better than anyone on this list);
 2. Address why the language _needs_  this functionality;   
 2. Describe the target audience of the said feature(s);
 3. Make a pitch as to the _number_ of additional users/projects who
    might join the "D" community as a result of said features.
 [...]

Before anyone spends too much time on this, it might be worth relating a tidbit that a little bird told me (yes, it happened just like a Disney movie): an imminent release of D is going to contain bitfield support (though not necessarily looking like any of the suggestions / implementations proposed here). Dave P.S. If you plan to use little birds as target practice for your .22, may I suggest you wait to see if the bird has anything to say, first?
Nov 24 2007
prev sibling next sibling parent Richard Bradley <darthalias yahoo.com> writes:
Yes, You are right Janice. We want to access the bitfields in a structure as if
they were the integers.

Like this:

packet.ip.type_of_service.reliability = 1;

(See http://www.protocols.com/pbook/tcpip2.htm#IP for ip protocol.)

I whole heartedly disagree that your workaround is sufficient. It is a bit like
saying C doesn't need native string support because you can just make a
character array, or that you can just make a C++ class to handle strings. (We
all know how well that works, everyone and their grandmother has a different
string class.)

If you look at your example you are replacing the four parameters in the C++
version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield
positions, and 8 masks.) not to mention the shifting and bitwise operations.
And the need of the user to keep track of the “bits” members size. Plus, what
if you wanted a structure 512 bytes long? Your ushort “bits” would have to be
an array, and you would need to be very careful then of how your bitfields
mapped into that array. This is adding a lot of accidental complexity to the
problem. It's is okay if you are doing this once or twice. But I am literally
talking about doing this on thousands of classes per project, on structures
that can be quite large.

The D workaround adds quite a bit of error-prone complexity.

I would also like to suggest that this isn't a rarely-used feature. You run
into this issue anytime you want to read or write to hardware, (or link to a
library that talks to hardware) or access any sort of network protocol stack,
or any type of communication protocol. 

As an outsider looking in that works in these domains, this becomes a glaring
hole in an otherwise wonderful language. One of my fundamental coding tasks is
to talk to hardware, and I don't see that task as at all being rare.

I would agree that the way C++ does bitfields is “inefficient.” I know of no
general-purpose language that does this well. The issues of portability (with
respect to byte ordering) is valid, but kinda beside the point. If you are
hacking hardware, it is by definition tied to the hardware. Just like the
windows library is not portable outside windows. Plus, if we fixed that whole
byte-order thing it would be a big justification to using the language, as the
way C does it is just ugly.

There seems to be a lot of support for a feature along these lines. There has
been a good discussion going on for quite some time. The underlying question is
do we want D to live in this “low level bare metal “ environment? And if so,
what do we (yes, I am including myself) need to do?

~Richard
Nov 23 2007
prev sibling next sibling parent reply 0ffh <frank frankhirsch.youknow.what.todo.net> writes:
Richard Bradley wrote:
 It's not a priority, it's something that could be added later without
  breaking anything -- just syntactic sugar, really. We can try again 
 later <g>.

Curious, Has there been any movement on this issue? Is there a less error-prone way to do this other then manual shifts and masks? [...] Comments, ideas welcome. Richard

FWIW I am quite sympathetic to the cause of C-like bitfields for D. My take on D in the context of Kelleher's "What languages fix" was always "D: C++ is a syntactic mess, and built-in GC is a win.". OTOH, the accessor templates from the D.learn thread look very nice. The question is probably: Do we really need native bitfield support, or is D mighty enough that a sufficient solution can be implemented without resorting to adding to the language itself? I think the answer may in part depend on the result of the planned "GCC bitfields" vs. "GDC templates" shootout as planned on D.learn by bearophile. Hi bearophile, any results yet? :) regards, frank
Nov 23 2007
parent bearophile <bearophileHUGS lycos.com> writes:
0ffh:

 OTOH, the accessor templates from the D.learn thread look very nice.

I think they aren't good enough yet for practical usage, they do too much upcasting (to 64 bit numbers, in that thread I have shown it that using 32 bit makes those bitfields faster), so they require improvement. And they need more testing.
 I think the answer may in part depend on the result of the planned
 "GCC bitfields" vs. "GDC templates" shootout as planned on D.learn
 by bearophile.
 Hi bearophile, any results yet? :)

Hello, but sadly os far I haven't succed yet to install GDC (I'll try again), despite I already have MinGW installed. So I have tried with just the MinGW, the following: #include <stdio.h> #define N 25 typedef union { unsigned int i; struct { unsigned int i; unsigned int sign(void) { return (i >> 0x0) & 0x1; } void sign(unsigned int val) { i = (i & 0xfffffffffffffffeLL) | ((val & 0x1) << 0x0); } unsigned int biasedexponent(void) { return (i >> 0x1) & 0xff; } void biasedexponent(unsigned int val) { i = (i & 0xfffffffffffffe01LL) | ((val & 0xff) << 0x1); } unsigned int significand(void) { return (i >> 0x9) & 0x7fffff; } void significand(unsigned int val) { i = (i & 0x1ff) | ((val & 0x7fffff) << 0x9); } } bits; } INTORFLOAT; int main(void) { int i, j; unsigned long long tot = 0; INTORFLOAT f; unsigned int v[N] = {1628676761, 1620574103, 1237153253, 1098880307, 87513741, 13181925, 14686126, 7429435, 16286706, 6474381, 4879794, 7734725, 3745958, 13353858, 4236193, 7587, 4309, 28846, 7313, 14516, 126, 143, 171, 221, 156}; for(j = 0; j < 4000000; j++) for(i = 0; i < N; i++) { f.i = v[i]; f.bits.biasedexponent(f.bits.biasedexponent() / 2); tot += f.bits.biasedexponent() + f.bits.sign(); } printf("%d\n", tot); } It runs even a bit faster than the normal C bitfields ;-) So they may run fast on GDC too. Bye, bearophile
Nov 24 2007
prev sibling parent Nick B <nick.barbalich gmail.com> writes:
Richard Bradley wrote:
 It's not a priority, it's something that could be added later without 
 breaking anything -- just syntactic sugar, really. We can try again 
 later <g>.

Curious, Has there been any movement on this issue? Is there a less error-prone way to do this other then manual shifts and masks? I am looking to write an open-source generic library for hardware design, I thought D would make a much better base then C++. I'm a big fan of what I've seen so far. Sorry to say, but D can't be a serious low-level contender without bitfield support. I'm not talking about using it once or twice in a project, but thousands of structures for each project, and each structure having dozens of bitfields. Having to do masks and shifts just seems like a nightmare of bad code. I can't imagine my friends on the firmware side (or any low-level developer) wouldn't agree. Miles' idea of making D somehow order aware would be a huge win for this close-to-the-hardware stuff. Or even non-order aware bitfields at least isn't worse then what's allready out there. Comments, ideas welcome. Richard

Does this help at all ? http://www.digitalmars.com/d/phobos/std_bitmanip.html Nick B
Dec 03 2007
prev sibling next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
I don't like the idea of referring to e.g. n[3..4]. The way I see it,
that's just as error prone as bitshifting. No, what you really want to
do is separate interface from implementation. The interface is, you
want to be able to manipulate bitfields as though they were integers.
The implementation is how you want to store them in a struct. The
whole business of using slice operators pulls implementation details
into the algorithm, where they don't belong.

Howver, it seems to me that we already /have/ bitfields in D. Or at
least, the ability to make them. In the example below, the D usage is
identical to the C usage. All that's different is the way they're
declared. Not only that, it seems to me that the D way offers far more
precision and flexibility. Compare...

C/C++:

    /* define the structure */
    struct S
    {
        unsigned int a : 3;
        unsigned int b : 6;
        unsigned int c : 2;
        unsigned int d : 5;
    }

    /* declare an instance */
    S s;

    /* use the bits */
    s.b = 4;
    s.c = 3;
    s.d = s.b + 1;
    // etc....

D

    /* define the structure */
    struct S
    {
        ushort bits;
	
        uint a() { return (bits >> 13) & 0x07;
        uint b() { return (bits >>  7) & 0x3F;
        uint c() { return (bits >>  5) & 0x03;
        uint d() { return (bits >>  0) & 0x1F;

        void c(uint n) { bits &= 0x1FFF; bits |= (n << 13); }
        void b(uint n) { bits &= 0xE07F; bits |= (n <<  7); }
        void c(uint n) { bits &= 0xFF9F; bits |= (n <<  5); }
        void d(uint n) { bits &= 0xFFE0; bits |= (n <<  0); }
    }

    /* declare an instance */
    S s;

    /* use the bits */
    s.b = 4;
    s.c = 3;
    s.d = s.b + 1;
    // etc....

I'm sure some smart person can come up with a mixin to automate even this!
Nov 23 2007
prev sibling next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 11/23/07, Richard Bradley <darthalias yahoo.com> wrote:
 I whole heartedly disagree that your workaround is sufficient.

I did not claim that the workaround was sufficient, I only claimed that the workaround exists. You can't disagree with a claim I didn't make.
 If you look at your example you are replacing the four parameters in the C++
version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield
positions, and 8 masks.) not to mention the shifting and bitwise operations.
And the need of the user to keep track of the "bits" members size. Plus, what
if you wanted a structure 512 bytes long?

Agreed. That would make it tricky to write getter and setter functions for each bitfield. I've certainly no objection to the compiler making things easier for the programmer. The shifting and bitwise operations will happen behind the scenes though, even if you're not aware of it. I think the way to do it to have the compiler automagically write getter and setter functions for you. That way, the following would be allowed: n = s.a; // read a bitfield s.a = n; // write a bitfield but things like s.a+=, &s.a, passing bitfields by reference, and so on, would still be disallowed. That would force the programmer to do all the calculations in "real" ints, using the bitfields only for storage.
Nov 23 2007
prev sibling next sibling parent Martin d Anjou <point14 magma.ca> writes:
 Nick B wrote:
  [...]
  As no one else if volunteering at present,
  I would like to suggest the following.

  You create one document that covers the following:

  1. Detail exactly the Bitfield functionality what you require (you
     seem to know this better than anyone on this list);
  2. Address why the language _needs_  this functionality;   2. Describe the
  target audience of the said feature(s);
  3. Make a pitch as to the _number_ of additional users/projects who
     might join the "D" community as a result of said features.
  [...]


I'm interested in this too. // Pseudo-code D-like, do not try this at home bitvector!(13) bv; bv[3] = 1; assert(bv == 0b0000000001000); bv[10:8] = 3; assert(bv == 0b0011100001000); alias bv[6:4] bv.field1; bv.field1 = 2; assert(bv == 0b0011100101000); assert(bv && true); bv = 0; assert(bv && false); bv++; assert(bv == 1); bv = 0b1111111111111; bv++; assert(bv == 0); bitvector!(122031) bv2; // Should also work. Limit is memory. Martin
Nov 27 2007
prev sibling next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On Nov 23, 2007 5:50 PM, Janice Caron <caron800 googlemail.com> wrote:
 I think the way to do it to have the compiler automagically write
 getter and setter functions for you. That way, the following would be
 allowed:

     n = s.a; // read a bitfield
     s.a = n; // write a bitfield

 but things like s.a+=, &s.a, passing bitfields by reference, and so
 on, would still be disallowed. That would force the programmer to do
 all the calculations in "real" ints, using the bitfields only for
 storage.

Hey, someone's been listening to me! :-) Check out std.bitmanip in D2.008. We have bitfields.
Nov 29 2007
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
Janice Caron, el 29 de noviembre a las 08:45 me escribiste:
 On Nov 23, 2007 5:50 PM, Janice Caron <caron800 googlemail.com> wrote:
 I think the way to do it to have the compiler automagically write
 getter and setter functions for you. That way, the following would be
 allowed:

     n = s.a; // read a bitfield
     s.a = n; // write a bitfield

 but things like s.a+=, &s.a, passing bitfields by reference, and so
 on, would still be disallowed. That would force the programmer to do
 all the calculations in "real" ints, using the bitfields only for
 storage.

Hey, someone's been listening to me! :-) Check out std.bitmanip in D2.008. We have bitfields.

*Ugly* bitfields, but bitfield at last (language support would be much nicer... :S) -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ ---------------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------------- Charles Chaplin once won 3rd price in a Charles Chaplin look-a-like contest
Nov 29 2007
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Wolfgang Draxinger wrote:
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.
 
 Has somebody an idea how I best interface with it elegantly?

Yes, http://www.digitalmars.com/d/phobos/std_bitmanip.html
Nov 28 2007
parent "Craig Black" <craigblack2 cox.net> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:fiksel$28qu$2 digitalmars.com...
 Wolfgang Draxinger wrote:
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.

 Has somebody an idea how I best interface with it elegantly?

Yes, http://www.digitalmars.com/d/phobos/std_bitmanip.html

D has bit fields! Sweet!
Nov 29 2007