www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - interfacing C bitfield structs

reply Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
I'm currently writing D includes for several game and graphics
related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
Freetype2).

Now with SDL I've encountered a problem: SDL makes use of serval
bitfields and I've no idea, how to interface them to D.

Has somebody an idea how I best interface with it elegantly?

Greetings

Wolfgang Draxinger
Mar 17 2006
next sibling parent reply Kyle Furlong <kylefurlong gmail.com> writes:
Wolfgang Draxinger wrote:

 I'm currently writing D includes for several game and graphics
 related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
 Freetype2).
 
 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.
 
 Has somebody an idea how I best interface with it elegantly?
 
 Greetings
 
 Wolfgang Draxinger
 


I would just like to make you aware of the fact that all these have been
interfaced to D already. You can find many of these 
bindings (and more) on www.dsource.org under a few projects: bindings, terra,
and derelict.
Mar 17 2006
parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Kyle Furlong wrote:


 I would just like to make you aware of the fact that all these
 have been interfaced to D already. You can find many of these
 bindings (and more) on www.dsource.org under a few projects:
 bindings, terra, and derelict.


I know, but they failed to work for me, last time I checked.
Don't know how it's unter Windows but I need it for Linux.
Additionally I made bindings to GLEW so that you can use OpenGL
extensions with a breeze; due to some heavy use of macros I
haven't yet added interfaces to the GLEW multi context function.
This requires for every extension function either a template
wrapper or intermediary function built around, that calls the
function pointer from a context struct.

-- 
Wolfgang Draxinger
Mar 17 2006
prev sibling next sibling parent reply Lucas Goss <lgoss007 gmail.com> writes:
Wolfgang Draxinger wrote:

 I'm currently writing D includes for several game and graphics
 related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
 Freetype2).


Me too :)


 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.
 
 Has somebody an idea how I best interface with it elegantly?


Not sure what problem you are having.

Kyle Furlong wrote:

 ...You can find many of these
 bindings (and more) on www.dsource.org under a few projects:
 bindings, terra, and derelict.





 I know, but they failed to work for me, last time I checked.


Well I know derelict works on linux. But if you don't want to take that 
approach (the dynamic loading), Anders has sdl bindings here:
http://www.algonet.se/~afb/d/

Which you can use or might help point you in the right direction.

Lucas
Mar 17 2006
parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Lucas Goss wrote:


 But if you don't want to 
 take that approach (the dynamic loading), Anders has sdl
 bindings here: http://www.algonet.se/~afb/d/


Nice, eventually I even use his bindings.

Eventually I put my GLEW bindings over his.

My GLEW bindings are up and running, without multicontext support
though. I still must write a small script the generates the D
template functions for this.

If compiled with multitexture supprort GLEW expects the user to
supply a function GLEWContext *glewGetContext(). This is used in
macros like

#define glActiveTextureUnitARB
GLEW_GET_FUN(__glewActiveTextureUnitARB)

And GLEW_GET_FUN is a macro

#ifdef GLEW_MX
#define GLEW_GET_FUN(x) glewGetContext()->x
#else
#define GLEW_GET_FUN(x) x
#endif

So far the only way I know to bring this to D is to do something
like this

version(GLEW_MX)
{
template glActiveTextureUnitARB(unit){ GLvoid
glActiveTextureUnitARB(unit) { return
__glewActiveTextureUnitARB(unit); } }
}
else
{
alias __glewActiveTextureUnitARB glActiveTextureUnitARB;
}

I'm using templates since using normal functions would eventually
be compiled not as inline, which is preferred.

This got me to an idea for a new D feature, that would allow to
use aliases as a replacement for macros, if I could have written

template glActiveTextureUnitARB(...) function { return
glewGetContext().__glewActiveTextureUnitARB(...) ; }

It would be much more readable. I mean, that by adding the
keyword 'function' after the template definition you don't have
to repeat the template name to make it a function template and
the ellipsis '...' as a placeholder for an arbitrary long
parameter list, eventually supporting some scheme to extract a
certain parameter.

But so I have to extract the parameter list and the return type
from the extension functions definition and generate template
worms for it.

-- 
Wolfgang Draxinger
Mar 18 2006
prev sibling next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 17 Mar 2006 23:50:39 +0100, Wolfgang Draxinger  
<wdraxinger darkstargames.de> wrote:

 I'm currently writing D includes for several game and graphics
 related libraries (OpenGL, OpenAL, DevIL, SDL, cairo,
 Freetype2).

 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.

 Has somebody an idea how I best interface with it elegantly?


I don't know if you'd call this elegant but here is what I did. The  
important thing is using a type which is the correct size for the  
bitfields and then creating some way to interface the bits you need.

struct DCB {
     DWORD DCBlength;       /* sizeof(DCB)                     */
     DWORD BaudRate;        /* Baudrate at which running       */
     uint BITS;
     //bit fBinary;           /* Binary Mode (skip EOF check)    */
     //bit fParity;           /* Enable parity checking          */
     //bit fOutxCtsFlow;      /* CTS handshaking on output       */
     //bit fOutxDsrFlow;      /* DSR handshaking on output       */
     //bit[2] fDtrControl;    /* DTR Flow control                */
     //bit fDsrSensitivity;   /* DSR Sensitivity              */
     //bit fTXContinueOnXoff; /* Continue TX when Xoff sent */
     //bit fOutX;             /* Enable output X-ON/X-OFF        */
     //bit fInX;              /* Enable input X-ON/X-OFF         */
     //bit fErrorChar;        /* Enable Err Replacement          */
     //bit fNull;             /* Enable Null stripping           */
     //bit[2] fRtsControl;    /* Rts Flow control                */
     //bit fAbortOnError;     /* Abort all reads and writes on Error */
     //bit[17] fDummy2;       /* Reserved                        */
     WORD wReserved;        /* Not currently used              */
     WORD XonLim;           /* Transmit X-ON threshold         */
     WORD XoffLim;          /* Transmit X-OFF threshold        */
     BYTE ByteSize;         /* Number of bits/byte, 4-8        */
     BYTE Parity;           /* 0-4=None,Odd,Even,Mark,Space    */
     BYTE StopBits;         /* 0,1,2 = 1, 1.5, 2               */
     char XonChar;          /* Tx and Rx X-ON character        */
     char XoffChar;         /* Tx and Rx X-OFF character       */
     char ErrorChar;        /* Error replacement char          */
     char EofChar;          /* End of Input character          */
     char EvtChar;          /* Received Event character        */
     WORD wReserved1;       /* Fill for now.                   */
}
alias DCB* LPDCB;

private import std.intrinsic;

void fBinary(DCB* dcb, uint b)           { (b) ? bts(&dcb.BITS,0) :  
btr(&dcb.BITS,0); }	
void fParity(DCB* dcb, uint b)           { (b) ? bts(&dcb.BITS,1) :  
btr(&dcb.BITS,1); }
void fOutxCtsFlow(DCB* dcb, uint b)      { (b) ? bts(&dcb.BITS,2) :  
btr(&dcb.BITS,2); }
void fOutxDsrFlow(DCB* dcb, uint b)      { (b) ? bts(&dcb.BITS,3) :  
btr(&dcb.BITS,3); }
void fDtrControl(DCB* dcb, uint b)       { (bt(&b,0)) ? bts(&dcb.BITS,4) :  
btr(&dcb.BITS,4); (bt(&b,1)) ? bts(&dcb.BITS,5) : btr(&dcb.BITS,5); }
void fDsrSensitivity(DCB* dcb, uint b)   { (b) ? bts(&dcb.BITS,6) :  
btr(&dcb.BITS,6); }
void fTXContinueOnXoff(DCB* dcb, uint b) { (b) ? bts(&dcb.BITS,7) :  
btr(&dcb.BITS,7); }
void fOutX(DCB* dcb, uint b)             { (b) ? bts(&dcb.BITS,8) :  
btr(&dcb.BITS,8); }
void fInX(DCB* dcb, uint b)              { (b) ? bts(&dcb.BITS,9) :  
btr(&dcb.BITS,9); }
void fErrorChar(DCB* dcb, uint b)        { (b) ? bts(&dcb.BITS,10) :  
btr(&dcb.BITS,10); }
void fNull(DCB* dcb, uint b)             { (b) ? bts(&dcb.BITS,11) :  
btr(&dcb.BITS,11); }
void fRtsControl(DCB* dcb, uint b)       { (bt(&b,0)) ? bts(&dcb.BITS,12)  
: btr(&dcb.BITS,12); (bt(&b,1)) ? bts(&dcb.BITS,13) : btr(&dcb.BITS,13); }
void fAbortOnError(DCB* dcb, uint b)     { (b) ? bts(&dcb.BITS,14) :  
btr(&dcb.BITS,14); }

Note: The commented types shown in the struct were an attempt to use D's  
bit type. They were originally C bitfields.

Regan
Mar 18 2006
next sibling parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Regan Heath wrote:


 I don't know if you'd call this elegant but here is what I did.
 The important thing is using a type which is the correct size
 for the bitfields and then creating some way to interface the
 bits you need.


I had the same idea and solved it this way. Doesn't seem elegant
to me though.


 Note: The commented types shown in the struct were an attempt
 to use D's bit type. They were originally C bitfields.


The same crossed my mind, too, but I was unsure how data
alignment would interfere with this.

-- 
Wolfgang Draxinger
Mar 18 2006
prev sibling parent reply "Walter Bright" <newshound digitalmars.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:ops6lu2fsx23k2f5 nrage.netwin.co.nz...

 I don't know if you'd call this elegant but here is what I did. The 
 important thing is using a type which is the correct size for the 
 bitfields and then creating some way to interface the bits you need.

 struct DCB {
     DWORD DCBlength;       /* sizeof(DCB)                     */
     DWORD BaudRate;        /* Baudrate at which running       */
     uint BITS;
     //bit fBinary;           /* Binary Mode (skip EOF check)    */
     //bit fParity;           /* Enable parity checking          */


Add two member functions for each:

    bool fBinary() { return BITS & 1; }
    bool fBinary(bool b) { BITS |= b; return b; }

    bool fParity() { return (BITS & 2) != 0; }
    bool fParity(bool b) { BITS |= b << 1; return b; }

and then, because functions work like properties, you can use them as if 
they were properties.
Mar 18 2006
next sibling parent reply Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Walter Bright wrote:


 Add two member functions for each:
 
     bool fBinary() { return BITS & 1; }
     bool fBinary(bool b) { BITS |= b; return b; }
 
     bool fParity() { return (BITS & 2) != 0; }
     bool fParity(bool b) { BITS |= b << 1; return b; }
 
 and then, because functions work like properties, you can use
 them as if they were properties.


These set functions won't work if b==0, since it's just an OR.
What's needed is 

BITS = (BITS & ~(1<<n)) | b ? 1<<n : 0 ;

Anyways that's a cool idea to do it. Just looks a bit strange.

-- 
Wolfgang Draxinger
Mar 18 2006
next sibling parent "Walter Bright" <newshound digitalmars.com> writes:
"Wolfgang Draxinger" <wdraxinger darkstargames.de> wrote in message 
news:dvi5gv$qtg$1 digitaldaemon.com...

 These set functions won't work if b==0, since it's just an OR.
 What's needed is

 BITS = (BITS & ~(1<<n)) | b ? 1<<n : 0 ;


You're right.
Mar 19 2006
prev sibling parent reply Don Clugston <dac nospam.com.au> writes:
Wolfgang Draxinger wrote:

 Walter Bright wrote:
 

 Add two member functions for each:

     bool fBinary() { return BITS & 1; }
     bool fBinary(bool b) { BITS |= b; return b; }

     bool fParity() { return (BITS & 2) != 0; }
     bool fParity(bool b) { BITS |= b << 1; return b; }

 and then, because functions work like properties, you can use
 them as if they were properties.


 
 These set functions won't work if b==0, since it's just an OR.
 What's needed is 
 
 BITS = (BITS & ~(1<<n)) | b ? 1<<n : 0 ;


Or you could do

BITS = (BITS & ~(1<<n)) | (b << n) ;
to remove the need for a branch instruction.
Works just as well for multiple bits, eg for b = 0..7:
BITS = (BITS & ~(7<<n)) | (b << n) ;


I really think it would be worth adding some functions to Phobos for 
this sort of thing, and it should also duplicate the bit functions from
std.intrinsic. (I don't think the fact that bsr is an intrinsic should 
be exposed in application code. It might not be true on all processors).

I have a few functions for it in my MathExtra library.
Anyone have any ideas of what it should be called?

"std.bitops"?
"std.bitoperations"?
"std.bitmanip" ?
"std.bittwiddle" ?
Mar 20 2006
next sibling parent "Chris Miller" <chris dprogramming.com> writes:
On Mon, 20 Mar 2006 04:14:02 -0500, Don Clugston <dac nospam.com.au> wrote:


 Or you could do

 BITS = (BITS & ~(1<<n)) | (b << n) ;
 to remove the need for a branch instruction.
 Works just as well for multiple bits, eg for b = 0..7:
 BITS = (BITS & ~(7<<n)) | (b << n) ;


 I really think it would be worth adding some functions to Phobos for  
 this sort of thing, and it should also duplicate the bit functions from
 std.intrinsic. (I don't think the fact that bsr is an intrinsic should  
 be exposed in application code. It might not be true on all processors).


I've attached code to simplify this. Here's example usage:

struct FlagTest
{
    uint bits;

    mixin .BitFlag!(uint, bits, 0x1) foo;
    mixin .BitFlag!(uint, bits, 0x2) bar;
    mixin .BitFlag!(uint, bits, 0x4) baz;
}

FlagTest ftest;

ftest.foo.flag = true;
ftest.bar.flag = false;
ftest.baz.flag = true;

assert(ftest.bits == 0b101);
assert(ftest.foo.flag == true);
assert(ftest.bar.flag == false);
assert(ftest.baz.flag == true);


If only the name in the mixin (if matching mixin name) could be promoted  
to the name of the mixin identifier, it could elegantly become: ftest.foo  
= true; e.g.:

template Foo() { int Foo = 33; }
mixin Foo bar; // The int Foo in the template could become named bar and  
allow:
assert(bar == 33);
bar = 22; // etc..


Note: the attached code would look a bit nicer if it wasn't for the  
digitalmars.D.bugs I just posted.
Mar 20 2006
prev sibling parent Wolfgang Draxinger <wdraxinger darkstargames.de> writes:
Don Clugston wrote:


 BITS = (BITS & ~(1<<n)) | (b << n) ;
 to remove the need for a branch instruction.
 Works just as well for multiple bits, eg for b = 0..7:
 BITS = (BITS & ~(7<<n)) | (b << n) ;


Yes, after posting I had the same idea. You see, I'm still
thinking a lot in terms of C/C++ where it isn't gauranteed, that
a boolean value of "true" also is always the LSB==1.

D is just so cool.

-- 
Wolfgang Draxinger
Mar 20 2006
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Walter Bright wrote:


 Add two member functions for each:
 
     bool fBinary() { return BITS & 1; }
     bool fBinary(bool b) { BITS |= b; return b; }
 
     bool fParity() { return (BITS & 2) != 0; }
     bool fParity(bool b) { BITS |= b << 1; return b; }
 
 and then, because functions work like properties, you can use them as if 
 they were properties. 


You'll need to add some casts, though, for GDC:

"cannot implicitly convert expression (BITS & 1) of type uint to bit"

Or just change it to: return ((BITS & 1) != 0);

--anders
Mar 20 2006
prev sibling next sibling parent reply Miles <_______ _______.____> writes:
Wolfgang Draxinger wrote:

 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.


I would like to make use of this thread and ask Walter why it was chosen
to keep bitfields out of D. Since D has C ABI support, inline assembly,
structs and unions, it seems natural to have bitfields also, to
complement low-level programming capabilities and simplify interfacing
with C code.

Recently I needed to parse and construct float numbers, and I had to do
all this funny shift stuff in order to translate ieee754.h to a D
module. The final result was ugly and also more error-prone compared to
C (yes, abusing bit shifts made D code look worse than C).

Something I think that D could improve over C in this field is to take
bitfields to the next step and make them endian-aware. If you look into
ieee754.h, you will find something like:

union ieee754_float {
    float f;

    struct {
#if	__BYTE_ORDER == __BIG_ENDIAN
	unsigned int negative:1;
	unsigned int exponent:8;
	unsigned int mantissa:23;
#endif
#if	__BYTE_ORDER == __LITTLE_ENDIAN
	unsigned int mantissa:23;
	unsigned int exponent:8;
	unsigned int negative:1;
#endif
      } ieee;
};

This is the raw definition of an IEEE 32-bit float number. Things gets
worse for 64-bit and 80-bit floats. The byte-order dependency could be
abstracted by a possible D implementation of bitfields by specifying an
attribute that defines if bits should be in register order or in memory
order. For example:

union ieee754_float {
    float f;

    pragma(bitfield_order, register, 4)
    struct {
        uint negative:1; // most significant bits of a 32-bit register
        uint exponent:8;
        uint mantissa:23; // least significant bits of a 32-bit register
    } ieee;
};

Or, for example, when the position in memory is important (communicating
with microcontrolled devices via a serial line):

union my_float {
    byte raw[4];

    pragma(bitfield_order, memory)
    struct {
        uint negative:1; // first bits in memory
        uint exponent:8;
        uint mantissa:23; // last bits in memory
    };
};

Another suggestion, not related to bitfields but related to floats,
would be to add these properties (.negative, .mantissa, .exponent and
.quiet_nan) to the floating-point types of D.

Best regards,
Miles.
Mar 20 2006
next sibling parent reply Don Clugston <dac nospam.com.au> writes:
Miles wrote:

 Wolfgang Draxinger wrote:

 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.


 
 I would like to make use of this thread and ask Walter why it was chosen
 to keep bitfields out of D. Since D has C ABI support, inline assembly,
 structs and unions, it seems natural to have bitfields also, to
 complement low-level programming capabilities and simplify interfacing
 with C code.
 
 Recently I needed to parse and construct float numbers, and I had to do
 all this funny shift stuff in order to translate ieee754.h to a D
 module. The final result was ugly and also more error-prone compared to
 C (yes, abusing bit shifts made D code look worse than C).
 
 Something I think that D could improve over C in this field is to take
 bitfields to the next step and make them endian-aware. If you look into
 ieee754.h, you will find something like:
 
 union ieee754_float {
     float f;
 
     struct {
 #if	__BYTE_ORDER == __BIG_ENDIAN
 	unsigned int negative:1;
 	unsigned int exponent:8;
 	unsigned int mantissa:23;
 #endif
 #if	__BYTE_ORDER == __LITTLE_ENDIAN
 	unsigned int mantissa:23;
 	unsigned int exponent:8;
 	unsigned int negative:1;
 #endif
       } ieee;
 };
 
 This is the raw definition of an IEEE 32-bit float number. Things gets
 worse for 64-bit and 80-bit floats. The byte-order dependency could be
 abstracted by a possible D implementation of bitfields by specifying an
 attribute that defines if bits should be in register order or in memory
 order. For example:
 
 union ieee754_float {
     float f;
 
     pragma(bitfield_order, register, 4)
     struct {
         uint negative:1; // most significant bits of a 32-bit register
         uint exponent:8;
         uint mantissa:23; // least significant bits of a 32-bit register
     } ieee;
 };
 
 Or, for example, when the position in memory is important (communicating
 with microcontrolled devices via a serial line):
 
 union my_float {
     byte raw[4];
 
     pragma(bitfield_order, memory)
     struct {
         uint negative:1; // first bits in memory
         uint exponent:8;
         uint mantissa:23; // last bits in memory
     };
 };
 
 Another suggestion, not related to bitfields but related to floats,
 would be to add these properties (.negative, .mantissa, .exponent and
 .quiet_nan) to the floating-point types of D.


I agree, that would be great, it would allow us to remove the ugly 
cast(char [])cast(void *).. hacks from std.math.
(It would also allow them to be accessed at compile time, which I would 
find very useful).
Maybe only the mantissa and exponent are required, the position of the 
negative bit is fixed in the exponent, isn't it?
(provided that mantissa is an integral type).

Immediate interesting application: efficiently converting random 
integers into a random real in the range 0..1, just by storing the int 
into the mantissa of a real.
Mar 21 2006
parent reply Miles <_______ _______.____> writes:
Don Clugston wrote:

 Maybe only the mantissa and exponent are required, the position of the
 negative bit is fixed in the exponent, isn't it?
 (provided that mantissa is an integral type).


I don't think so. The negative bit is not part of the exponent, it is
really a separate field. Perhaps you are thinking about the signalling
NaN indicator, that is the highest bit of the mantissa.

But sure would be appropriate, with a little overhead, to have better
cooked .exponent and .mantissa properties. Only that the user wouldn't
be able to detect a negative zero.


 Immediate interesting application: efficiently converting random
 integers into a random real in the range 0..1, just by storing the int
 into the mantissa of a real.


Sure!

Too bad Walter didn't seem to notice my previous post.

Best regards.
Mar 21 2006
parent reply Don Clugston <dac nospam.com.au> writes:
Miles wrote:

 Don Clugston wrote:

 Maybe only the mantissa and exponent are required, the position of the
 negative bit is fixed in the exponent, isn't it?
 (provided that mantissa is an integral type).


 
 I don't think so. The negative bit is not part of the exponent, it is
 really a separate field. Perhaps you are thinking about the signalling
 NaN indicator, that is the highest bit of the mantissa.


Well, the sign bit is always adjacent to the highest bit of the 
exponent. I was thinking that the exponent and sign bit could be 
combined to make a standard-sized D type -- but this only works for 80 
bit reals, not float, double, or quadruple.


 But sure would be appropriate, with a little overhead, to have better
 cooked .exponent and .mantissa properties. Only that the user wouldn't
 be able to detect a negative zero.
 

 Immediate interesting application: efficiently converting random
 integers into a random real in the range 0..1, just by storing the int
 into the mantissa of a real.


 
 Sure!
 
 Too bad Walter didn't seem to notice my previous post.


It's not a priority, it's something that could be added later without 
breaking anything -- just syntactic sugar, really. We can try again 
later <g>.
Mar 22 2006
parent reply Richard Bradley <darthalias yahoo.com> writes:
 It's not a priority, it's something that could be added later without 
 breaking anything -- just syntactic sugar, really. We can try again 
 later <g>.


Curious, Has there been any movement on this issue? Is there a less error-prone
way to do this other then manual shifts and masks?

I am looking to write an open-source generic library for hardware design, I
thought D would make a much better base then C++. I'm a big fan of what I've
seen so far. Sorry to say, but D can't be a serious low-level contender without
bitfield support.

I'm not talking about using it once or twice in a project, but thousands of
structures for each project, and each structure having dozens of bitfields.
Having to do masks and shifts just seems like a nightmare of bad code. 

I can't imagine my friends on the firmware side (or any low-level developer)
wouldn't agree. 

Miles' idea of making D somehow order aware would be a huge win for this
close-to-the-hardware stuff. Or even non-order aware bitfields at least isn't
worse then what's allready out there.

Comments, ideas welcome.
Richard
Nov 22 2007
next sibling parent reply "dominik" <aha aha.com> writes:
"Richard Bradley" <darthalias yahoo.com> wrote in message 
news:fi5arc$6mu$1 digitalmars.com...

 Comments, ideas welcome.
 Richard


I shall open a bottle of champagne the day we get bitfields in D!
Nov 22 2007
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
D originally had a 'bit' type in place of 'bool' which could be packed
in structs.  However, people didn't think very much of it, and fairly
loudly complained that they wanted a "real" bool type.  Hence, 'bit' is
now aliased to 'bool'.

The other problem was that 'bit' introduced various problems.  For
instance, you can take the address of anything *except* a bitfield,
since addresses only have byte-level granularity.  Bitfields also cannot
have sensible sizeof properties, which could break generic code.

Off the top of my head, maybe it would suffice to give types a special
'bits' slice type.

int some_float;
int negative = some_float.bits[0..1];
int exponent = some_float.bits[1..9];
int mantissa = some_float.bits[9..$];

Basically just tarting up manual shifts and masking.  Just a thought :)

	-- Daniel
Nov 22 2007
parent Regan Heath <regan netmail.co.nz> writes:
Daniel Keep wrote:

 D originally had a 'bit' type in place of 'bool' which could be packed
 in structs.  However, people didn't think very much of it, and fairly
 loudly complained that they wanted a "real" bool type.  Hence, 'bit' is
 now aliased to 'bool'.
 
 The other problem was that 'bit' introduced various problems.  For
 instance, you can take the address of anything *except* a bitfield,
 since addresses only have byte-level granularity.  Bitfields also cannot
 have sensible sizeof properties, which could break generic code.
 
 Off the top of my head, maybe it would suffice to give types a special
 'bits' slice type.
 
 int some_float;
 int negative = some_float.bits[0..1];
 int exponent = some_float.bits[1..9];
 int mantissa = some_float.bits[9..$];
 
 Basically just tarting up manual shifts and masking.  Just a thought :)


You're not the first to think so :)

In fact, why not just slice the int directly:

int one;
int two;

one[0..4] = two[4..8];
one[4..8] = two[0..4];

In the meantime, in order to avoid having to re-code all the bit shifts 
in every struct with bitfields isn't there a mixin which could implement 
properties for accessing specific bits.

Regan
Nov 23 2007
prev sibling next sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
I don't like the idea of referring to e.g. n[3..4]. The way I see it,
that's just as error prone as bitshifting. No, what you really want to
do is separate interface from implementation. The interface is, you
want to be able to manipulate bitfields as though they were integers.
The implementation is how you want to store them in a struct. The
whole business of using slice operators pulls implementation details
into the algorithm, where they don't belong.

Howver, it seems to me that we already /have/ bitfields in D. Or at
least, the ability to make them. In the example below, the D usage is
identical to the C usage. All that's different is the way they're
declared. Not only that, it seems to me that the D way offers far more
precision and flexibility. Compare...

C/C++:

    /* define the structure */
    struct S
    {
        unsigned int a : 3;
        unsigned int b : 6;
        unsigned int c : 2;
        unsigned int d : 5;
    }

    /* declare an instance */
    S s;

    /* use the bits */
    s.b = 4;
    s.c = 3;
    s.d = s.b + 1;
    // etc....

D

    /* define the structure */
    struct S
    {
        ushort bits;
	
        uint a() { return (bits >> 13) & 0x07;
        uint b() { return (bits >>  7) & 0x3F;
        uint c() { return (bits >>  5) & 0x03;
        uint d() { return (bits >>  0) & 0x1F;

        void c(uint n) { bits &= 0x1FFF; bits |= (n << 13); }
        void b(uint n) { bits &= 0xE07F; bits |= (n <<  7); }
        void c(uint n) { bits &= 0xFF9F; bits |= (n <<  5); }
        void d(uint n) { bits &= 0xFFE0; bits |= (n <<  0); }
    }

    /* declare an instance */
    S s;

    /* use the bits */
    s.b = 4;
    s.c = 3;
    s.d = s.b + 1;
    // etc....

I'm sure some smart person can come up with a mixin to automate even this!
Nov 23 2007
next sibling parent reply Richard Bradley <darthalias yahoo.com> writes:
Yes, You are right Janice. We want to access the bitfields in a structure as if
they were the integers.

Like this:

packet.ip.type_of_service.reliability = 1;

(See http://www.protocols.com/pbook/tcpip2.htm#IP for ip protocol.)

I whole heartedly disagree that your workaround is sufficient. It is a bit like
saying C doesn't need native string support because you can just make a
character array, or that you can just make a C++ class to handle strings. (We
all know how well that works, everyone and their grandmother has a different
string class.)

If you look at your example you are replacing the four parameters in the C++
version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield
positions, and 8 masks.) not to mention the shifting and bitwise operations.
And the need of the user to keep track of the “bits” members size. Plus, what
if you wanted a structure 512 bytes long? Your ushort “bits” would have to be
an array, and you would need to be very careful then of how your bitfields
mapped into that array. This is adding a lot of accidental complexity to the
problem. It's is okay if you are doing this once or twice. But I am literally
talking about doing this on thousands of classes per project, on structures
that can be quite large.

The D workaround adds quite a bit of error-prone complexity.

I would also like to suggest that this isn't a rarely-used feature. You run
into this issue anytime you want to read or write to hardware, (or link to a
library that talks to hardware) or access any sort of network protocol stack,
or any type of communication protocol. 

As an outsider looking in that works in these domains, this becomes a glaring
hole in an otherwise wonderful language. One of my fundamental coding tasks is
to talk to hardware, and I don't see that task as at all being rare.

I would agree that the way C++ does bitfields is “inefficient.” I know of no
general-purpose language that does this well. The issues of portability (with
respect to byte ordering) is valid, but kinda beside the point. If you are
hacking hardware, it is by definition tied to the hardware. Just like the
windows library is not portable outside windows. Plus, if we fixed that whole
byte-order thing it would be a big justification to using the language, as the
way C does it is just ugly.

There seems to be a lot of support for a feature along these lines. There has
been a good discussion going on for quite some time. The underlying question is
do we want D to live in this “low level bare metal “ environment? And if so,
what do we (yes, I am including myself) need to do?

~Richard
Nov 23 2007
next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
On 11/23/07, Richard Bradley <darthalias yahoo.com> wrote:

 I whole heartedly disagree that your workaround is sufficient.


I did not claim that the workaround was sufficient, I only claimed
that the workaround exists. You can't disagree with a claim I didn't
make.



 If you look at your example you are replacing the four parameters in the C++
version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield
positions, and 8 masks.) not to mention the shifting and bitwise operations.
And the need of the user to keep track of the "bits" members size. Plus, what
if you wanted a structure 512 bytes long?


Agreed. That would make it tricky to write getter and setter functions
for each bitfield. I've certainly no objection to the compiler making
things easier for the programmer. The shifting and bitwise operations
will happen behind the scenes though, even if you're not aware of it.

I think the way to do it to have the compiler automagically write
getter and setter functions for you. That way, the following would be
allowed:

    n = s.a; // read a bitfield
    s.a = n; // write a bitfield

but things like s.a+=, &s.a, passing bitfields by reference, and so
on, would still be disallowed. That would force the programmer to do
all the calculations in "real" ints, using the bitfields only for
storage.
Nov 23 2007
prev sibling next sibling parent Robert Fraser <fraserofthenight gmail.com> writes:
Richard Bradley wrote:

 I would also like to suggest that this isn't a rarely-used feature. You run
into this issue anytime you want to read or write to hardware, (or link to a
library that talks to hardware) or access any sort of network protocol stack,
or any type of communication protocol.


It's also used to reduce database throughput and in serialization of 
large structures to files, among other things. Although I'm not at all a 
"low-level"/"close-to-the-metal" programmer, I would definitely 
appreciate such a feature.
Nov 23 2007
prev sibling next sibling parent reply Nick B <nick.barbalich gmail.com> writes:
Richard Bradley wrote:

There seems to be a lot of support for a feature along these lines. 
There has been a good discussion going on for quite some time. The 
underlying question is do we want D to live in this “low level bare 
metal “ environment? And if so, what do we (yes, I am including 


myself) > need to do?

Richard


As no one else if volunteering at present,
I would like to suggest the following.

You create one document that covers the following:

1. Detail exactly the Bitfield functionality what you require (you
    seem to know this better than anyone on this list);
2. Address why the language _needs_  this functionality;	
2. Describe the target audience of the said feature(s);
3. Make a pitch as to the _number_ of additional users/projects who
    might join the "D" community as a result of said features.

When completed, ask for feedback from the "D" community, incorporate any 
feedback if required, and then, finally send it to Walter directly, 
asking for comment/feedback.

Good Luck

Nick B
Nov 23 2007
parent reply "David B. Held" <dheld codelogicconsulting.com> writes:
Nick B wrote:

 [...]
 As no one else if volunteering at present,
 I would like to suggest the following.
 
 You create one document that covers the following:
 
 1. Detail exactly the Bitfield functionality what you require (you
    seem to know this better than anyone on this list);
 2. Address why the language _needs_  this functionality;   
 2. Describe the target audience of the said feature(s);
 3. Make a pitch as to the _number_ of additional users/projects who
    might join the "D" community as a result of said features.
 [...]


Before anyone spends too much time on this, it might be worth relating a 
tidbit that a little bird told me (yes, it happened just like a Disney 
movie): an imminent release of D is going to contain bitfield support 
(though not necessarily looking like any of the suggestions / 
implementations proposed here).

Dave


P.S.  If you plan to use little birds as target practice for your .22, 
may I suggest you wait to see if the bird has anything to say, first?
Nov 24 2007
parent Martin d Anjou <point14 magma.ca> writes:
 Nick B wrote:

  [...]
  As no one else if volunteering at present,
  I would like to suggest the following.

  You create one document that covers the following:

  1. Detail exactly the Bitfield functionality what you require (you
     seem to know this better than anyone on this list);
  2. Address why the language _needs_  this functionality;   2. Describe the
  target audience of the said feature(s);
  3. Make a pitch as to the _number_ of additional users/projects who
     might join the "D" community as a result of said features.
  [...]




I'm interested in this too.

// Pseudo-code D-like, do not try this at home
bitvector!(13) bv;
bv[3] = 1;
assert(bv == 0b0000000001000); 
bv[10:8] = 3;
assert(bv == 0b0011100001000);
alias bv[6:4] bv.field1;
bv.field1 = 2;
assert(bv == 0b0011100101000);
assert(bv && true);
bv = 0;
assert(bv && false);
bv++;
assert(bv == 1);
bv = 0b1111111111111;
bv++;
assert(bv == 0);

bitvector!(122031) bv2; // Should also work. Limit is memory.

Martin
Nov 27 2007
prev sibling parent reply "Janice Caron" <caron800 googlemail.com> writes:
On Nov 23, 2007 5:50 PM, Janice Caron <caron800 googlemail.com> wrote:

 I think the way to do it to have the compiler automagically write
 getter and setter functions for you. That way, the following would be
 allowed:

     n = s.a; // read a bitfield
     s.a = n; // write a bitfield

 but things like s.a+=, &s.a, passing bitfields by reference, and so
 on, would still be disallowed. That would force the programmer to do
 all the calculations in "real" ints, using the bitfields only for
 storage.


Hey, someone's been listening to me! :-)

Check out std.bitmanip in D2.008.

We have bitfields.
Nov 29 2007
parent Leandro Lucarella <llucax gmail.com> writes:
Janice Caron, el 29 de noviembre a las 08:45 me escribiste:

 On Nov 23, 2007 5:50 PM, Janice Caron <caron800 googlemail.com> wrote:

 I think the way to do it to have the compiler automagically write
 getter and setter functions for you. That way, the following would be
 allowed:

     n = s.a; // read a bitfield
     s.a = n; // write a bitfield

 but things like s.a+=, &s.a, passing bitfields by reference, and so
 on, would still be disallowed. That would force the programmer to do
 all the calculations in "real" ints, using the bitfields only for
 storage.


 
 Hey, someone's been listening to me! :-)
 
 Check out std.bitmanip in D2.008.
 
 We have bitfields.


*Ugly* bitfields, but bitfield at last (language support would be much
nicer... :S)

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Charles Chaplin once won 3rd price in a Charles Chaplin look-a-like contest
Nov 29 2007
prev sibling parent Richard Bradley <darthalias yahoo.com> writes:
Yes, You are right Janice. We want to access the bitfields in a structure as if
they were the integers.

Like this:

packet.ip.type_of_service.reliability = 1;

(See http://www.protocols.com/pbook/tcpip2.htm#IP for ip protocol.)

I whole heartedly disagree that your workaround is sufficient. It is a bit like
saying C doesn't need native string support because you can just make a
character array, or that you can just make a C++ class to handle strings. (We
all know how well that works, everyone and their grandmother has a different
string class.)

If you look at your example you are replacing the four parameters in the C++
version ( the four bitfield sizes) with twelve parameters in D ( 4 bitfield
positions, and 8 masks.) not to mention the shifting and bitwise operations.
And the need of the user to keep track of the “bits” members size. Plus, what
if you wanted a structure 512 bytes long? Your ushort “bits” would have to be
an array, and you would need to be very careful then of how your bitfields
mapped into that array. This is adding a lot of accidental complexity to the
problem. It's is okay if you are doing this once or twice. But I am literally
talking about doing this on thousands of classes per project, on structures
that can be quite large.

The D workaround adds quite a bit of error-prone complexity.

I would also like to suggest that this isn't a rarely-used feature. You run
into this issue anytime you want to read or write to hardware, (or link to a
library that talks to hardware) or access any sort of network protocol stack,
or any type of communication protocol. 

As an outsider looking in that works in these domains, this becomes a glaring
hole in an otherwise wonderful language. One of my fundamental coding tasks is
to talk to hardware, and I don't see that task as at all being rare.

I would agree that the way C++ does bitfields is “inefficient.” I know of no
general-purpose language that does this well. The issues of portability (with
respect to byte ordering) is valid, but kinda beside the point. If you are
hacking hardware, it is by definition tied to the hardware. Just like the
windows library is not portable outside windows. Plus, if we fixed that whole
byte-order thing it would be a big justification to using the language, as the
way C does it is just ugly.

There seems to be a lot of support for a feature along these lines. There has
been a good discussion going on for quite some time. The underlying question is
do we want D to live in this “low level bare metal “ environment? And if so,
what do we (yes, I am including myself) need to do?

~Richard
Nov 23 2007
prev sibling next sibling parent reply 0ffh <frank frankhirsch.youknow.what.todo.net> writes:
Richard Bradley wrote:

 It's not a priority, it's something that could be added later without
  breaking anything -- just syntactic sugar, really. We can try again 
 later <g>.


 
 Curious, Has there been any movement on this issue? Is there a less
 error-prone way to do this other then manual shifts and masks?
 
 [...]
 
 Comments, ideas welcome. Richard


FWIW I am quite sympathetic to the cause of C-like bitfields for D.
My take on D in the context of Kelleher's "What languages fix" was
always "D: C++ is a syntactic mess, and built-in GC is a win.".

OTOH, the accessor templates from the D.learn thread look very nice.

The question is probably: Do we really need native bitfield support,
or is D mighty enough that a sufficient solution can be implemented
without resorting to adding to the language itself?
I think the answer may in part depend on the result of the planned
"GCC bitfields" vs. "GDC templates" shootout as planned on D.learn
by bearophile.

Hi bearophile, any results yet? :)

regards, frank
Nov 23 2007
parent bearophile <bearophileHUGS lycos.com> writes:
0ffh:


 OTOH, the accessor templates from the D.learn thread look very nice.


I think they aren't good enough yet for practical usage, they do too much
upcasting (to 64 bit numbers, in that thread I have shown it that using 32 bit
makes those bitfields faster), so they require improvement. And they need more
testing.



 I think the answer may in part depend on the result of the planned
 "GCC bitfields" vs. "GDC templates" shootout as planned on D.learn
 by bearophile.
 Hi bearophile, any results yet? :)


Hello, but sadly os far I haven't succed yet to install GDC (I'll try again),
despite I already have MinGW installed. So I have tried with just the MinGW,
the following:

#include <stdio.h>
#define N 25

typedef union {
  unsigned int i;
  struct {
    unsigned int i;

    unsigned int sign(void) { return (i >> 0x0) & 0x1; }
    void sign(unsigned int val) { i = (i & 0xfffffffffffffffeLL) | ((val & 0x1)
<< 0x0); }

    unsigned int biasedexponent(void) { return (i >> 0x1) & 0xff; }
    void biasedexponent(unsigned int val) { i = (i & 0xfffffffffffffe01LL) |
((val & 0xff) << 0x1); }

    unsigned int significand(void) { return (i >> 0x9) & 0x7fffff; }
    void significand(unsigned int val) { i = (i & 0x1ff) | ((val & 0x7fffff) <<
0x9); }
  } bits;
} INTORFLOAT;

int main(void) {
  int i, j;
  unsigned long long tot = 0;
  INTORFLOAT f;

  unsigned int v[N] = {1628676761, 1620574103, 1237153253, 1098880307, 87513741,
             13181925, 14686126, 7429435, 16286706, 6474381,
             4879794, 7734725, 3745958, 13353858, 4236193,
             7587, 4309, 28846, 7313, 14516,
             126, 143, 171, 221, 156};

  for(j = 0; j < 4000000; j++)
    for(i = 0; i < N; i++) {
      f.i = v[i];
      f.bits.biasedexponent(f.bits.biasedexponent() / 2);
      tot += f.bits.biasedexponent() + f.bits.sign();
    }

   printf("%d\n", tot);
}

It runs even a bit faster than the normal C bitfields ;-) So they may run fast
on GDC too.

Bye,
bearophile
Nov 24 2007
prev sibling parent Nick B <nick.barbalich gmail.com> writes:
Richard Bradley wrote:

 It's not a priority, it's something that could be added later without 
 breaking anything -- just syntactic sugar, really. We can try again 
 later <g>.


 
 Curious, Has there been any movement on this issue? Is there a less
error-prone way to do this other then manual shifts and masks?
 
 I am looking to write an open-source generic library for hardware design, I
thought D would make a much better base then C++. I'm a big fan of what I've
seen so far. Sorry to say, but D can't be a serious low-level contender without
bitfield support.
 
 I'm not talking about using it once or twice in a project, but thousands of
structures for each project, and each structure having dozens of bitfields.
Having to do masks and shifts just seems like a nightmare of bad code. 
 
 I can't imagine my friends on the firmware side (or any low-level developer)
wouldn't agree. 
 
 Miles' idea of making D somehow order aware would be a huge win for this
close-to-the-hardware stuff. Or even non-order aware bitfields at least isn't
worse then what's allready out there.
 
 Comments, ideas welcome.
 Richard
 
 


Richard

Does this help at all ?


http://www.digitalmars.com/d/phobos/std_bitmanip.html

Nick B
Dec 03 2007
prev sibling parent bearophile <bearophileHUGS mailas.com> writes:
Regan Heath:

 In the meantime, in order to avoid having to re-code all the bit shifts 
 in every struct with bitfields isn't there a mixin which could implement 
 properties for accessing specific bits.


Take a look at the "Bitfield accessors" thread in digitalmars.D.learn
Nov 23 2007
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Wolfgang Draxinger wrote:

 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.
 
 Has somebody an idea how I best interface with it elegantly?


Yes,

http://www.digitalmars.com/d/phobos/std_bitmanip.html
Nov 28 2007
parent "Craig Black" <craigblack2 cox.net> writes:
"Walter Bright" <newshound1 digitalmars.com> wrote in message 
news:fiksel$28qu$2 digitalmars.com...

 Wolfgang Draxinger wrote:

 Now with SDL I've encountered a problem: SDL makes use of serval
 bitfields and I've no idea, how to interface them to D.

 Has somebody an idea how I best interface with it elegantly?


 Yes,

 http://www.digitalmars.com/d/phobos/std_bitmanip.html


D has bit fields!  Sweet!
Nov 29 2007