www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - proper bit fields in the D2 language?

reply nobody <hehe hehe.com> writes:
I remember read D's introduction page that C's bit field is something will be
dropped from D.

But now I see dmd/src/phobos/std/bitmanip.d line 30 ~ 98: bit field functions
are generated at compile time,

private template createAccessors(...) {
  // getter
  // setter
  ...
}

I wonder how efficient is this. If we are going to have bit fields in the
*std* library, why not support it properly in the language. I think the
compiler can generate much better code than these compile-time generated
functions, which looks like a hack to me.

comments?
Apr 21 2009
parent reply Brad Roberts <braddr puremagic.com> writes:
nobody wrote:
 I remember read D's introduction page that C's bit field is something will be
 dropped from D.
 
 But now I see dmd/src/phobos/std/bitmanip.d line 30 ~ 98: bit field functions
 are generated at compile time,
 
 private template createAccessors(...) {
   // getter
   // setter
   ...
 }
 
 I wonder how efficient is this. If we are going to have bit fields in the
 *std* library, why not support it properly in the language. I think the
 compiler can generate much better code than these compile-time generated
 functions, which looks like a hack to me.
 
 comments?

The auto-generated code from the library is the same code the compiler would end up generating. You can test that theory by comparing the produced assembly for a C vs a D implementation. Later, Brad
Apr 21 2009
parent reply nobody <hehe hehe.com> writes:
== Quote from Brad Roberts (braddr puremagic.com)'s article
 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.

If the generated code are same, and it's in the std library, which means no difference in the backend. Then I'd rather write: struct A { bool flag1: 1; bool flag2: 1; // uint "", 6; // this should be auto-magically generated by the compiler } (the code is more clear, and the compiler can give better message). than this: struct A { mixin(bitfields!( bool, "flag1", 1, bool, "flag2", 1, uint, "", 6)); }
Apr 21 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
nobody wrote:
 == Quote from Brad Roberts (braddr puremagic.com)'s article
 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.

If the generated code are same, and it's in the std library, which means no difference in the backend. Then I'd rather write: struct A { bool flag1: 1; bool flag2: 1; // uint "", 6; // this should be auto-magically generated by the compiler } (the code is more clear, and the compiler can give better message). than this: struct A { mixin(bitfields!( bool, "flag1", 1, bool, "flag2", 1, uint, "", 6)); }

Except that bitfields don't appear to be a widely used feature. If you can put it in the library with no performance penalty AND simplify the compiler at the same time, that looks like a good thing to me, and I believe Walter agrees. -- Daniel
Apr 21 2009
parent reply grauzone <none example.net> writes:
Daniel Keep wrote:
 
 nobody wrote:
 == Quote from Brad Roberts (braddr puremagic.com)'s article
 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.

difference in the backend. Then I'd rather write: struct A { bool flag1: 1; bool flag2: 1; // uint "", 6; // this should be auto-magically generated by the compiler } (the code is more clear, and the compiler can give better message). than this: struct A { mixin(bitfields!( bool, "flag1", 1, bool, "flag2", 1, uint, "", 6)); }

Except that bitfields don't appear to be a widely used feature. If you can put it in the library with no performance penalty AND simplify the compiler at the same time, that looks like a good thing to me, and I believe Walter agrees.

Yeah, let's move everything from the compiler to the library. Damn, why couldn't they have const implemented as library feature?
   -- Daniel

Apr 21 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
grauzone wrote:
 Daniel Keep wrote:
 nobody wrote:
 == Quote from Brad Roberts (braddr puremagic.com)'s article
 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.

means no difference in the backend. Then I'd rather write: struct A { bool flag1: 1; bool flag2: 1; // uint "", 6; // this should be auto-magically generated by the compiler } (the code is more clear, and the compiler can give better message). than this: struct A { mixin(bitfields!( bool, "flag1", 1, bool, "flag2", 1, uint, "", 6)); }

Except that bitfields don't appear to be a widely used feature. If you can put it in the library with no performance penalty AND simplify the compiler at the same time, that looks like a good thing to me, and I believe Walter agrees.

Yeah, let's move everything from the compiler to the library. Damn, why couldn't they have const implemented as library feature?
   -- Daniel


Except that bitfields don't appear to be a widely used feature. -- Daniel
Apr 21 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Daniel Keep:
 Except that bitfields don't appear to be a widely used feature.

I have found plenty of them in code to translate from C to other languages. Another possible solution is the same intermediate one that will be used for the associative arrays: keep only the syntax in the language and implement their semantics in the std lib. Bye, bearophile
Apr 22 2009
prev sibling parent reply Mattias Holm <mattias.holm openorbit.REMOVE.THIS.org> writes:
 Except that bitfields don't appear to be a widely used feature.

You clearly have not written systems code in Ada... :) C-bitfields are problematic because the bit-ordering is implementation defined, GCC have the bits appear in the order of definition on big-endian machines and the reverse order on little endian machines. The appropriate way is to have them as the big-endian fields in GCC, this is also what most Ada-compilers seem to be doing. They are incredibly useful for any systems and protocol programming, and this is why they are nice to have in a language like D. Bitfields are not used in C because they are not platform neutral, a proper definition in D would mean that people would use them, at least in the mentioned domains. / Mattias
Apr 27 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Mattias Holm:
 You clearly have not written systems code in Ada... :)

Bitfields are quite useful, also take a look at the high level language Erlang, that has some high-level ways to use them. In the end the current design of D2 bitfields is acceptable. Inecessary some syntax sugar can be added later, as macro form. Bye, bearophile
Apr 27 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Mattias Holm wrote:
 Except that bitfields don't appear to be a widely used feature.

You clearly have not written systems code in Ada... :) C-bitfields are problematic because the bit-ordering is implementation defined, GCC have the bits appear in the order of definition on big-endian machines and the reverse order on little endian machines. The appropriate way is to have them as the big-endian fields in GCC, this is also what most Ada-compilers seem to be doing. They are incredibly useful for any systems and protocol programming, and this is why they are nice to have in a language like D. Bitfields are not used in C because they are not platform neutral, a proper definition in D would mean that people would use them, at least in the mentioned domains.

Bitfields in Phobos are defined portably: always populated from lsb to msb, the total size must be 8, 16, 32, or 64, and there is no hidden padding (you obtain padding with anonymous fields). Andrei
Apr 27 2009
parent reply Mattias Holm <mattias.holm openorbit.REMOVE.THIS.org> writes:
 Bitfields in Phobos are defined portably: always populated from lsb to 
 msb, the total size must be 8, 16, 32, or 64, and there is no hidden 
 padding (you obtain padding with anonymous fields).
 
 Andrei

If this is the case, the bitfields are not portable and the behaviour is the same as for GCC, except it is reversed. The order should be: LSByte to MSByte on little endian machines MSByte to LSByte on big endian machines This means that bitfields are defined in human readable order with respect to the physical layout on the machine. The point being: struct { uint a:4; uint b:28; } should give a structure: _ _______ |.|.......| where each dot represents 4 bits. And this structure should be identical if the data is moved between different systems, that is a TCP header definition mapping into a byte stream, should be identical on both PPC and x86. If the LSB to MSB ordering is used in phobos, then the bitfields are as useful as the GCC bitfields, i.e. not at all. /Mattias
Apr 27 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Mattias Holm wrote:
 Bitfields in Phobos are defined portably: always populated from lsb to 
 msb, the total size must be 8, 16, 32, or 64, and there is no hidden 
 padding (you obtain padding with anonymous fields).

 Andrei

If this is the case, the bitfields are not portable and the behaviour is the same as for GCC, except it is reversed. The order should be: LSByte to MSByte on little endian machines MSByte to LSByte on big endian machines This means that bitfields are defined in human readable order with respect to the physical layout on the machine. The point being: struct { uint a:4; uint b:28; } should give a structure: _ _______ |.|.......| where each dot represents 4 bits. And this structure should be identical if the data is moved between different systems, that is a TCP header definition mapping into a byte stream, should be identical on both PPC and x86. If the LSB to MSB ordering is used in phobos, then the bitfields are as useful as the GCC bitfields, i.e. not at all.

Not at all is a tad extreme as I'm using them already. I see your point though. If you could submit a bug report, that would be great. Andrei
Apr 27 2009
prev sibling parent grauzone <none example.net> writes:
nobody wrote:
 == Quote from Brad Roberts (braddr puremagic.com)'s article
 The auto-generated code from the library is the same code the compiler
 would end up generating.  You can test that theory by comparing the
 produced assembly for a C vs a D implementation.

If the generated code are same, and it's in the std library, which means no difference in the backend. Then I'd rather write: struct A { bool flag1: 1; bool flag2: 1; // uint "", 6; // this should be auto-magically generated by the compiler } (the code is more clear, and the compiler can give better message). than this: struct A { mixin(bitfields!( bool, "flag1", 1, bool, "flag2", 1, uint, "", 6)); }

Bitfields are a rather obscure feature. There's no real reason to have it in the compiler. But because D is so great, you still can have it in the standard library. Sadly, this comes with a bit more ugly syntax. If we had macros (which were ditched for the oh-so-great [actually questionable] features const and immutable), then it could be at least a _bit_ more beautiful. Like defining identifiers as identifiers, and not strings. What I'm really concerned about is that this mixin MAGICALLY inserts unknown symbols into the current scope. I was too lazy to look in the Phobos code, but it's clear that the mixin must generate "hidden" fields to store the actual bit values. (With a bit of luck, the author [Andrei?] was careful enough to allow several bitfield mixins in the same struct.) I also will blow up if later you add serialization or anything in this direction. A clean way would to to let the bitfields thing generate a real struct type: alias Bitfields!(bool, "flag1", 1, bool, "flag2", 6) MyBitfields; No more magical insertion of identifiers into your code. And a serialization mechanism just could detect the type Bitfields and write its actual members, and not the hidden bit array field (or whatever the implementation uses, it just insert its own, unknown-to-you symbols into YOUR scope).
Apr 21 2009