www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Reading and converting binary file 2 bits at a time

reply "Andrew Brown" <aabrown24 hotmail.com> writes:
Hi,

I need to read a binary file, and then process it two bits at a 
time. But I'm a little stuck on the first step. So far I have:

import std.file;
import std.stdio;

void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
}

but all the values of g then are just true, could you tell me 
what I'm doing wrong? I've also looked at the bitmanip module, I 
couldn't get it to help, but is that the direction I should be 
looking?

Thanks very much

Andrew
Aug 27 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 27/08/15 9:00 PM, Andrew Brown wrote:

 Hi,

 I need to read a binary file, and then process it two bits at a time.
 But I'm a little stuck on the first step. So far I have:

 import std.file;
 import std.stdio;

 void main(){
    auto f = std.file.read("binaryfile");
    auto g = cast(bool[])    f;
    writeln(g);
 }

 but all the values of g then are just true, could you tell me what I'm
 doing wrong? I've also looked at the bitmanip module, I couldn't get it
 to help, but is that the direction I should be looking?

 Thanks very much

 Andrew


You do not use the std.file to prefix call read.
The bool type is one byte in size. A value of 0 is false, anything else 
is true. But commonly defined as 1.

So if you want to read only one bit you will need to use e.g. bit shifts 
and bitwise and operator.

I believe bitsSet may help you.
Aug 27 2015
prev sibling next sibling parent reply "rumbu" <rumbu rumbu.ro> writes:
On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:

 Hi,

 I need to read a binary file, and then process it two bits at a 
 time. But I'm a little stuck on the first step. So far I have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip module, 
 I couldn't get it to help, but is that the direction I should 
 be looking?

 Thanks very much

 Andrew


auto bytes = cast(ubyte[])read("binaryfile");
foreach(b; bytes)
{
     writeln((b & 0xC0) >> 6);  //bits 7, 6
     writeln((b & 0x30) >> 4);  //bits 5, 4
     writeln((b & 0x0C) >> 2);  //bits 3, 2
     writeln((b & 0x03));       //bits 1, 0
}
Aug 27 2015
parent reply "Andrew Brown" <aabrown24 hotmail.com> writes:
On Thursday, 27 August 2015 at 09:26:55 UTC, rumbu wrote:

 On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:

 Hi,

 I need to read a binary file, and then process it two bits at 
 a time. But I'm a little stuck on the first step. So far I 
 have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip module, 
 I couldn't get it to help, but is that the direction I should 
 be looking?

 Thanks very much

 Andrew


 auto bytes = cast(ubyte[])read("binaryfile");
 foreach(b; bytes)
 {
     writeln((b & 0xC0) >> 6);  //bits 7, 6
     writeln((b & 0x30) >> 4);  //bits 5, 4
     writeln((b & 0x0C) >> 2);  //bits 3, 2
     writeln((b & 0x03));       //bits 1, 0
 }


That's lovely, thank you. One quick question, the length of the 
file is not a multiple of the length of ubyte,  but the cast 
still seems to work. Do you know how it converts a truncated 
final section?

Thanks again

Andrew
Aug 27 2015
next sibling parent "Olivier Pisano" <olivier.pisano laposte.net> writes:
On Thursday, 27 August 2015 at 09:38:52 UTC, Andrew Brown wrote:


 That's lovely, thank you. One quick question, the length of the 
 file is not a multiple of the length of ubyte,  but the cast 
 still seems to work. Do you know how it converts a truncated 
 final section?

 Thanks again

 Andrew


Well, since the length of ubyte is 1, the length of the file is 
necessarily a multiple of the length of ubyte.
Aug 27 2015
prev sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Thursday, 27 August 2015 at 09:38:52 UTC, Andrew Brown wrote:

 On Thursday, 27 August 2015 at 09:26:55 UTC, rumbu wrote:

 On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown 
 wrote:

 Hi,

 I need to read a binary file, and then process it two bits at 
 a time. But I'm a little stuck on the first step. So far I 
 have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip 
 module, I couldn't get it to help, but is that the direction 
 I should be looking?

 Thanks very much

 Andrew


 auto bytes = cast(ubyte[])read("binaryfile");
 foreach(b; bytes)
 {
     writeln((b & 0xC0) >> 6);  //bits 7, 6
     writeln((b & 0x30) >> 4);  //bits 5, 4
     writeln((b & 0x0C) >> 2);  //bits 3, 2
     writeln((b & 0x03));       //bits 1, 0
 }


 That's lovely, thank you. One quick question, the length of the 
 file is not a multiple of the length of ubyte,  but the cast 
 still seems to work. Do you know how it converts a truncated 
 final section?

 Thanks again

 Andrew


You can also avoid the bitshifts to make the code a little more 
readable like this:

import std.stdio;
import std.bitmanip;

struct Crumbs
{
	mixin(bitfields!(
		uint, "one", 2,
		uint, "two", 2,
		uint, "three", 2,
		uint, "four", 2
	));
}

void main(string[] args)
{
	ubyte[] buffer = [123, 12, 126, 244, 35];

	foreach (octet; buffer)
	{
		auto crumbs = Crumbs(octet);
		ubyte* representation = cast(ubyte*)&crumbs;

		writefln("Crumb:       %08b", *representation);
		writefln("Crumb one:   %s", crumbs.one);
		writefln("Crumb two:   %s", crumbs.two);
		writefln("Crumb three: %s", crumbs.three);
		writefln("Crumb four:  %s", crumbs.four);
		writeln("---------------------");
	}
}

Now you can read a crumb at a time.
Aug 27 2015
prev sibling parent reply "Mike James" <foo bar.com> writes:
On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:

 Hi,

 I need to read a binary file, and then process it two bits at a 
 time. But I'm a little stuck on the first step. So far I have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip module, 
 I couldn't get it to help, but is that the direction I should 
 be looking?

 Thanks very much

 Andrew


How about...

module main;

import std.bitmanip;
import std.stdio;

struct Crumbs {
      property ref ubyte whole() {
         return m_whole;
     }

     union {
         private ubyte m_whole;
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }
}

void main(string[] argv)
{
     ubyte[] buffer = [123, 12, 126, 244, 35];
     Crumbs cmb;

     foreach (octet; buffer) {
         cmb.whole = octet;

         writefln("Crumb:       %08b", octet);
         writefln("Crumb one:   %s", cmb.one);
         writefln("Crumb two:   %s", cmb.two);
         writefln("Crumb three: %s", cmb.three);
         writefln("Crumb four:  %s", cmb.four);
     }
}


Regards, Mike.
Aug 27 2015
next sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Thursday, 27 August 2015 at 12:40:07 UTC, Mike James wrote:

 How about...


A lot nicer. :)
Aug 27 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
Just cast to `Crumbs[]` directly:

     import std.bitmanip;
     import std.stdio;
     import std.file;

     struct Crumbs {
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }

     void main(string[] argv)
     {
         auto raw = read("binaryfile");
         auto buffer = cast(Crumbs[]) raw;

         foreach (cmb; buffer) {
             writefln("Crumb one:   %s", cmb.one);
             writefln("Crumb two:   %s", cmb.two);
             writefln("Crumb three: %s", cmb.three);
             writefln("Crumb four:  %s", cmb.four);
         }
     }
Aug 29 2015
parent reply "Mike James" <foo bar.com> writes:
On Saturday, 29 August 2015 at 20:15:53 UTC, Marc Schütz wrote:

 Just cast to `Crumbs[]` directly:

     import std.bitmanip;
     import std.stdio;
     import std.file;

     struct Crumbs {
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }

     void main(string[] argv)
     {
         auto raw = read("binaryfile");
         auto buffer = cast(Crumbs[]) raw;

         foreach (cmb; buffer) {
             writefln("Crumb one:   %s", cmb.one);
             writefln("Crumb two:   %s", cmb.two);
             writefln("Crumb three: %s", cmb.three);
             writefln("Crumb four:  %s", cmb.four);
         }
     }


I like that :-)
Aug 29 2015
parent reply "Gary Willoughby" <dev nomad.so> writes:
On Saturday, 29 August 2015 at 21:50:12 UTC, Mike James wrote:

 On Saturday, 29 August 2015 at 20:15:53 UTC, Marc Schütz wrote:

 Just cast to `Crumbs[]` directly:

     import std.bitmanip;
     import std.stdio;
     import std.file;

     struct Crumbs {
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }

     void main(string[] argv)
     {
         auto raw = read("binaryfile");
         auto buffer = cast(Crumbs[]) raw;

         foreach (cmb; buffer) {
             writefln("Crumb one:   %s", cmb.one);
             writefln("Crumb two:   %s", cmb.two);
             writefln("Crumb three: %s", cmb.three);
             writefln("Crumb four:  %s", cmb.four);
         }
     }


 I like that :-)


But it might not be safe: 
http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.org
Aug 29 2015
parent reply "anonymous" <anonymous example.com> writes:
On Saturday, 29 August 2015 at 23:34:47 UTC, Gary Willoughby 
wrote:

 But it might not be safe: 
 http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.org


That link just takes me to this thread here again.
Aug 29 2015
parent reply "Gary Willoughby" <dev nomad.so> writes:
On Sunday, 30 August 2015 at 00:02:16 UTC, anonymous wrote:

 On Saturday, 29 August 2015 at 23:34:47 UTC, Gary Willoughby 
 wrote:

 But it might not be safe: 
 http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.org


 That link just takes me to this thread here again.


Here's the correct link.
http://forum.dlang.org/thread/sugxdshytelayxnsthwl forum.dlang.org
Aug 30 2015
parent reply "Andrew Brown" <aabrown24 hotmail.com> writes:
Thanks very much for all the help, your advice worked a treat. 
One final question, originally I was defining the struct inside 
the main loop and it was using 4 bytes per field rather than 2 
bits, e.g.:

import std.bitmanip;
import std.stdio;

struct Crumb1
{
   mixin(bitfields!(
		   ubyte, "one", 2,
		   ubyte, "two", 2,
		   ubyte, "three", 2,
		   ubyte, "four", 2));
}

void main()
{
   struct Crumb2
   {
     mixin(bitfields!(
		     ubyte, "one", 2,
		     ubyte, "two", 2,
		     ubyte, "three", 2,
		     ubyte, "four", 2));
   }

   writeln(Crumb1.sizeof, " ", Crumb2.sizeof);

}

outputs:

1 16

Is this correect behaviour?

Andrew
Aug 31 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 31 August 2015 at 18:00:54 UTC, Andrew Brown wrote:

 Is this correect behaviour?


Yes, the reason is because the nested struct has a hidden member 
- a pointer to its stack context. This allows it to access 
variables from the surrounding local scope.

It adds 8 bytes on 64 bit cuz that's pointer size, and the other 
bytes are padding so the pointer is aligned on a word boundary.

If you change it to `static struct` inside the main function (or 
any other function), you'll find it then has the size of one 
again, since a static struct does not have access to outer 
variables, so it leaves that hidden pointer out.
Aug 31 2015