www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Reading and converting binary file 2 bits at a time

reply "Andrew Brown" <aabrown24 hotmail.com> writes:
Hi,

I need to read a binary file, and then process it two bits at a 
time. But I'm a little stuck on the first step. So far I have:

import std.file;
import std.stdio;

void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
}

but all the values of g then are just true, could you tell me 
what I'm doing wrong? I've also looked at the bitmanip module, I 
couldn't get it to help, but is that the direction I should be 
looking?

Thanks very much

Andrew
Aug 27 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 27/08/15 9:00 PM, Andrew Brown wrote:
 Hi,

 I need to read a binary file, and then process it two bits at a time.
 But I'm a little stuck on the first step. So far I have:

 import std.file;
 import std.stdio;

 void main(){
    auto f = std.file.read("binaryfile");
    auto g = cast(bool[])    f;
    writeln(g);
 }

 but all the values of g then are just true, could you tell me what I'm
 doing wrong? I've also looked at the bitmanip module, I couldn't get it
 to help, but is that the direction I should be looking?

 Thanks very much

 Andrew
You do not use the std.file to prefix call read. The bool type is one byte in size. A value of 0 is false, anything else is true. But commonly defined as 1. So if you want to read only one bit you will need to use e.g. bit shifts and bitwise and operator. I believe bitsSet may help you. http://dlang.org/phobos/std_bitmanip.html#.bitsSet
Aug 27 2015
prev sibling next sibling parent reply "rumbu" <rumbu rumbu.ro> writes:
On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:
 Hi,

 I need to read a binary file, and then process it two bits at a 
 time. But I'm a little stuck on the first step. So far I have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip module, 
 I couldn't get it to help, but is that the direction I should 
 be looking?

 Thanks very much

 Andrew
auto bytes = cast(ubyte[])read("binaryfile"); foreach(b; bytes) { writeln((b & 0xC0) >> 6); //bits 7, 6 writeln((b & 0x30) >> 4); //bits 5, 4 writeln((b & 0x0C) >> 2); //bits 3, 2 writeln((b & 0x03)); //bits 1, 0 }
Aug 27 2015
parent reply "Andrew Brown" <aabrown24 hotmail.com> writes:
On Thursday, 27 August 2015 at 09:26:55 UTC, rumbu wrote:
 On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:
 Hi,

 I need to read a binary file, and then process it two bits at 
 a time. But I'm a little stuck on the first step. So far I 
 have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip module, 
 I couldn't get it to help, but is that the direction I should 
 be looking?

 Thanks very much

 Andrew
auto bytes = cast(ubyte[])read("binaryfile"); foreach(b; bytes) { writeln((b & 0xC0) >> 6); //bits 7, 6 writeln((b & 0x30) >> 4); //bits 5, 4 writeln((b & 0x0C) >> 2); //bits 3, 2 writeln((b & 0x03)); //bits 1, 0 }
That's lovely, thank you. One quick question, the length of the file is not a multiple of the length of ubyte, but the cast still seems to work. Do you know how it converts a truncated final section? Thanks again Andrew
Aug 27 2015
next sibling parent "Olivier Pisano" <olivier.pisano laposte.net> writes:
On Thursday, 27 August 2015 at 09:38:52 UTC, Andrew Brown wrote:

 That's lovely, thank you. One quick question, the length of the 
 file is not a multiple of the length of ubyte,  but the cast 
 still seems to work. Do you know how it converts a truncated 
 final section?

 Thanks again

 Andrew
Well, since the length of ubyte is 1, the length of the file is necessarily a multiple of the length of ubyte.
Aug 27 2015
prev sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Thursday, 27 August 2015 at 09:38:52 UTC, Andrew Brown wrote:
 On Thursday, 27 August 2015 at 09:26:55 UTC, rumbu wrote:
 On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown 
 wrote:
 Hi,

 I need to read a binary file, and then process it two bits at 
 a time. But I'm a little stuck on the first step. So far I 
 have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip 
 module, I couldn't get it to help, but is that the direction 
 I should be looking?

 Thanks very much

 Andrew
auto bytes = cast(ubyte[])read("binaryfile"); foreach(b; bytes) { writeln((b & 0xC0) >> 6); //bits 7, 6 writeln((b & 0x30) >> 4); //bits 5, 4 writeln((b & 0x0C) >> 2); //bits 3, 2 writeln((b & 0x03)); //bits 1, 0 }
That's lovely, thank you. One quick question, the length of the file is not a multiple of the length of ubyte, but the cast still seems to work. Do you know how it converts a truncated final section? Thanks again Andrew
You can also avoid the bitshifts to make the code a little more readable like this: import std.stdio; import std.bitmanip; struct Crumbs { mixin(bitfields!( uint, "one", 2, uint, "two", 2, uint, "three", 2, uint, "four", 2 )); } void main(string[] args) { ubyte[] buffer = [123, 12, 126, 244, 35]; foreach (octet; buffer) { auto crumbs = Crumbs(octet); ubyte* representation = cast(ubyte*)&crumbs; writefln("Crumb: %08b", *representation); writefln("Crumb one: %s", crumbs.one); writefln("Crumb two: %s", crumbs.two); writefln("Crumb three: %s", crumbs.three); writefln("Crumb four: %s", crumbs.four); writeln("---------------------"); } } Now you can read a crumb at a time.
Aug 27 2015
prev sibling parent reply "Mike James" <foo bar.com> writes:
On Thursday, 27 August 2015 at 09:00:02 UTC, Andrew Brown wrote:
 Hi,

 I need to read a binary file, and then process it two bits at a 
 time. But I'm a little stuck on the first step. So far I have:

 import std.file;
 import std.stdio;

 void main(){
   auto f = std.file.read("binaryfile");
   auto g = cast(bool[])	f;
   writeln(g);
 }

 but all the values of g then are just true, could you tell me 
 what I'm doing wrong? I've also looked at the bitmanip module, 
 I couldn't get it to help, but is that the direction I should 
 be looking?

 Thanks very much

 Andrew
How about... module main; import std.bitmanip; import std.stdio; struct Crumbs { property ref ubyte whole() { return m_whole; } union { private ubyte m_whole; mixin(bitfields!( ubyte, "one", 2, ubyte, "two", 2, ubyte, "three", 2, ubyte, "four", 2 )); } } void main(string[] argv) { ubyte[] buffer = [123, 12, 126, 244, 35]; Crumbs cmb; foreach (octet; buffer) { cmb.whole = octet; writefln("Crumb: %08b", octet); writefln("Crumb one: %s", cmb.one); writefln("Crumb two: %s", cmb.two); writefln("Crumb three: %s", cmb.three); writefln("Crumb four: %s", cmb.four); } } Regards, Mike.
Aug 27 2015
next sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Thursday, 27 August 2015 at 12:40:07 UTC, Mike James wrote:
 How about...
A lot nicer. :)
Aug 27 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
Just cast to `Crumbs[]` directly:

     import std.bitmanip;
     import std.stdio;
     import std.file;

     struct Crumbs {
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }

     void main(string[] argv)
     {
         auto raw = read("binaryfile");
         auto buffer = cast(Crumbs[]) raw;

         foreach (cmb; buffer) {
             writefln("Crumb one:   %s", cmb.one);
             writefln("Crumb two:   %s", cmb.two);
             writefln("Crumb three: %s", cmb.three);
             writefln("Crumb four:  %s", cmb.four);
         }
     }
Aug 29 2015
parent reply "Mike James" <foo bar.com> writes:
On Saturday, 29 August 2015 at 20:15:53 UTC, Marc Schütz wrote:
 Just cast to `Crumbs[]` directly:

     import std.bitmanip;
     import std.stdio;
     import std.file;

     struct Crumbs {
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }

     void main(string[] argv)
     {
         auto raw = read("binaryfile");
         auto buffer = cast(Crumbs[]) raw;

         foreach (cmb; buffer) {
             writefln("Crumb one:   %s", cmb.one);
             writefln("Crumb two:   %s", cmb.two);
             writefln("Crumb three: %s", cmb.three);
             writefln("Crumb four:  %s", cmb.four);
         }
     }
I like that :-)
Aug 29 2015
parent reply "Gary Willoughby" <dev nomad.so> writes:
On Saturday, 29 August 2015 at 21:50:12 UTC, Mike James wrote:
 On Saturday, 29 August 2015 at 20:15:53 UTC, Marc Schütz wrote:
 Just cast to `Crumbs[]` directly:

     import std.bitmanip;
     import std.stdio;
     import std.file;

     struct Crumbs {
         mixin(bitfields!(
             ubyte, "one",   2,
             ubyte, "two",   2,
             ubyte, "three", 2,
             ubyte, "four",  2
         ));
     }

     void main(string[] argv)
     {
         auto raw = read("binaryfile");
         auto buffer = cast(Crumbs[]) raw;

         foreach (cmb; buffer) {
             writefln("Crumb one:   %s", cmb.one);
             writefln("Crumb two:   %s", cmb.two);
             writefln("Crumb three: %s", cmb.three);
             writefln("Crumb four:  %s", cmb.four);
         }
     }
I like that :-)
But it might not be safe: http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.org
Aug 29 2015
parent reply "anonymous" <anonymous example.com> writes:
On Saturday, 29 August 2015 at 23:34:47 UTC, Gary Willoughby 
wrote:
 But it might not be safe: 
 http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.org
That link just takes me to this thread here again.
Aug 29 2015
parent reply "Gary Willoughby" <dev nomad.so> writes:
On Sunday, 30 August 2015 at 00:02:16 UTC, anonymous wrote:
 On Saturday, 29 August 2015 at 23:34:47 UTC, Gary Willoughby 
 wrote:
 But it might not be safe: 
 http://forum.dlang.org/thread/ztefzijqhwrouzlagrpq forum.dlang.org
That link just takes me to this thread here again.
Here's the correct link. http://forum.dlang.org/thread/sugxdshytelayxnsthwl forum.dlang.org
Aug 30 2015
parent reply "Andrew Brown" <aabrown24 hotmail.com> writes:
Thanks very much for all the help, your advice worked a treat. 
One final question, originally I was defining the struct inside 
the main loop and it was using 4 bytes per field rather than 2 
bits, e.g.:

import std.bitmanip;
import std.stdio;

struct Crumb1
{
   mixin(bitfields!(
		   ubyte, "one", 2,
		   ubyte, "two", 2,
		   ubyte, "three", 2,
		   ubyte, "four", 2));
}

void main()
{
   struct Crumb2
   {
     mixin(bitfields!(
		     ubyte, "one", 2,
		     ubyte, "two", 2,
		     ubyte, "three", 2,
		     ubyte, "four", 2));
   }

   writeln(Crumb1.sizeof, " ", Crumb2.sizeof);

}

outputs:

1 16

Is this correect behaviour?

Andrew
Aug 31 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 31 August 2015 at 18:00:54 UTC, Andrew Brown wrote:
 Is this correect behaviour?
Yes, the reason is because the nested struct has a hidden member - a pointer to its stack context. This allows it to access variables from the surrounding local scope. It adds 8 bytes on 64 bit cuz that's pointer size, and the other bytes are padding so the pointer is aligned on a word boundary. If you change it to `static struct` inside the main function (or any other function), you'll find it then has the size of one again, since a static struct does not have access to outer variables, so it leaves that hidden pointer out.
Aug 31 2015