www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Reading in files as int/float array or one String/char[] ( OpenGL Data )

reply ParticlePeter <ParticlePeter gmx.de> writes:
Hi,

I read several posts about reading in files, but I can't find the "best" way
for my purpose. I am using std.file in the examples bellow.

1.) I'm trying to read an OpenGL Vertex and Fragment Shader. The wired thing on
my approach is that it works ... sometimes, but sometimes the Shaders cannot be
compiled. 
So how should I read a file as one complete char[] in a safe way ? This is my
way:
  auto fileFrag = cast( GLchar[] )readText( "Shader/Shader_01.frag" ) ;
  const GLchar * fragSource = cast( GLchar * )fileFrag ;

2.) I am reading in Vertex Data from a file. It holds only floats, e.g.:
-38.3887 97.7612 -10.5231 
-38.3572 98.8543 -10.5064 
...
There is no metadata in the file giving information about the count of values.
I use a dynamic array, and append each line to my array, which I think is not
the most clever or efficient approach:
  vertices = new GLfloat[0] ;
  auto file = File( "Data/Vertices_01.txt" ) ;
  foreach( line ; file.byLine() )
    vertices ~= to!( GLfloat[] )( line.split ) ;
  file.close() ;

Can I get all floats ( words as float ) from a file at once without using
file.byLine into a Dynamic array ( or fixed array, I know the word count ) ? If
not, what would be the most efficient approach for these kind of files ( they
might get huge ) ?

3.) When reading a file line by line it would be nice to have the count of
Lines of the file ( numLines ) and the line number in a variable, so I could
random access a fixed size array with this line number. I tried this code,
which works for arrays, but unfortunately not for lines ( for me ).
  GLfloat arrVertex[ numLines ][ 3 ] ;
  foreach( i , line ; file.byLine() )
    arrVertex[ i ] = to!( GLfloat[3] )( line.split ) ;

How can I get the Line count from a file, and how can I get the line numbers
without initializing a counter in front of the foreach loop and increase it
manually inside the loop ?

Thank you for any advice,

Cheers, ParticlePeter !
Dec 12 2011
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I have a funky feeling you're reading NeHe? Try this:

https://github.com/AndrejMitrovic/DNeonHelium/blob/master/Samples/win32/lesson25.d#L87
Dec 12 2011
parent ParticlePeter <ParticlePeter gmx.de> writes:
Andrej Mitrovic Wrote:


 I have a funky feeling you're reading NeHe? Try this:


THX, that's very cool, will definitly look deeper into these samples. I know
NeHe, but am following The OpenGL Book, and using DerelictSDL. This shows me
how you do it, but does not answer my questions, my Shader Issue in particular.
I need to add that when I use the Shaders from within my code, defined as a
String, the Shader Program compiles and links always, but when I read the same
Shaders from a file with my method they compile and link randomly. How do you
read in Shaders ?

Cheers, ParticlePeter !
Dec 12 2011
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
ParticlePeter:


   auto fileFrag = cast( GLchar[] )readText( "Shader/Shader_01.frag" ) ;
   const GLchar * fragSource = cast( GLchar * )fileFrag ;


Avoid raw casts every time it is possible. The second cast is not necessary,
using the ".ptr".



 I use a dynamic array, and append each line to my array, which I think is not
the most clever or efficient approach:


Try std.array.appender.



 3.) When reading a file line by line it would be nice to have the count of
Lines of the file ( numLines ) and the line number in a variable, so I could
random access a fixed size array with this line number.


Fixed-sized arrays are often not good if you need to put lot of data inside
them, because the stack has limited space (unless you allocate the fixed sized
array on the heap).



 Can I get all floats ( words as float ) from a file at once without using
file.byLine into a Dynamic array ( or fixed array, I know the word count ) ? 


In theory std.conv.parse is useful for this:

import std.stdio, std.conv;
void main() {
    string s = "[[1,2],[3,4]]";
    auto p = parse!(double[][])(s);
    writeln(p);
}

In practice the signature of one parse overload is:
Target parse(Target, Source)(ref Source s, dchar lbracket = '[', dchar rbracket
= ']', dchar comma = ','); 

lbracket is a char instead of being a string, so I don't know how to parse a
row that lacks commas and brackets. This seems an enhancement request.



 How can I get the Line count from a file,


You probably need to just iterate the file lines, and count them.



 and how can I get the line numbers without initializing a counter in front of
the foreach loop and increase it manually inside the loop ?


With walkLength:


import std.stdio, std.range;
void main() {
    immutable size_t nLines = walkLength(File("data.txt").byLine());
    writeln(nLines);
}

Bye,
bearophile
Dec 12 2011
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
 2.) I am reading in Vertex Data from a file. It holds only 
 floats, e.g.:
 -38.3887 97.7612 -10.5231 -38.3572 98.8543 -10.5064 ...
 There is no metadata in the file giving information about the 
 count of values.
 3.) When reading a file line by line it would be nice to have 
 the count of Lines of the file ( numLines )


If each line takes at least 25 bytes, you can divide file size by 
25 - this will be an estimation of number of lines.
Dec 12 2011
prev sibling parent Simon <s.d.hammett gmail.com> writes:
On 12/12/2011 09:31, ParticlePeter wrote:

 Hi,

 I read several posts about reading in files, but I can't find the "best" way
for my purpose. I am using std.file in the examples bellow.

 1.) I'm trying to read an OpenGL Vertex and Fragment Shader. The wired thing
on my approach is that it works ... sometimes, but sometimes the Shaders cannot
be compiled.
 So how should I read a file as one complete char[] in a safe way ? This is my
way:
    auto fileFrag = cast( GLchar[] )readText( "Shader/Shader_01.frag" ) ;
    const GLchar * fragSource = cast( GLchar * )fileFrag ;


string literals are zero terminated, but strings returned by library 
functions may not be.

Try appending a zero to the read string:

string fragString = readText( "Shader.vert" ) ~ "\0";

-- 
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
Dec 12 2011