www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - readLine

reply nilesr rodsnet.com writes:
I am new to D and have been reading documentation. I had a couple question about
the readLine function. After looking through some of the archive posts I had
seen there were some performance issues using this function, have those been
corrected in the current version?
If not does anyone have any suggestion on how to parse a large file by line, or
better yet with specific character rather than just \n? This is for part of a
larger project I am looking to use D for or I would use perl or something to do
it.

Thanks,
Chris
Apr 04 2006
next sibling parent Justin C Calvarese <technocrat7 gmail.com> writes:
nilesr rodsnet.com wrote:
 I am new to D and have been reading documentation. I had a couple question
about
 the readLine function. After looking through some of the archive posts I had
 seen there were some performance issues using this function, have those been
 corrected in the current version?
 If not does anyone have any suggestion on how to parse a large file by line, or
 better yet with specific character rather than just \n? This is for part of a
 larger project I am looking to use D for or I would use perl or something to do
 it.
 
 Thanks,
 Chris

I have a vague memory of the problem in std.stream that you've described, and I think it was fixed quite a while ago. But if you're looking for more horsepower than what std.stream provides you might check out the mango.io package in Mango (http://www.dsource.org/projects/mango). I haven't tried Mango myself, but many people swear by it. -- jcc7
Apr 04 2006
prev sibling parent reply BenHinkle <BenHinkle_member pathlink.com> writes:
In article <e0up1s$2e3j$1 digitaldaemon.com>, nilesr rodsnet.com says...
I am new to D and have been reading documentation. I had a couple question about
the readLine function. After looking through some of the archive posts I had
seen there were some performance issues using this function, have those been
corrected in the current version?
If not does anyone have any suggestion on how to parse a large file by line, or
better yet with specific character rather than just \n? This is for part of a
larger project I am looking to use D for or I would use perl or something to do
it.

Thanks,
Chris

Please post any performance issues you find. I'd guess the posts you refer to either use unbuffered streams or do not pass a scratch buffer to readLine. I recommend using the example from the doc as a model Stream file = new BufferedFile("sample.txt"); foreach(ulong n, char[] line; file) { stdout.writefln("line %d: %s",n,line); } file.close(); the opApply for Stream uses a 128 bytes stack buffer and will only allocate a temporary buffer from the heap if a line is greater than 128. No memory allocations happen during that loop in the usual case. -Ben
Apr 05 2006
parent Niko Korhonen <niktheblak hotmail.com> writes:
BenHinkle wrote:
 Stream file = new BufferedFile("sample.txt");
 foreach(ulong n, char[] line; file) {
 stdout.writefln("line %d: %s",n,line);
 }
 file.close();
 the opApply for Stream uses a 128 bytes stack buffer and will only allocate a
 temporary buffer from the heap if a line is greater than 128. No memory
 allocations happen during that loop in the usual case.

...which also means that you have to .dup the variable 'line' if you plan to use it later. If you only pass a reference to 'line', it's contents will change at the next iteration. -- Niko Korhonen SW Developer
Apr 06 2006