• nilesr rodsnet.com Apr 04 2006
• Justin C Calvarese <technocrat7 gmail.com> Apr 04 2006
• BenHinkle <BenHinkle_member pathlink.com> Apr 05 2006
• Niko Korhonen <niktheblak hotmail.com> Apr 06 2006

nilesr rodsnet.com writes:

I am new to D and have been reading documentation. I had a couple question about
the readLine function. After looking through some of the archive posts I had
seen there were some performance issues using this function, have those been
corrected in the current version?
If not does anyone have any suggestion on how to parse a large file by line, or
better yet with specific character rather than just \n? This is for part of a
larger project I am looking to use D for or I would use perl or something to do
it.

Thanks,
Chris

Justin C Calvarese <technocrat7 gmail.com> writes:

nilesr rodsnet.com wrote:
 I am new to D and have been reading documentation. I had a couple question
about
 the readLine function. After looking through some of the archive posts I had
 seen there were some performance issues using this function, have those been
 corrected in the current version?
 If not does anyone have any suggestion on how to parse a large file by line, or
 better yet with specific character rather than just \n? This is for part of a
 larger project I am looking to use D for or I would use perl or something to do
 it.
 
 Thanks,
 Chris


I have a vague memory of the problem in std.stream that you've 
described, and I think it was fixed quite a while ago.

But if you're looking for more horsepower than what std.stream provides 
you might check out the mango.io package in Mango 
(http://www.dsource.org/projects/mango). I haven't tried Mango myself, 
but many people swear by it.

-- 
jcc7

BenHinkle <BenHinkle_member pathlink.com> writes:

In article <e0up1s$2e3j$1 digitaldaemon.com>, nilesr rodsnet.com says...
I am new to D and have been reading documentation. I had a couple question about
the readLine function. After looking through some of the archive posts I had
seen there were some performance issues using this function, have those been
corrected in the current version?
If not does anyone have any suggestion on how to parse a large file by line, or
better yet with specific character rather than just \n? This is for part of a
larger project I am looking to use D for or I would use perl or something to do
it.

Thanks,
Chris


Please post any performance issues you find. I'd guess the posts you refer to
either use unbuffered streams or do not pass a scratch buffer to readLine. I
recommend using the example from the doc as a model
Stream file = new BufferedFile("sample.txt");
foreach(ulong n, char[] line; file) {
stdout.writefln("line %d: %s",n,line);
}
file.close();
the opApply for Stream uses a 128 bytes stack buffer and will only allocate a
temporary buffer from the heap if a line is greater than 128. No memory
allocations happen during that loop in the usual case.

-Ben

Niko Korhonen <niktheblak hotmail.com> writes:

BenHinkle wrote:
 Stream file = new BufferedFile("sample.txt");
 foreach(ulong n, char[] line; file) {
 stdout.writefln("line %d: %s",n,line);
 }
 file.close();
 the opApply for Stream uses a 128 bytes stack buffer and will only allocate a
 temporary buffer from the heap if a line is greater than 128. No memory
 allocations happen during that loop in the usual case.


...which also means that you have to .dup the variable 'line' if you 
plan to use it later. If you only pass a reference to 'line', it's 
contents will change at the next iteration.

-- 
Niko Korhonen
SW Developer