www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Buffered output

reply =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak invalid_utu.fi> writes:
I need to create a very simple program module that prints some runtime 
generated strings to the standard output. Ok, that's not a problem. The 
real problem is that I need to output these as fast as possible. The 
output consists of big string blocks (>100.000 chars) that mostly 
contain only two different characters.

I've ported the program from C. The original program doesn't use any 
sort of buffering, just some printf:s like for(i=1;i<n;i++) if 
(foobar()) printf("*.-");

Here are some simple benchmarks using the GNU time utility:


dmd test.c -o test -O3 -s
time ./test_c_printf > /dev/null
real	0m0.117s

dmd test.d -O -release -inline -version=printf
time ./test > /dev/null
real	0m0.140s

dmd test.d -O -release -inline -version=writef
time ./test > /dev/null
real	0m0.225s

How come writef() is so much slower than printf()?
Is there a fast way to do buffered output. I tried to use dynamic & 
static arrays and printf the full arrays, but that wasn't actually any 
faster.
Oct 24 2005
next sibling parent "Kris" <fu bar.com> writes:
You may wish to try Mango.io (Stdout)?  The IO model is designed to take 
advantage of buffering, and I once ran into a problem with the Win32 console 
where it barfed on more than 32K at one time :-)

To illustrate, there's a specific subclass to handle that condition at the 
end of this module: 
http://svn.dsource.org/projects/mango/trunk/mango/io/DeviceConduit.d

Mango.io has formatting options similar to printf(), the full gamut of 
discrete IO methods, and a whole lot more besides. The mango.io equivalent 
to writef() should be faster in that it emits arrays of utf8, rather than 
one dchar at a time; and Stdout employs buffering by default. It's worth 
taking a look here http://mango.dsource.org/ and here 
http://www.dsource.org/forums/viewtopic.php?t=148

Those who prefer OOP may well prefer Mango.io over the phobos IO package(s) 
~ they're designed for different audiences.

- Kris


"Jari-Matti Mäkelä" <jmjmak invalid_utu.fi> wrote in message 
news:djjlb6$1l22$1 digitaldaemon.com...
I need to create a very simple program module that prints some runtime 
generated strings to the standard output. Ok, that's not a problem. The 
real problem is that I need to output these as fast as possible. The output 
consists of big string blocks (>100.000 chars) that mostly contain only two 
different characters.

 I've ported the program from C. The original program doesn't use any sort 
 of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar()) 
 printf("*.-");

 Here are some simple benchmarks using the GNU time utility:


 dmd test.c -o test -O3 -s
 time ./test_c_printf > /dev/null
 real 0m0.117s

 dmd test.d -O -release -inline -version=printf
 time ./test > /dev/null
 real 0m0.140s

 dmd test.d -O -release -inline -version=writef
 time ./test > /dev/null
 real 0m0.225s

 How come writef() is so much slower than printf()?
 Is there a fast way to do buffered output. I tried to use dynamic & static 
 arrays and printf the full arrays, but that wasn't actually any faster. 

Oct 24 2005
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <djjlb6$1l22$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= says...
I need to create a very simple program module that prints some runtime 
generated strings to the standard output. Ok, that's not a problem. The 
real problem is that I need to output these as fast as possible. The 
output consists of big string blocks (>100.000 chars) that mostly 
contain only two different characters.

How come writef() is so much slower than printf()?

writef does some fancy things that printf doesn't, though I had expected performance to be comparable to printf for ASCII character strings. The implementation has changed since I last looked at it, but you might want to look at the source for writef in std.format.d (doFormat) to see if you can find any hangups.
Is there a fast way to do buffered output. I tried to use dynamic & 
static arrays and printf the full arrays, but that wasn't actually any 
faster.

If you just want to print ASCII strings as fast as possible, printf and writef are probably not ideal. You may be better off with puts or fwrite, and possibly even messing with the buffer settings using setbuf/setvbuf. Sean
Oct 24 2005
parent =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak invalid_utu.fi> writes:
Sean Kelly wrote:
How come writef() is so much slower than printf()?

writef does some fancy things that printf doesn't, though I had expected performance to be comparable to printf for ASCII character strings. The implementation has changed since I last looked at it, but you might want to look at the source for writef in std.format.d (doFormat) to see if you can find any hangups.

std.format.d is too complex for me to optimize. I need to take a second look at it sometime, but for now another approach may be more useful.
Oct 25 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
printf and writef are already buffered - I believe stdout is line-buffered. 
Posting the actual code might help people help you. I also recommend using 
the -profile compiler option and/or std.perf. If you like you can also wrap 
a std.stream.Stream with a std.stream.BufferedStream but I suspect that you 
can do what you want with regular C buffering. See for example setvbuf.

"Jari-Matti Mäkelä" <jmjmak invalid_utu.fi> wrote in message 
news:djjlb6$1l22$1 digitaldaemon.com...
I need to create a very simple program module that prints some runtime 
generated strings to the standard output. Ok, that's not a problem. The 
real problem is that I need to output these as fast as possible. The output 
consists of big string blocks (>100.000 chars) that mostly contain only two 
different characters.

 I've ported the program from C. The original program doesn't use any sort 
 of buffering, just some printf:s like for(i=1;i<n;i++) if (foobar()) 
 printf("*.-");

 Here are some simple benchmarks using the GNU time utility:


 dmd test.c -o test -O3 -s
 time ./test_c_printf > /dev/null
 real 0m0.117s

 dmd test.d -O -release -inline -version=printf
 time ./test > /dev/null
 real 0m0.140s

 dmd test.d -O -release -inline -version=writef
 time ./test > /dev/null
 real 0m0.225s

 How come writef() is so much slower than printf()?
 Is there a fast way to do buffered output. I tried to use dynamic & static 
 arrays and printf the full arrays, but that wasn't actually any faster. 

Oct 24 2005
parent =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak invalid_utu.fi> writes:
Ben Hinkle wrote:
 printf and writef are already buffered - I believe stdout is line-buffered. 
 Posting the actual code might help people help you. I also recommend using 
 the -profile compiler option and/or std.perf. If you like you can also wrap 
 a std.stream.Stream with a std.stream.BufferedStream but I suspect that you 
 can do what you want with regular C buffering. See for example setvbuf.
 

Yes, you're right. writef & printf are already buffered. One can see this with multithreaded CLI applications. Unfortunately the original code isn't mine. I need to ask for permission if I want to post some code here. Ok, I'll give a shot at trying the std.stream-functions. Thanks. The strangest thing here is that according to the time program the c-version does much greater amount of work in kernel mode. How is it possible at all that the D program seems to do the same I/O in user mode?
Oct 25 2005