www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - stdio revised (writef/format/readf/unformat)

reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
I won't have time to clean it up myself,
but if someone else wants to play with
integrated the revised stdio into Phobos
- I put the source code hacks up at :

http://www.algonet.se/~afb/d/stdio/

Since TypeInfo for all pointer types is
horribly broken (or at least missing...),
there's a pretty ugly workaround present.
I think you will notice it in the sources :-)


Anyway, the way it is meant to work is:

import std.stdio;

// writeln should be familiar to Java users
std.stdio.writeln("100% formatless");

write(100,"%"); //without format
writef("%d%%", 100); // with format

char[] name;
writeln("What is your name?");

read(&name); // without format
readf("%s", &name); // with format

writeln("Hello ", name, ", pleased to meet you");


If you need to access files instead of stdin / stdout,
there are fread and fwrite versions that take a FILE*.


For working with strings, there are:

import std.string;

char[] s;
int i,j;

s = format("%d%d",i,j); // traditional
s = format("%d",i,"%d",j); // mixed
s = format(i,j); // formatless

unformat(s, "%d%d",&i,&j); // traditional
unformat(s, "%d",&i,"%d",&j); // mixed
unformat(s, &i,&j); // formatless


The formatless are simpler for the beginner and
for the common cases, but the formats are very
handy for formatting input/output - especially
for any integers and floating point numbers.


For advanced users, there is also an "unFormat":

module std.unformat;
void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
               TypeInfo[] arguments, va_list argptr);

It is very similar to the old "doFormat" routine:

module std.format;
void doFormat(void delegate(dchar) putc,
               TypeInfo[] arguments, va_list argptr,
               bool ignorePercent = false);

Those are the workhorses of the formatting/unformatting.


unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.

http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

Such format/param errors, will throw "FormatError".
(for now, in want of a revised Exception hierarchy)
That's a change from the original version of unFormat,
also his "sreadf" has now been renamed into "unformat".

--anders


PS.
And the traditional disclaimer here: if you want
C++ << >> stream operators, look at Mango instead ?
(the IReader and IWriter interfaces, in mango.io)

import mango.io.Buffer;
import mango.io.Reader;
import mango.io.Writer;

Buffer buf = new Buffer (256);

// map same buffer into both reader and writer
IReader r = new Reader(buf);
IWriter w = new Writer(buf);

int i = 10;
long j = 20;
double d = 3.14159;
char[] c = "fred";

// write data types out
w << c << i << j << d;

// read them back again
r >> c >> i >> j >> d;

 From http://mango.dsource.org/ (The Mango Tree)
Apr 15 2005
next sibling parent reply "Kris" <fu bar.com> writes:
Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.
Here's an example pulled from the Mango docs:

#   int i = 10;
#  char[] msg = "green-bottles";
#
#  Stdout.put(i).put(msg);
#
#  Stdout << i << msg;
#
#  Stdout (i) (msg);

One nice thing about Mango.io lies in the symmetry for reading things back
again (no pointers):

#  Stdin.get(i).get(msg);
#
#  Stdin >> i >> msg;
#
#  Stdin (i) (msg);


Note that Stdout & Stdin are instances of a fundamental type in Mango:
IWriter & IReader. Because of this, you apply the same approach for file IO,
socket IO, memory buffers, etc. The 'whisper' format is the unofficial
standard for Mango ~ it's simple, since the only thing that changes between
input and output 'modes' is the verb itself (not the syntax ~ see the two
examples above).

Note also that mango.io handles arrays for all primitive types; not just
char[]. It also supports plugable encoding and decoding options (for wchar &
dchar). Everything operates bi-directionally.


Memory OutBuffer/InBuffer example:

#  Buffer b = new Buffer;
#
#  // write to buffer
#  Writer output = new Writer (b);
#  output (i) (msg);
#
#  // read back from buffer
#  Reader input = new Reader (b);
#  input (i) (msg);


File random-IO example:

#  FileConduit file = new FileConduit ("myfile.txt");
#
#  // write to file, and flush output
#  Writer output = new Writer (file);
#  output (i) (msg) ();
#
#  file.seek (0);
#
#  Reader input = new Reader (file);
#  input (i) (msg);


Socket example (we'll assume an Echo server on the other end):

#  SocketConduit socket = new SocketConduit;
#  socket.connect (new InternetAddress("some Echo server"));
#
#  // write to socket, and flush output
#  Writer output = new Writer (socket);
#  output (i) (msg) ();
#
#  Reader input = new Reader (socket);
#  input (i) (msg);


There's a printf() style wrapper as well, for doing traditional formatting.
Plus a 'layout' wrapper for ordering text strings. In fact, I should ask
Anders and Sean if mango.io can adopt their formatters/unformatters. How
about it, you guys? Is that cool?

Note the seperation between IO actions and the formatting thereof? Mango has
multiple Reader/Writer instances for doing different kinds of formatting
(text, binary, endian, pickling, etc). This is one way in which mango.io
supports extensibility. Another mechanism supported is the ability to
directly read and write your own classes ~ this maintains encapsulation at
the class level, and supports things like class serialization (mango.cluster
shuttles class instances around a network). You'll also find all the other
usual file oriented functionality in there; laid out in a clear, simple, and
non-redundant fashion.

Mango.io also has a simple, extensible, Tokenizer framework. It handles
serial  tokenizing from an input Buffer, and there are predefined Tokenizers
for splitting upon lines, commas, punctuation, regex, etc.

In general, mango.io is designed for high throughput server-style
environments. But you may find it useful for general purpose usage. Quite a
few others do.

- Kris



"Anders F Björklund" <afb algonet.se> wrote in message
news:d3odh2$aot$1 digitaldaemon.com...
 I won't have time to clean it up myself,
 but if someone else wants to play with
 integrated the revised stdio into Phobos
 - I put the source code hacks up at :

 http://www.algonet.se/~afb/d/stdio/

 Since TypeInfo for all pointer types is
 horribly broken (or at least missing...),
 there's a pretty ugly workaround present.
 I think you will notice it in the sources :-)


 Anyway, the way it is meant to work is:

 import std.stdio;

 // writeln should be familiar to Java users
 std.stdio.writeln("100% formatless");

 write(100,"%"); //without format
 writef("%d%%", 100); // with format

 char[] name;
 writeln("What is your name?");

 read(&name); // without format
 readf("%s", &name); // with format

 writeln("Hello ", name, ", pleased to meet you");


 If you need to access files instead of stdin / stdout,
 there are fread and fwrite versions that take a FILE*.


 For working with strings, there are:

 import std.string;

 char[] s;
 int i,j;

 s = format("%d%d",i,j); // traditional
 s = format("%d",i,"%d",j); // mixed
 s = format(i,j); // formatless

 unformat(s, "%d%d",&i,&j); // traditional
 unformat(s, "%d",&i,"%d",&j); // mixed
 unformat(s, &i,&j); // formatless


 The formatless are simpler for the beginner and
 for the common cases, but the formats are very
 handy for formatting input/output - especially
 for any integers and floating point numbers.


 For advanced users, there is also an "unFormat":

 module std.unformat;
 void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
                TypeInfo[] arguments, va_list argptr);

 It is very similar to the old "doFormat" routine:

 module std.format;
 void doFormat(void delegate(dchar) putc,
                TypeInfo[] arguments, va_list argptr,
                bool ignorePercent = false);

 Those are the workhorses of the formatting/unformatting.


 unformat was written by Sean Kelly, and accepts
 most of the "scanf" format characters - just as
 format accepts most of the "printf" characters.
 Unlike the C variants, they do check the types.

 http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

 Such format/param errors, will throw "FormatError".
 (for now, in want of a revised Exception hierarchy)
 That's a change from the original version of unFormat,
 also his "sreadf" has now been renamed into "unformat".

 --anders


 PS.
 And the traditional disclaimer here: if you want
 C++ << >> stream operators, look at Mango instead ?
 (the IReader and IWriter interfaces, in mango.io)

 import mango.io.Buffer;
 import mango.io.Reader;
 import mango.io.Writer;

 Buffer buf = new Buffer (256);

 // map same buffer into both reader and writer
 IReader r = new Reader(buf);
 IWriter w = new Writer(buf);

 int i = 10;
 long j = 20;
 double d = 3.14159;
 char[] c = "fred";

 // write data types out
 w << c << i << j << d;

 // read them back again
 r >> c >> i >> j >> d;

  From http://mango.dsource.org/ (The Mango Tree)

Apr 15 2005
next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <d3p73i$15gg$1 digitaldaemon.com>, Kris says...
Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.

FWIW, I love the new 'whisper' syntax, but it makes me wonder... what happens with classes that implement both IReader and IWriter? As for the rest, I prefer Mango over readf/writef in most instances, though I think readf/writef do handle cases involving complex formatting rather succinctly (even if it is nearly unreadable). For example, here are two unit tests for unformat: int i; double x; char name[50]; sreadf( "56789 0123 56a72", "%2d%f%*d %[0-9]", &i, &x, &name[0] ); assert( i == 56 && x == 789.0 && !strcmp( "56", name ) ); double f1, f2, f3; char c1, c2, c3; sreadf( "1;-5.9:+0x1pag", "%f%c%f%c%f%c", &f1, &c1, &f2, &c2, &f3, &c3 ); assert( f1 == 1.0 && c1 == ';' && f2 == -5.9 && c2 == ':' && f3 == 1024.0 && c3 == 'g' ); I don't have enough experience with Mango to say, but I know that things like this are a pain with C++ iostreams. Sean
Apr 15 2005
parent reply "Kris" <fu bar.com> writes:
"Sean Kelly" <sean f4.ca> wrote ...
 In article <d3p73i$15gg$1 digitaldaemon.com>, Kris says...
Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.

FWIW, I love the new 'whisper' syntax, but it makes me wonder... what

 with classes that implement both IReader and IWriter?

:-) The interfaces declare everything using get/put, and then alias the (other) operators. It would be *interesting* to combine IReader & IWriter into one class, but I can't think of a good reason to do so. Regardless, it's a point well made, and well noted.
 As for the rest, I prefer Mango over readf/writef in most instances,

 think readf/writef do handle cases involving complex formatting rather
 succinctly (even if it is nearly unreadable).  For example, here are two

 tests for unformat:

 int i; double x; char name[50];
 sreadf( "56789 0123 56a72", "%2d%f%*d %[0-9]", &i, &x, &name[0] );
 assert( i == 56 && x == 789.0 && !strcmp( "56", name ) );

 double f1, f2, f3;
 char c1, c2, c3;
 sreadf( "1;-5.9:+0x1pag", "%f%c%f%c%f%c", &f1, &c1, &f2, &c2, &f3, &c3 );
 assert( f1 == 1.0 && c1 == ';' && f2 == -5.9 && c2 == ':' && f3 == 1024.0

 == 'g' );

 I don't have enough experience with Mango to say, but I know that things

 this are a pain with C++ iostreams.

Aye, they are. I've tried various approaches with mango.io, and one that results in the most cleanliness is binding of classes directly to the IO system. That way, you can simply read in a number of class instances rather than a long stream of individual primitives. Plus, everything is encapsulated nicely. A silly example for illustrative purpose: # class Foo : IReadable {} # { # char[] name, address; # double salary; # # void read (IReader input) # { # input (name) (address) (salary); # } # } # # class Bar : IReadable # { # char[] header; # Foo f1, f2, f3; # ... # void read (IReader input) # { # input (header) (f1) (f2) (f3); # } # } # # IReader input; # # input (new Bar); You chain these together in whatever manner works for the application. The premise is that one generally keeps the values around for a while, so why not encapsulate them to begin with. Output with classes is performed in a similar manner. Mango has formatters and parsers for doing format-twiddling similar to printf/scanf (mango.io does not use printf internally) That aside; readf() is truly handy. I've been trying to get similar functionality working that doesn't require the & operator (FormatReader & FormatWriter), and it's looking good. But I'll probably also hookup the traditional functions as an option too, if I may leverage the various works you (and Anders) have made available? - Kris
Apr 15 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <d3pf1f$1bpe$1 digitaldaemon.com>, Kris says...
That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader &
FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various works
you (and Anders) have made available?

Feel free. The code was a pain to get right, and I wrote it intending it to be wrapped by other stuff anyway. Sean
Apr 15 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <d3phbd$1djm$1 digitaldaemon.com>, Sean Kelly says...
In article <d3pf1f$1bpe$1 digitaldaemon.com>, Kris says...
That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader &
FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various works
you (and Anders) have made available?

Feel free. The code was a pain to get right, and I wrote it intending it to be wrapped by other stuff anyway.

Just a quick follow-up. I checked the new source and a ton of changes have been made from the version I released. I trust it all works as advertised, but it's sufficiently unfamiliar that you're better off asking Anders about it if you have any specific questions. My old version is still available in the usual spot as well. Sean
Apr 15 2005
parent "Kris" <fu bar.com> writes:
Thanks, Sean.

"Sean Kelly" <sean f4.ca> wrote in message
news:d3pm92$1h8v$1 digitaldaemon.com...
 In article <d3phbd$1djm$1 digitaldaemon.com>, Sean Kelly says...
In article <d3pf1f$1bpe$1 digitaldaemon.com>, Kris says...
That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader



FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various



you (and Anders) have made available?

Feel free. The code was a pain to get right, and I wrote it intending it


wrapped by other stuff anyway.

Just a quick follow-up. I checked the new source and a ton of changes

 made from the version I released.  I trust it all works as advertised, but

 sufficiently unfamiliar that you're better off asking Anders about it if

 have any specific questions.  My old version is still available in the

 spot as well.


 Sean

Apr 15 2005
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Kris wrote:

unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.

There's a printf() style wrapper as well, for doing traditional formatting. Plus a 'layout' wrapper for ordering text strings. In fact, I should ask Anders and Sean if mango.io can adopt their formatters/unformatters. How about it, you guys? Is that cool?

Sure, Sean wrote most of it - and I just massacred it to fit into a new API... :-) And I'm not sure I got it all right yet, since it was using Exceptions for flow handling and some stuff I didn't really figure out. But there was a pretty permissive license on Seans stuff in unformat, my own stuff is under the same license as Phobos: zlib/png or even PD. So, yes, it's cool... (I didn't really change so much of the formatting internals, except for changed delegate definitions and new exceptions) In case it wasn't clear, it's not really working correctly or tested, just yet - and in any way, the pointer stuff wants new TypeInfo first ? --anders
Apr 16 2005
prev sibling next sibling parent Sean Kelly <sean f4.ca> writes:
In article <d3odh2$aot$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.

FWIW, unformat/readf is completely compliant with the C99 spec for scanf (as far as I know). The only difference is that this revised version of unformat throws exceptions in the same instances where format throws exceptions, while scanf obviously never throws exceptions :) The unit tests are somewhat sparse, but I'm not aware of any bugs in the original implementation (and I don't expect any in this re-release, though I haven't seen the code yet). Questions are probably best directed to this newsgroup. Sean
Apr 15 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
Cool!

What is the status of unformat.d?
Will it be added to standard phobos?

Andrew.


"Anders F Björklund" <afb algonet.se> wrote in message 
news:d3odh2$aot$1 digitaldaemon.com...
I won't have time to clean it up myself,
 but if someone else wants to play with
 integrated the revised stdio into Phobos
 - I put the source code hacks up at :

 http://www.algonet.se/~afb/d/stdio/

 Since TypeInfo for all pointer types is
 horribly broken (or at least missing...),
 there's a pretty ugly workaround present.
 I think you will notice it in the sources :-)


 Anyway, the way it is meant to work is:

 import std.stdio;

 // writeln should be familiar to Java users
 std.stdio.writeln("100% formatless");

 write(100,"%"); //without format
 writef("%d%%", 100); // with format

 char[] name;
 writeln("What is your name?");

 read(&name); // without format
 readf("%s", &name); // with format

 writeln("Hello ", name, ", pleased to meet you");


 If you need to access files instead of stdin / stdout,
 there are fread and fwrite versions that take a FILE*.


 For working with strings, there are:

 import std.string;

 char[] s;
 int i,j;

 s = format("%d%d",i,j); // traditional
 s = format("%d",i,"%d",j); // mixed
 s = format(i,j); // formatless

 unformat(s, "%d%d",&i,&j); // traditional
 unformat(s, "%d",&i,"%d",&j); // mixed
 unformat(s, &i,&j); // formatless


 The formatless are simpler for the beginner and
 for the common cases, but the formats are very
 handy for formatting input/output - especially
 for any integers and floating point numbers.


 For advanced users, there is also an "unFormat":

 module std.unformat;
 void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
               TypeInfo[] arguments, va_list argptr);

 It is very similar to the old "doFormat" routine:

 module std.format;
 void doFormat(void delegate(dchar) putc,
               TypeInfo[] arguments, va_list argptr,
               bool ignorePercent = false);

 Those are the workhorses of the formatting/unformatting.


 unformat was written by Sean Kelly, and accepts
 most of the "scanf" format characters - just as
 format accepts most of the "printf" characters.
 Unlike the C variants, they do check the types.

 http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

 Such format/param errors, will throw "FormatError".
 (for now, in want of a revised Exception hierarchy)
 That's a change from the original version of unFormat,
 also his "sreadf" has now been renamed into "unformat".

 --anders


 PS.
 And the traditional disclaimer here: if you want
 C++ << >> stream operators, look at Mango instead ?
 (the IReader and IWriter interfaces, in mango.io)

 import mango.io.Buffer;
 import mango.io.Reader;
 import mango.io.Writer;

 Buffer buf = new Buffer (256);

 // map same buffer into both reader and writer
 IReader r = new Reader(buf);
 IWriter w = new Writer(buf);

 int i = 10;
 long j = 20;
 double d = 3.14159;
 char[] c = "fred";

 // write data types out
 w << c << i << j << d;

 // read them back again
 r >> c >> i >> j >> d;

 From http://mango.dsource.org/ (The Mango Tree) 

Apr 15 2005
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Andrew Fedoniouk wrote:

 What is the status of unformat.d?

Something of an "early alpha", at least my changed version. Seems to pass the quick unittest I did, but probably buggy.
 Will it be added to standard phobos?

There's nothing really stopping that, once it's completed ? I thought the API came out nicely, the reason for posting it. It needs to be merged with the new DMD 0.120/0.121, as well. --anders
Apr 16 2005