www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - stdio revised (writef/format/readf/unformat)

reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
I won't have time to clean it up myself,
but if someone else wants to play with
integrated the revised stdio into Phobos
- I put the source code hacks up at :

http://www.algonet.se/~afb/d/stdio/

Since TypeInfo for all pointer types is
horribly broken (or at least missing...),
there's a pretty ugly workaround present.
I think you will notice it in the sources :-)


Anyway, the way it is meant to work is:

import std.stdio;

// writeln should be familiar to Java users
std.stdio.writeln("100% formatless");

write(100,"%"); //without format
writef("%d%%", 100); // with format

char[] name;
writeln("What is your name?");

read(&name); // without format
readf("%s", &name); // with format

writeln("Hello ", name, ", pleased to meet you");


If you need to access files instead of stdin / stdout,
there are fread and fwrite versions that take a FILE*.


For working with strings, there are:

import std.string;

char[] s;
int i,j;

s = format("%d%d",i,j); // traditional
s = format("%d",i,"%d",j); // mixed
s = format(i,j); // formatless

unformat(s, "%d%d",&i,&j); // traditional
unformat(s, "%d",&i,"%d",&j); // mixed
unformat(s, &i,&j); // formatless


The formatless are simpler for the beginner and
for the common cases, but the formats are very
handy for formatting input/output - especially
for any integers and floating point numbers.


For advanced users, there is also an "unFormat":

module std.unformat;
void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
               TypeInfo[] arguments, va_list argptr);

It is very similar to the old "doFormat" routine:

module std.format;
void doFormat(void delegate(dchar) putc,
               TypeInfo[] arguments, va_list argptr,
               bool ignorePercent = false);

Those are the workhorses of the formatting/unformatting.


unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.

http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

Such format/param errors, will throw "FormatError".
(for now, in want of a revised Exception hierarchy)
That's a change from the original version of unFormat,
also his "sreadf" has now been renamed into "unformat".

--anders


PS.
And the traditional disclaimer here: if you want
C++ << >> stream operators, look at Mango instead ?
(the IReader and IWriter interfaces, in mango.io)

import mango.io.Buffer;
import mango.io.Reader;
import mango.io.Writer;

Buffer buf = new Buffer (256);

// map same buffer into both reader and writer
IReader r = new Reader(buf);
IWriter w = new Writer(buf);

int i = 10;
long j = 20;
double d = 3.14159;
char[] c = "fred";

// write data types out
w << c << i << j << d;

// read them back again
r >> c >> i >> j >> d;

 From http://mango.dsource.org/ (The Mango Tree)
Apr 15 2005
next sibling parent reply "Kris" <fu bar.com> writes:
Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.
Here's an example pulled from the Mango docs:










One nice thing about Mango.io lies in the symmetry for reading things back
again (no pointers):








Note that Stdout & Stdin are instances of a fundamental type in Mango:
IWriter & IReader. Because of this, you apply the same approach for file IO,
socket IO, memory buffers, etc. The 'whisper' format is the unofficial
standard for Mango ~ it's simple, since the only thing that changes between
input and output 'modes' is the verb itself (not the syntax ~ see the two
examples above).

Note also that mango.io handles arrays for all primitive types; not just
char[]. It also supports plugable encoding and decoding options (for wchar &
dchar). Everything operates bi-directionally.


Memory OutBuffer/InBuffer example:












File random-IO example:













Socket example (we'll assume an Echo server on the other end):












There's a printf() style wrapper as well, for doing traditional formatting.
Plus a 'layout' wrapper for ordering text strings. In fact, I should ask
Anders and Sean if mango.io can adopt their formatters/unformatters. How
about it, you guys? Is that cool?

Note the seperation between IO actions and the formatting thereof? Mango has
multiple Reader/Writer instances for doing different kinds of formatting
(text, binary, endian, pickling, etc). This is one way in which mango.io
supports extensibility. Another mechanism supported is the ability to
directly read and write your own classes ~ this maintains encapsulation at
the class level, and supports things like class serialization (mango.cluster
shuttles class instances around a network). You'll also find all the other
usual file oriented functionality in there; laid out in a clear, simple, and
non-redundant fashion.

Mango.io also has a simple, extensible, Tokenizer framework. It handles
serial  tokenizing from an input Buffer, and there are predefined Tokenizers
for splitting upon lines, commas, punctuation, regex, etc.

In general, mango.io is designed for high throughput server-style
environments. But you may find it useful for general purpose usage. Quite a
few others do.

- Kris



"Anders F Björklund" <afb algonet.se> wrote in message
news:d3odh2$aot$1 digitaldaemon.com...

 I won't have time to clean it up myself,
 but if someone else wants to play with
 integrated the revised stdio into Phobos
 - I put the source code hacks up at :

 http://www.algonet.se/~afb/d/stdio/

 Since TypeInfo for all pointer types is
 horribly broken (or at least missing...),
 there's a pretty ugly workaround present.
 I think you will notice it in the sources :-)


 Anyway, the way it is meant to work is:

 import std.stdio;

 // writeln should be familiar to Java users
 std.stdio.writeln("100% formatless");

 write(100,"%"); //without format
 writef("%d%%", 100); // with format

 char[] name;
 writeln("What is your name?");

 read(&name); // without format
 readf("%s", &name); // with format

 writeln("Hello ", name, ", pleased to meet you");


 If you need to access files instead of stdin / stdout,
 there are fread and fwrite versions that take a FILE*.


 For working with strings, there are:

 import std.string;

 char[] s;
 int i,j;

 s = format("%d%d",i,j); // traditional
 s = format("%d",i,"%d",j); // mixed
 s = format(i,j); // formatless

 unformat(s, "%d%d",&i,&j); // traditional
 unformat(s, "%d",&i,"%d",&j); // mixed
 unformat(s, &i,&j); // formatless


 The formatless are simpler for the beginner and
 for the common cases, but the formats are very
 handy for formatting input/output - especially
 for any integers and floating point numbers.


 For advanced users, there is also an "unFormat":

 module std.unformat;
 void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
                TypeInfo[] arguments, va_list argptr);

 It is very similar to the old "doFormat" routine:

 module std.format;
 void doFormat(void delegate(dchar) putc,
                TypeInfo[] arguments, va_list argptr,
                bool ignorePercent = false);

 Those are the workhorses of the formatting/unformatting.


 unformat was written by Sean Kelly, and accepts
 most of the "scanf" format characters - just as
 format accepts most of the "printf" characters.
 Unlike the C variants, they do check the types.

 http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

 Such format/param errors, will throw "FormatError".
 (for now, in want of a revised Exception hierarchy)
 That's a change from the original version of unFormat,
 also his "sreadf" has now been renamed into "unformat".

 --anders


 PS.
 And the traditional disclaimer here: if you want
 C++ << >> stream operators, look at Mango instead ?
 (the IReader and IWriter interfaces, in mango.io)

 import mango.io.Buffer;
 import mango.io.Reader;
 import mango.io.Writer;

 Buffer buf = new Buffer (256);

 // map same buffer into both reader and writer
 IReader r = new Reader(buf);
 IWriter w = new Writer(buf);

 int i = 10;
 long j = 20;
 double d = 3.14159;
 char[] c = "fred";

 // write data types out
 w << c << i << j << d;

 // read them back again
 r >> c >> i >> j >> d;

  From http://mango.dsource.org/ (The Mango Tree)
Apr 15 2005
next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <d3p73i$15gg$1 digitaldaemon.com>, Kris says...

Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.


FWIW, I love the new 'whisper' syntax, but it makes me wonder... what happens
with classes that implement both IReader and IWriter?

As for the rest, I prefer Mango over readf/writef in most instances, though I
think readf/writef do handle cases involving complex formatting rather
succinctly (even if it is nearly unreadable).  For example, here are two unit
tests for unformat:

int i; double x; char name[50];
sreadf( "56789 0123 56a72", "%2d%f%*d %[0-9]", &i, &x, &name[0] );
assert( i == 56 && x == 789.0 && !strcmp( "56", name ) );

double f1, f2, f3;
char c1, c2, c3;
sreadf( "1;-5.9:+0x1pag", "%f%c%f%c%f%c", &f1, &c1, &f2, &c2, &f3, &c3 );
assert( f1 == 1.0 && c1 == ';' && f2 == -5.9 && c2 == ':' && f3 == 1024.0 && c3
== 'g' );

I don't have enough experience with Mango to say, but I know that things like
this are a pain with C++ iostreams.


Sean
Apr 15 2005
parent reply "Kris" <fu bar.com> writes:
"Sean Kelly" <sean f4.ca> wrote ...

 In article <d3p73i$15gg$1 digitaldaemon.com>, Kris says...

Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.


 FWIW, I love the new 'whisper' syntax, but it makes me wonder... what


happens

 with classes that implement both IReader and IWriter?


:-)

The interfaces declare everything using get/put, and then alias the (other)
operators. It would be *interesting* to combine IReader & IWriter into one
class, but I can't think of a good reason to do so. Regardless, it's a point
well made, and well noted.


 As for the rest, I prefer Mango over readf/writef in most instances,


though I

 think readf/writef do handle cases involving complex formatting rather
 succinctly (even if it is nearly unreadable).  For example, here are two


unit

 tests for unformat:

 int i; double x; char name[50];
 sreadf( "56789 0123 56a72", "%2d%f%*d %[0-9]", &i, &x, &name[0] );
 assert( i == 56 && x == 789.0 && !strcmp( "56", name ) );

 double f1, f2, f3;
 char c1, c2, c3;
 sreadf( "1;-5.9:+0x1pag", "%f%c%f%c%f%c", &f1, &c1, &f2, &c2, &f3, &c3 );
 assert( f1 == 1.0 && c1 == ';' && f2 == -5.9 && c2 == ':' && f3 == 1024.0


&& c3

 == 'g' );

 I don't have enough experience with Mango to say, but I know that things


like

 this are a pain with C++ iostreams.


Aye, they are. I've tried various approaches with mango.io, and one that
results in the most cleanliness is binding of classes directly to the IO
system. That way, you can simply read in a number of class instances rather
than a long stream of individual primitives. Plus, everything is
encapsulated nicely. A silly example for illustrative purpose:




























You chain these together in whatever manner works for the application. The
premise is that one generally keeps the values around for a while, so why
not encapsulate them to begin with. Output with classes is performed in a
similar manner. Mango has formatters and parsers for doing format-twiddling
similar to printf/scanf (mango.io does not use printf internally)

That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader &
FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various works
you (and Anders) have made available?

- Kris
Apr 15 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <d3pf1f$1bpe$1 digitaldaemon.com>, Kris says...

That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader &
FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various works
you (and Anders) have made available?


Feel free.  The code was a pain to get right, and I wrote it intending it to be
wrapped by other stuff anyway.


Sean
Apr 15 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <d3phbd$1djm$1 digitaldaemon.com>, Sean Kelly says...

In article <d3pf1f$1bpe$1 digitaldaemon.com>, Kris says...

That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader &
FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various works
you (and Anders) have made available?


Feel free.  The code was a pain to get right, and I wrote it intending it to be
wrapped by other stuff anyway.


Just a quick follow-up.  I checked the new source and a ton of changes have been
made from the version I released.  I trust it all works as advertised, but it's
sufficiently unfamiliar that you're better off asking Anders about it if you
have any specific questions.  My old version is still available in the usual
spot as well.


Sean
Apr 15 2005
parent "Kris" <fu bar.com> writes:
Thanks, Sean.

"Sean Kelly" <sean f4.ca> wrote in message
news:d3pm92$1h8v$1 digitaldaemon.com...

 In article <d3phbd$1djm$1 digitaldaemon.com>, Sean Kelly says...

In article <d3pf1f$1bpe$1 digitaldaemon.com>, Kris says...

That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader






&

FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various






works

you (and Anders) have made available?


Feel free.  The code was a pain to get right, and I wrote it intending it




to be

wrapped by other stuff anyway.


 Just a quick follow-up.  I checked the new source and a ton of changes


have been

 made from the version I released.  I trust it all works as advertised, but


it's

 sufficiently unfamiliar that you're better off asking Anders about it if


you

 have any specific questions.  My old version is still available in the


usual

 spot as well.


 Sean
Apr 15 2005
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Kris wrote:


unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.


 There's a printf() style wrapper as well, for doing traditional formatting.
 Plus a 'layout' wrapper for ordering text strings. In fact, I should ask
 Anders and Sean if mango.io can adopt their formatters/unformatters. How
 about it, you guys? Is that cool?


Sure, Sean wrote most of it - and I just massacred it to fit into a new 
API... :-) And I'm not sure I got it all right yet, since it was using 
Exceptions for flow handling and some stuff I didn't really figure out.

But there was a pretty permissive license on Seans stuff in unformat,
my own stuff is under the same license as Phobos: zlib/png or even PD.


So, yes, it's cool... (I didn't really change so much of the formatting
internals, except for changed delegate definitions and new exceptions)

In case it wasn't clear, it's not really working correctly or tested,
just yet - and in any way, the pointer stuff wants new TypeInfo first ?

--anders
Apr 16 2005
prev sibling next sibling parent Sean Kelly <sean f4.ca> writes:
In article <d3odh2$aot$1 digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...

unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.


FWIW, unformat/readf is completely compliant with the C99 spec for scanf (as far
as I know).  The only difference is that this revised version of unformat throws
exceptions in the same instances where format throws exceptions, while scanf
obviously never throws exceptions :)  The unit tests are somewhat sparse, but
I'm not aware of any bugs in the original implementation (and I don't expect any
in this re-release, though I haven't seen the code yet).  Questions are probably
best directed to this newsgroup.


Sean
Apr 15 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
Cool!

What is the status of unformat.d?
Will it be added to standard phobos?

Andrew.


"Anders F Björklund" <afb algonet.se> wrote in message 
news:d3odh2$aot$1 digitaldaemon.com...

I won't have time to clean it up myself,
 but if someone else wants to play with
 integrated the revised stdio into Phobos
 - I put the source code hacks up at :

 http://www.algonet.se/~afb/d/stdio/

 Since TypeInfo for all pointer types is
 horribly broken (or at least missing...),
 there's a pretty ugly workaround present.
 I think you will notice it in the sources :-)


 Anyway, the way it is meant to work is:

 import std.stdio;

 // writeln should be familiar to Java users
 std.stdio.writeln("100% formatless");

 write(100,"%"); //without format
 writef("%d%%", 100); // with format

 char[] name;
 writeln("What is your name?");

 read(&name); // without format
 readf("%s", &name); // with format

 writeln("Hello ", name, ", pleased to meet you");


 If you need to access files instead of stdin / stdout,
 there are fread and fwrite versions that take a FILE*.


 For working with strings, there are:

 import std.string;

 char[] s;
 int i,j;

 s = format("%d%d",i,j); // traditional
 s = format("%d",i,"%d",j); // mixed
 s = format(i,j); // formatless

 unformat(s, "%d%d",&i,&j); // traditional
 unformat(s, "%d",&i,"%d",&j); // mixed
 unformat(s, &i,&j); // formatless


 The formatless are simpler for the beginner and
 for the common cases, but the formats are very
 handy for formatting input/output - especially
 for any integers and floating point numbers.


 For advanced users, there is also an "unFormat":

 module std.unformat;
 void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
               TypeInfo[] arguments, va_list argptr);

 It is very similar to the old "doFormat" routine:

 module std.format;
 void doFormat(void delegate(dchar) putc,
               TypeInfo[] arguments, va_list argptr,
               bool ignorePercent = false);

 Those are the workhorses of the formatting/unformatting.


 unformat was written by Sean Kelly, and accepts
 most of the "scanf" format characters - just as
 format accepts most of the "printf" characters.
 Unlike the C variants, they do check the types.

 http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

 Such format/param errors, will throw "FormatError".
 (for now, in want of a revised Exception hierarchy)
 That's a change from the original version of unFormat,
 also his "sreadf" has now been renamed into "unformat".

 --anders


 PS.
 And the traditional disclaimer here: if you want
 C++ << >> stream operators, look at Mango instead ?
 (the IReader and IWriter interfaces, in mango.io)

 import mango.io.Buffer;
 import mango.io.Reader;
 import mango.io.Writer;

 Buffer buf = new Buffer (256);

 // map same buffer into both reader and writer
 IReader r = new Reader(buf);
 IWriter w = new Writer(buf);

 int i = 10;
 long j = 20;
 double d = 3.14159;
 char[] c = "fred";

 // write data types out
 w << c << i << j << d;

 // read them back again
 r >> c >> i >> j >> d;

 From http://mango.dsource.org/ (The Mango Tree)
Apr 15 2005
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Andrew Fedoniouk wrote:


 What is the status of unformat.d?


Something of an "early alpha", at least my changed version.
Seems to pass the quick unittest I did, but probably buggy.


 Will it be added to standard phobos?


There's nothing really stopping that, once it's completed ?
I thought the API came out nicely, the reason for posting it.

It needs to be merged with the new DMD 0.120/0.121, as well.

--anders
Apr 16 2005