www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Streams: an open discussion

reply Sean Kelly <sean f4.ca> writes:
I think the D streams provide a good base but are missing some of the more
complex features that I miss from C++, mostly to do with formatted i/o.  This
raises the rather large issue of localization which I'm going to ignore for the
moment.

As things stand, the basic D stream framework has an InputStream and
OutputStream interface and then a single Stream class that implements both of
these.  File, MemoryStream, SliceStream, and BufferedStream all inherit from
Stream, while BufferedFile inherits from BufferedStream.  Stream and File do not
buffer by default, BufferedStream acts as an adaptor class, and BufferedFile is
the buffered version of the File stream class.  Currently, all formatted i/o
goes through the printf method, with the read/write methods offering unformatted
i/o.  There is also an OutBuffer class (though no corresponding InBuffer) which
seems functionally similar to MemoryStream.

I'd like separate InputStream, OutputStream, and Stream base classes to allow
support for streams where only input or output makes sense.  To make it clear
these are base classes and prevent naming problems with the interfaces I propose
these names: InputStreamBase, OutputStreamBase, and StreamBase (or
InputOutputStreamBase, though that's pretty darn verbose).  Because of the
single inheritance rule, StreamBase won't be able to inherit from
InputStreamBase and OutputStreamBase which will liekly result in some toying
with the design to avoid code duplication.  These classes will be obviously be
abstract base classes.  If I wanted to nit-pick I might suggest renaming File to
FileStream for consistency.

In addition to the existing functionality I propose new formatted i/o methods
for all types that already have unformatted methods.  Apart from a few instances
these can forward most of the work to the existing printf method.  The default
methods would be "get" and "put" with the possibility of adding overloaded
operator support in the future.  I'd also like to see something similar to the
callback mechanism in C++ streams so arbitrary classes can hook in formatting
information.  And Jill had mentioned wanting something similar to readsome()
from C++ streams so I suspect this list will grow a bit.

As for open issues, someone with more experience than myself should probably
address localization.  At the very least, there should be some support for
proper numeric formatting, but that raises the issue of how formatting rules
will be supplied to the stream classes and what else the localization rules
should cover.

I'm willing to go and do this myself, but as it's a modification of existing
Phobos stuff I wanted to get some feedback.  Does anyone have any objections or
suggestions?  Assuming the design doesn't suck will it have a chance for
inclusion?


Sean
Jun 09 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Sean Kelly wrote:

 I think the D streams provide a good base but are missing some of the more
 complex features that I miss from C++, mostly to do with formatted i/o. 
 This raises the rather large issue of localization which I'm going to
 ignore for the moment.
 
 As things stand, the basic D stream framework has an InputStream and
 OutputStream interface and then a single Stream class that implements both
 of
 these.  File, MemoryStream, SliceStream, and BufferedStream all inherit
 from
 Stream, while BufferedFile inherits from BufferedStream.  Stream and File
 do not buffer by default, BufferedStream acts as an adaptor class, and
 BufferedFile is
 the buffered version of the File stream class.  Currently, all formatted
 i/o goes through the printf method, with the read/write methods offering
 unformatted
 i/o.  There is also an OutBuffer class (though no corresponding InBuffer)
 which seems functionally similar to MemoryStream.
 
 I'd like separate InputStream, OutputStream, and Stream base classes to
 allow
 support for streams where only input or output makes sense.  To make it
 clear these are base classes and prevent naming problems with the
 interfaces I propose these names: InputStreamBase, OutputStreamBase, and
 StreamBase (or
 InputOutputStreamBase, though that's pretty darn verbose).  Because of the
 single inheritance rule, StreamBase won't be able to inherit from
 InputStreamBase and OutputStreamBase which will liekly result in some
 toying
 with the design to avoid code duplication.  These classes will be
 obviously be
 abstract base classes.  If I wanted to nit-pick I might suggest renaming
 File to FileStream for consistency.
 
 In addition to the existing functionality I propose new formatted i/o
 methods
 for all types that already have unformatted methods.  Apart from a few
 instances
 these can forward most of the work to the existing printf method.  The
 default methods would be "get" and "put" with the possibility of adding
 overloaded
 operator support in the future.  I'd also like to see something similar to
 the callback mechanism in C++ streams so arbitrary classes can hook in
 formatting
 information.  And Jill had mentioned wanting something similar to
 readsome() from C++ streams so I suspect this list will grow a bit.
 
 As for open issues, someone with more experience than myself should
 probably
 address localization.  At the very least, there should be some support for
 proper numeric formatting, but that raises the issue of how formatting
 rules will be supplied to the stream classes and what else the
 localization rules should cover.
 
 I'm willing to go and do this myself, but as it's a modification of
 existing
 Phobos stuff I wanted to get some feedback.  Does anyone have any
 objections or
 suggestions?  Assuming the design doesn't suck will it have a chance for
 inclusion?
 
 
 Sean

It would be nice to have mixins DefaultInputStream and DefaultOutputStream that would contain the default implementations from Stream and then Stream would be a mixin of those and the seeking and other odds and ends. That would make it easier to implement InputStream and OutputStream since users would just have to supply readBlock and writeBlock, respectively. So for example to define a FooInputStream class FooInputStream : InputStream { mixin DefaultInputStream; // ultimately call readBlock uint readBlock(...) {...} ... whatever else ... }
Jun 09 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <ca8kcl$1nvb$1 digitaldaemon.com>, Ben Hinkle says...
It would be nice to have mixins DefaultInputStream and DefaultOutputStream
that would contain the default implementations from Stream and then Stream
would be a mixin of those and the seeking and other odds and ends.

What a wonderful idea :) I knew those mixins would come in handy for something! I tend to develop ideas by coding so I've gone ahead and started the modifications. I decided to refactor the design a bit so the streams won't throw exceptions on eof() or on read errors. At the moment I'm going with having the streams castable to bit as an easy way to check state, much the same way C++ streams work. Otherwise the existing methods are staying pretty much as-is. I'll post more in a few days once I get more of the grunt work done. One question about character output. The docs say dchar is a 32-bit unicode character but I'm pretty sure that it's currently implemented as wchar_t in Windows right now (which would make it compatible with wchar). Is there any reason there shouldn't be methods to operate on all three char types: char, wchar, and dchar? Sean
Jun 10 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
On Thu, 10 Jun 2004 23:35:52 +0000 (UTC), Sean Kelly <sean f4.ca>
wrote:

In article <ca8kcl$1nvb$1 digitaldaemon.com>, Ben Hinkle says...
It would be nice to have mixins DefaultInputStream and DefaultOutputStream
that would contain the default implementations from Stream and then Stream
would be a mixin of those and the seeking and other odds and ends.

What a wonderful idea :) I knew those mixins would come in handy for something! I tend to develop ideas by coding so I've gone ahead and started the modifications.

cool. That's usually the best way to gives ideas focus.
I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.  

Are you talking about the ReadError in readExact? I think that error should stay since readBlock returns the number of bytes read if there isn't enough data. If one asks to read exactly 10 bytes and that is impossible it should throw an error.
At the moment I'm going with
having the streams castable to bit as an easy way to check state, much the same
way C++ streams work.  

Two things come to mind: 1) Can opCast be implicit? The DMD spec says "Overloading the cast operator does not affect implicit casts, it only applies to explicit casts." Since writing cast(bit)foo isn't more readable than foo.eof() I'd stay away from opCast for bit. 2) if (foo) ... for any other object checks that foo is non-null. Making it anything else would be wierd.
Otherwise the existing methods are staying pretty much
as-is.  I'll post more in a few days once I get more of the grunt work done.

If you have time and/or desire check out the MmFile wrapper for streams that I mentioned on digitalmars.D.bugs available at http://home.comcast.net/~benhinkle/stream.d Down near the bottom there is ArrayStream(Buffer) that wraps "array-like" types with a stream API.
One question about character output.  The docs say dchar is a 32-bit unicode
character but I'm pretty sure that it's currently implemented as wchar_t in
Windows right now (which would make it compatible with wchar).  Is there any
reason there shouldn't be methods to operate on all three char types: char,
wchar, and dchar?

I wondered that, too, when I was poking around. It could be that stream.d came before dchar? I don't know. I haven't thought very much about it but adding dchar support seems reasonable to me. I can't think of any big reason not to.
Sean

Jun 10 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <3i7ic0dlu2mdmb8q47bmvro8n4og30jvad 4ax.com>, Ben Hinkle says...
On Thu, 10 Jun 2004 23:35:52 +0000 (UTC), Sean Kelly <sean f4.ca>
wrote:

I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.  

Are you talking about the ReadError in readExact? I think that error should stay since readBlock returns the number of bytes read if there isn't enough data. If one asks to read exactly 10 bytes and that is impossible it should throw an error.

What bothered me about the exception throwing in Stream was that since all methods are implemented in terms of readExact, an exception will always be throw when eof() is reached. In my opinion, end of file is an expected rather than an exceptional condition. If I had to wrap all my stream operations in try blocks I'd go nuts :)
At the moment I'm going with
having the streams castable to bit as an easy way to check state, much the same
way C++ streams work.  

Two things come to mind: 1) Can opCast be implicit? The DMD spec says "Overloading the cast operator does not affect implicit casts, it only applies to explicit casts." Since writing cast(bit)foo isn't more readable than foo.eof() I'd stay away from opCast for bit.

Darn. I'd forgotten about this. And I so liked the C++ semantics: while( istr.get(x) ); This would work if all operations returned a fail state, but it's much nicer to have them return the stream itself so operations can be chained. I suppose this works however: while( !istr.get(x).fail() ); and it is perhaps more clear.
If you have time and/or desire check out the MmFile wrapper for
streams that I mentioned on digitalmars.D.bugs available at
http://home.comcast.net/~benhinkle/stream.d
Down near the bottom there is ArrayStream(Buffer) that wraps
"array-like" types with a stream API.

Will do. I have to say, this lack of multiple inheritance has been interesting to deal with. As both InputStream and OutputStream share some common features (state tracking) it made sense to have them both inherit from a common interface. But as the IOStream class was to incorporate stuff from both of them this meant a diamond-shaped inheritance tree, which I think tosses the idea of having only two mixins. I haven't tested it, but assuming this code: class CommonBase { int x; } class DerivedA { mixin CommonBase; } template DerivedB { mixin CommonBase; } template Derived { mixin DerivedA; mixin DerivedB; } Derived will end up with two instances of x defined, won't it? Or perhaps a compiler error? Or is the code intelligent enough to gurantee that the stuff in CommonBase is only imported once? Sean
Jun 11 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
err... replace "template" with "class" in my example.

Sean
Jun 11 2004
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:cacidg$1jn3$1 digitaldaemon.com...
 In article <3i7ic0dlu2mdmb8q47bmvro8n4og30jvad 4ax.com>, Ben Hinkle

On Thu, 10 Jun 2004 23:35:52 +0000 (UTC), Sean Kelly <sean f4.ca>
wrote:

I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.

Are you talking about the ReadError in readExact? I think that error should stay since readBlock returns the number of bytes read if there isn't enough data. If one asks to read exactly 10 bytes and that is impossible it should throw an error.

What bothered me about the exception throwing in Stream was that since all methods are implemented in terms of readExact, an exception will always be

 when eof() is reached.  In my opinion, end of file is an expected rather

 exceptional condition.  If I had to wrap all my stream operations in try

 I'd go nuts :)

Eof in the middle of reading an int (for example) shouldn't be expected, though. If one is trying to read an int and you only get half an int and a return value of false then every call to read(out int) will have to be wrapped in "if" tests instead of try-catches. Reading multiple values would look like if (!s.read(x1) || !s.read(x2) || !s.read(x3)) {evasive action} instead of try{ s.read(x1); s.read(x2); s.read(x3); } catch (ReadError re) { evasive action } Either way I think eof should be the "usual" way of checking if any data remains. [snip]
 I have to say, this lack of multiple inheritance has been interesting to

 with.  As both InputStream and OutputStream share some common features

 tracking) it made sense to have them both inherit from a common interface.

 as the IOStream class was to incorporate stuff from both of them this

 diamond-shaped inheritance tree, which I think tosses the idea of having

 two mixins.  I haven't tested it, but assuming this code:

 class CommonBase
 {
 int x;
 }


 class DerivedA
 {
 mixin CommonBase;
 }

 template DerivedB
 {
 mixin CommonBase;
 }

 template Derived
 {
 mixin DerivedA;
 mixin DerivedB;
 }

 Derived will end up with two instances of x defined, won't it?  Or perhaps

 compiler error?  Or is the code intelligent enough to gurantee that the

 CommonBase is only imported once?

Is it possible to just have the mixins be the non-overlapping parts? I'm looking at Stream class and the routines that implement InputStream only call readBlock and eof to do their thing. I can't see any overlap with the implementations of OutputStream. In other words, lines 219-796 (plus line 209 for "readable") form DefaultInputStream and lines 802-932 (plus 210) form DefaultOutputStream. So I'm picturing interface InputStream{...} interface OutputStream {...} template DefaultInputStream() {...} template DefaultOutputStream() {...} class Stream { mixin DefaultInputStream; mixin DefaultOutputStream; ... } That way everything is backwards compatible with the existing stream.d. Users can't instantiate a DefaultInputStream by itself but that's ok.
 Sean

Jun 11 2004
next sibling parent "Ben Hinkle" <bhinkle mathworks.com> writes:
  interface InputStream{...}
  interface OutputStream {...}
  template DefaultInputStream() {...}
  template DefaultOutputStream() {...}
  class Stream {
     mixin DefaultInputStream;
     mixin DefaultOutputStream;
     ...
  }

oops - I forgot that Stream wants to implement InputStream and OutputStream class Stream : InputStream, OutputStream {
Jun 11 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cact25$2406$1 digitaldaemon.com>, Ben Hinkle says...
Eof in the middle of reading an int (for example) shouldn't be expected,
though.

Agreed. But what about Eof as a result of reading a char?
 If one is trying to read an int and you only get half an int and a
return value of false then every call to read(out int) will have to be
wrapped in "if" tests instead of try-catches. Reading multiple values would
look like
 if (!s.read(x1) || !s.read(x2) || !s.read(x3)) {evasive action}
instead of
 try{
  s.read(x1);
  s.read(x2);
  s.read(x3);
 } catch (ReadError re) { evasive action }

It doesn't have to be quite so bad. I would probably do something like this: if( s.read(x1).read(x2).read(x3).fail() ) {evasive action} I admit this is a matter of preference, and there are certainly times when I want to throw an exception on read failure, but the fact that I don't *always* want an exception thrown makes me leery of building it into Stream. An alternative would be to allow the user to specify that he wants exceptions to be thrown by setting a flag.
Either way I think eof should be the "usual" way of checking if any data
remains.

I agree. Part of my problem was that I didn't investigate how eof() worked until just now. Here's the call: bit eof() { return position() == size(); } This does allow for foreknowledge of eof() location (a feature I wasn't aware of), but it does this by scanning to the end of the stream and then resetting the read position. In some cases this operation can be quite expensive or just plain impossible. An alternative might be to specialize such classes to read-ahead by one byte on every read to check for eof(), but this will wreak havoc with streams that may block. I'm personally used to the method of the C read() function where it will return 0 on eof(), in which case the stream class would set an eof() flag and return a failure (or throw an exception I suppose). But this obviously eliminates any foreknowledge of when eof() has been reached, moving detection back into the read method. But again this is all my personal style and there may be a better way to do it (not to mention Mango, which follows yet another approach). The other consequence of my approach is that it isn't compatible with code written against the existing stream model, which is something I was hoping to avoid.
Is it possible to just have the mixins be the non-overlapping parts? I'm
looking at Stream class and the routines that implement InputStream only
call readBlock and eof to do their thing. I can't see any overlap with the
implementations of OutputStream.

Definately. The overlap would only be new code and that could all be placed in a separate mixin. My question was as much academic as practical. I'm truly curious whether such a design would result in multiply defined data or not. In C/C++ I would say yes, but with D's module-level linkage model I'm not so sure.
That way everything is backwards compatible with the existing stream.d.
Users can't instantiate a DefaultInputStream by itself but that's ok.

I'm kind of torn so far as backwards compatibility is concerned. There are things I don't like about the original design, but at the same time I don't want to propose yet another API format. However, the differences truly aren't that great so I'm going to go with them for now and see how they turn out. I admit I'm using this as an excuse to become more familiar with the language as much as to add features to the stream lib, so whatever happens I won't consider it a wasted effort. Sean
Jun 11 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...

What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be throw
when eof() is reached.  In my opinion, end of file is an expected rather than an
exceptional condition.  If I had to wrap all my stream operations in try blocks
I'd go nuts :)

I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism. Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error. I like this way of working. It beats the hell out of having to test the return value all the time. Arcane Jill
Jun 11 2004
next sibling parent reply Derek <derek psyc.ward> writes:
On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:

 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...
 
What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be throw
when eof() is reached.  In my opinion, end of file is an expected rather than an
exceptional condition.  If I had to wrap all my stream operations in try blocks
I'd go nuts :)

I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism. Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error. I like this way of working. It beats the hell out of having to test the return value all the time. Arcane Jill

But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'. Likewise, reaching the end of file is not an exceptional event, in fact it is quite normal in most cases involving stream files. So although one can use the D exception mechanism to handle eof, the eof event is not really an exception. I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'? -- Derek Melbourne, Australia
Jun 11 2004
next sibling parent Andy Friesen <andy ikagames.com> writes:
Derek wrote:

 But isn't this sort of like the 'bit' verses 'bool' thing. They are
 different animals but a 'bit' can be used to simulate a 'bool'. Likewise,
 reaching the end of file is not an exceptional event, in fact it is quite
 normal in most cases involving stream files. So although one can use the D
 exception mechanism to handle eof, the eof event is not really an
 exception. I know this just semantics, but ... must we be forced into using
 exceptions to handle eof, just like we are forced to use 'bit' to handle
 'bool'?

If you think about it in terms of how many times the method is called and what fraction of those calls will be a read past the end of the stream, it doesn't seem too much of a stretch to call hitting EOF "exceptional". Further, this model ensures that, unlike C stdio, forgetting to check EOF causes reliable, reproducable behaviour. -- andy
Jun 11 2004
prev sibling next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Derek wrote:

 On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:
 
 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...
 
What bothered me about the exception throwing in Stream was that since
all methods are implemented in terms of readExact, an exception will
always be throw
when eof() is reached.  In my opinion, end of file is an expected rather
than an
exceptional condition.  If I had to wrap all my stream operations in try
blocks I'd go nuts :)

I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism. Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error. I like this way of working. It beats the hell out of having to test the return value all the time. Arcane Jill

But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'. Likewise, reaching the end of file is not an exceptional event, in fact it is quite normal in most cases involving stream files. So although one can use the D exception mechanism to handle eof, the eof event is not really an exception. I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'?

Reaching the end of a file is not exceptional. Reaching the end of a file before expected is exceptional.
Jun 11 2004
next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <caddo3$2spa$1 digitaldaemon.com>, Ben Hinkle says...
Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.

If the user calls a function that expects a 4-byte chunk of data and the function hits eof() after 2 bytes then the stream did not behave in an expected manner. Whether this is sufficiently bad to be worth an exception is really a matter of opinion, and no one agrees on how exceptions should be used anyway. Between what you and Andy said however, I'm inclined to agree that an exception may be warranted to protect the uncautious user from weird errors, even if I might not like the behavior myself :) How about this: if the stream expects a fixed-size chunk >1 byte and hits eof() before reading the expected number of bytes then it throws an exception. If the stream expects only 1 byte and hits eof() (ie. the read failed entirely) then it sets a failure state and returns without throwing. The only thing about this tactic that worries me is that it seems inconsistent and thus confusing. Here's an alternate: the unformatted read functions will throw an exception on eof() while the formatted read functions will not. The assumption being that if a programmer is using the unformatted functions then he's likely reading a data file with a known format and any violations of that format would be exceptional. If the programmer really wants to check for eof() to avoid this problem he could do: if( istr.peek().eof() ) Sean
Jun 11 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
Okay, I've been thinking about this and you're slowly winning me over.  I do
write a lot of code like this:

if( !istr.read( x ) ) throw new MyError();

and it would make a lot more sense to just expect the stream itself to do it.
But I'm still on the fence with the end of file issue.  If eof() can always be
evaluated proactively (ie. so I can test for it when I need to and thus avoid an
exception) then I'd say let's go ahead and throw excpetions on eof() during a
read.  The C functions have the eof() function, and I think in Windows I can
call ReadFile with a read length of 0.  Sockets can return closed status.  What
about other streams like USB ports and such?  Darn, I think I've just been
convinced ;)

Sean
Jun 11 2004
prev sibling next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Sean Kelly wrote:

 In article <caddo3$2spa$1 digitaldaemon.com>, Ben Hinkle says...
Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.

If the user calls a function that expects a 4-byte chunk of data and the function hits eof() after 2 bytes then the stream did not behave in an expected manner. Whether this is sufficiently bad to be worth an exception is really a matter of opinion, and no one agrees on how exceptions should be used anyway. Between what you and Andy said however, I'm inclined to agree that an exception may be warranted to protect the uncautious user from weird errors, even if I might not like the behavior myself :) How about this: if the stream expects a fixed-size chunk >1 byte and hits eof() before reading the expected number of bytes then it throws an exception. If the stream expects only 1 byte and hits eof() (ie. the read failed entirely) then it sets a failure state and returns without throwing. The only thing about this tactic that worries me is that it seems inconsistent and thus confusing.

readBlock is an option - though not at convenient.
 Here's an alternate: the unformatted read functions will throw an
 exception on
 eof() while the formatted read functions will not.  The assumption being
 that if a programmer is using the unformatted functions then he's likely
 reading a data file with a known format and any violations of that format
 would be exceptional. If the programmer really wants to check for eof() to
 avoid this problem he could do:
 
 if( istr.peek().eof() )

Looking at Stream.scanf it assumes one can read one character after the field being read. For example, the code to parse "%d" is while (isdigit(c) && width) { n = n * 10 + (c - '0'); width--; c = getc(); count++; } and getc() will throw an exception at eof(). That looks like a bug in scanf. I haven't tried any example, though, so I could just be reading the code incorrectly. It would be a little more involved to check for eof but the code would be more robust. Do you think that is a reasonable set of changes to make?
 
 Sean

Jun 11 2004
parent Sean Kelly <sean f4.ca> writes:
In article <cadj30$2b2$1 digitaldaemon.com>, Ben Hinkle says...
Looking at Stream.scanf it assumes one can read one character after the
field being read. For example, the code to parse "%d" is
  while (isdigit(c) && width)
  {
    n = n * 10 + (c - '0');
    width--;
    c = getc();
    count++;
  }
and getc() will throw an exception at eof(). That looks like a bug in scanf. 
I haven't tried any example, though, so I could just be reading the code
incorrectly. It would be a little more involved to check for eof but the
code would be more robust. Do you think that is a reasonable set of changes
to make? 

Definately. I already made a similar change in getLine. Sean
Jun 12 2004
prev sibling parent "The Dr ... who?" <thedr who.com> writes:
All this stems from the fact that what is currently implemented, or what is
being
proposed, - pardon my ignorance of both - is a stream for all seasons. The
times - all too many - I've written such things in the past I've separated an
exception-free, byte-only stream lower layer, upon which may be layered whatever
else one wants: object-based streams, streams that'll read ints/doubles/etc.,
text streams (\r\n <=> \n), etc.

Surely that's the way to go?

"Sean Kelly" <sean f4.ca> wrote in message
news:cadg1r$2vpf$1 digitaldaemon.com...
 In article <caddo3$2spa$1 digitaldaemon.com>, Ben Hinkle says...
Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.

If the user calls a function that expects a 4-byte chunk of data and the function hits eof() after 2 bytes then the stream did not behave in an expected manner. Whether this is sufficiently bad to be worth an exception is really a matter of opinion, and no one agrees on how exceptions should be used anyway. Between what you and Andy said however, I'm inclined to agree that an exception may be warranted to protect the uncautious user from weird errors, even if I might not like the behavior myself :) How about this: if the stream expects a fixed-size chunk >1 byte and hits eof() before reading the expected number of bytes then it throws an exception. If

 stream expects only 1 byte and hits eof() (ie. the read failed entirely) then

 sets a failure state and returns without throwing.  The only thing about this
 tactic that worries me is that it seems inconsistent and thus confusing.

 Here's an alternate: the unformatted read functions will throw an exception on
 eof() while the formatted read functions will not.  The assumption being that

 a programmer is using the unformatted functions then he's likely reading a data
 file with a known format and any violations of that format would be

 If the programmer really wants to check for eof() to avoid this problem he

 do:

 if( istr.peek().eof() )


 Sean

Jun 12 2004
prev sibling parent reply Regan Heath <regan netwin.co.nz> writes:
On Fri, 11 Jun 2004 19:03:36 -0400, Ben Hinkle <bhinkle4 juno.com> wrote:
 Derek wrote:

 On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:

 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...

 What bothered me about the exception throwing in Stream was that since
 all methods are implemented in terms of readExact, an exception will
 always be throw
 when eof() is reached.  In my opinion, end of file is an expected 
 rather
 than an
 exceptional condition.  If I had to wrap all my stream operations in 
 try
 blocks I'd go nuts :)

I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism. Maybe I'm just more comfortable working with exceptions than some. To my mind, best practice behavior is to throw an exception on end-of-file, and a /different/ exception on error. I like this way of working. It beats the hell out of having to test the return value all the time. Arcane Jill

But isn't this sort of like the 'bit' verses 'bool' thing. They are different animals but a 'bit' can be used to simulate a 'bool'. Likewise, reaching the end of file is not an exceptional event, in fact it is quite normal in most cases involving stream files. So although one can use the D exception mechanism to handle eof, the eof event is not really an exception. I know this just semantics, but ... must we be forced into using exceptions to handle eof, just like we are forced to use 'bit' to handle 'bool'?

Reaching the end of a file is not exceptional. Reaching the end of a file before expected is exceptional.

Exactly. Here is an example of how I typically read a file in C: f = fopen("filename","r"); if (!f) ..error.. while(fgets(bf,BFSZ,f)) { ..do stuff.. } if (ferror(f)) ..error.. fclose(f); getting to the end of the file in the above is not an exception in my mind, it's something that will happen every time this code is executed. I'd only want exceptions when: - the file open failed - the read failed (device error etc - not end of file) Plus the last line read may/may not have the end of line char(s) this is not exceptional either IMO. Arcane Jill's compiler code example showing using them as flow control can still be implemented without an exception on end of file in the stream code. It can be done by manually throwing an exception in the compiler code when the read hits eof. IMO this is actually better as it explicitly shows you're doing it. Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:

 Exactly.
 
 Here is an example of how I typically read a file in C:
 
 f = fopen("filename","r");
 if (!f) ..error..
 
 while(fgets(bf,BFSZ,f)) {
   ..do stuff..
 }
 
 if (ferror(f)) ..error..
 
 fclose(f);
 
 
 getting to the end of the file in the above is not an exception in my 
 mind, it's something that will happen every time this code is executed.

else) to test whether we've read the last byte, without blocking? If so, I think something along the lines of while(!f.eof) { bf=f.gets(BFSZ); ... } is likely clearer.
 I'd only want exceptions when:
 - the file open failed
 - the read failed (device error etc - not end of file)

6.03E+ and then hit EOF? (Maybe that's legal, it's not the point ;-) I'm keen on the idea that in a well-formed set of data, no read call fails, it means (when reading data of a known format) you don't have to put tests for error codes in your main code, and exceptions only handle errors of one sort or another.
 Plus the last line read may/may not have the end of line char(s) this is 
 not exceptional either IMO.

Sam
Jun 13 2004
next sibling parent reply Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 15:37:33 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Regan Heath wrote:

 Exactly.

 Here is an example of how I typically read a file in C:

 f = fopen("filename","r");
 if (!f) ..error..

 while(fgets(bf,BFSZ,f)) {
   ..do stuff..
 }

 if (ferror(f)) ..error..

 fclose(f);


 getting to the end of the file in the above is not an exception in my 
 mind, it's something that will happen every time this code is executed.


 Is there a way in theory in general streams (sockets, files, anything 
 else) to test whether we've read the last byte, without blocking?

In the case of stream sockets I reckon you're only 'at the end' if the socket has been closed, otherwise, there might be data to read now, or not, or some might come later. Same with COM ports, and probably USB ports also. Files are a little different as the data is all available as soon as you open it. So .eof for a file simply compares the read ptr position with the position of the end of the file. You can make stream sockets non-blocking in which case a read will return immediately. Same with COM ports and I assume USB ports also. Sockets are weird in that when they are closed by the other end you get told by 'select' that you can 'read'/'write' to/from the socket, and the actual read/write reports that it is closed. AFAIK there is no other way to tell if it's been closed.
 If so, I think something along the lines of
 while(!f.eof) {
 	bf=f.gets(BFSZ);
 	...
 }
 is likely clearer.

I was only giving my C style code, not suggesting a format for the D code, the D could look like this... { FileStream i; try { i = new FileStream(new File("a.txt",READ)); while(!i.eof()) { line = i.readLine(); ..do stuff.. } } catch (Exception e) { ..you can catch all exceptions here and decide what to do with each.. } finally { if (i) delete i; } } or something like that.
 I'd only want exceptions when:
 - the file open failed
 - the read failed (device error etc - not end of file)


 What if scanf was looking for a floating point number, and it got to 
 6.03E+ and then hit EOF? (Maybe that's legal, it's not the point ;-)

This is different, and should throw an exception :)
 I'm keen on the idea that in a well-formed set of data, no read call 
 fails, it means (when reading data of a known format) you don't have to 
 put tests for error codes in your main code, and exceptions only handle 
 errors of one sort or another.

Agreed.
 Plus the last line read may/may not have the end of line char(s) this 
 is not exceptional either IMO.


I only mentioned this as I imagine it would be a common mistake to make. Regan. -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:
 In the case of stream sockets I reckon you're only 'at the end' if the 
 socket has been closed, otherwise, there might be data to read now, or 
 not, or some might come later. Same with COM ports, and probably USB 
 ports also.
 ...
 You can make stream sockets non-blocking in which case a read will 
 return immediately. Same with COM ports and I assume USB ports also.
 
 Sockets are weird in that when they are closed by the other end you get 
 told by 'select' that you can 'read'/'write' to/from the socket, and the 
 actual read/write reports that it is closed. AFAIK there is no other way 
 to tell if it's been closed.

no advance warning in the normal course of events (HTTP?). The tricky case seems to be: data\n<EOF> If you're reading data one line at a time from an unbuffered socket, and it ends like that, then you'll terminate by reading an EOF when you expected a line, and you can't predict it. It's frustrating, because it seems in every other case you can get away with using exceptions without having to handle them in non-error situations.
 If so, I think something along the lines of
 while(!f.eof) {
     bf=f.gets(BFSZ);
     ...
 }
 is likely clearer.

I was only giving my C style code, not suggesting a format for the D code, the D could look like this...

beginning of a read, allowing no EOL at EOF) _is_ an exceptional circumstance. Sam
Jun 13 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 16:31:31 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Regan Heath wrote:
 In the case of stream sockets I reckon you're only 'at the end' if the 
 socket has been closed, otherwise, there might be data to read now, or 
 not, or some might come later. Same with COM ports, and probably USB 
 ports also.
 ...
 You can make stream sockets non-blocking in which case a read will 
 return immediately. Same with COM ports and I assume USB ports also.

 Sockets are weird in that when they are closed by the other end you get 
 told by 'select' that you can 'read'/'write' to/from the socket, and 
 the actual read/write reports that it is closed. AFAIK there is no 
 other way to tell if it's been closed.


 Ouch. Presumably there are protocols where the socket gets closed with 
 no advance warning in the normal course of events (HTTP?).

HTTP sends a Content-Length most/all? of the time.
 The tricky case seems to be:
 data\n<EOF>
 If you're reading data one line at a time from an unbuffered socket, and 
 it ends like that, then you'll terminate by reading an EOF when you 
 expected a line, and you can't predict it.

So unless the readLine function has a failure return value you have to throw an exception. The same would have to be true for readInt readFloat etc. If it had a failure return value then.. while(!i.eof()) { line = i.readLine(); if (line.length == 0) continue; } would work, as the readLine that read the EOF would flag eof and i.eof would then return true.
 It's frustrating, because it seems in every other case you can get away 
 with using exceptions without having to handle them in non-error 
 situations.

Yeah.. I have felt this frustration before as I briefly tried to write my own stream lib, just for fun.
 If so, I think something along the lines of
 while(!f.eof) {
     bf=f.gets(BFSZ);
     ...
 }
 is likely clearer.

I was only giving my C style code, not suggesting a format for the D code, the D could look like this...


 Right, the point being that now hitting EOF during a read (well, at the 
 beginning of a read, allowing no EOL at EOF) _is_ an exceptional 
 circumstance.

Yep. -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:
 HTTP sends a Content-Length most/all? of the time.

 So unless the readLine function has a failure return value you have to 
 throw an exception. The same would have to be true for readInt readFloat 
 etc.
 
 If it had a failure return value then..
 
 while(!i.eof()) {
     line = i.readLine();
     if (line.length == 0) continue;
 }

would be something like data \n \n <EOF> ^ readLine() returns "data" data \n \n <EOF> ^ readLine() returns "" data \n \n <EOF> ^ readLine() returns "" data \n \n <EOF> ^ readLine() returns null (EOF already been hit) Sam
Jun 13 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 17:27:52 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Regan Heath wrote:
 HTTP sends a Content-Length most/all? of the time.

 So unless the readLine function has a failure return value you have to 
 throw an exception. The same would have to be true for readInt 
 readFloat etc.

 If it had a failure return value then..

 while(!i.eof()) {
     line = i.readLine();
     if (line.length == 0) continue;
 }

would be something like data \n \n <EOF> ^ readLine() returns "data"

agreed.
   data \n \n <EOF>
          ^
 readLine() returns ""

shouldn't that be " "
   data \n \n <EOF>
             ^
 readLine() returns ""

shouldn't that be " "
   data \n \n <EOF>
                ^
 readLine() returns null (EOF already been hit)

you're right here. Regan. -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:
 shouldn't that be " "

clear what had and hadn't been read yet. Sam
Jun 13 2004
parent Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 17:35:35 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Regan Heath wrote:
 shouldn't that be " "

clear what had and hadn't been read yet. Sam

I thought they *might* have been, just checking. :) -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <caj6cj$254g$1 digitaldaemon.com>, Sam McCall says...
Is there a way in theory in general streams (sockets, files, anything 
else) to test whether we've read the last byte, without blocking? If so, 
I think something along the lines of
while(!f.eof) {
	bf=f.gets(BFSZ);
	...
}
is likely clearer.

In general, yes. I would consider a socket at EOF when the socket is closed. The ANSI C file API has an feof() function, and other file APIs typically have a way to determine this as well.
 I'd only want exceptions when:
 - the file open failed
 - the read failed (device error etc - not end of file)

What if scanf was looking for a floating point number, and it got to 6.03E+ and then hit EOF?

This is what convinced me that exceptions made sense for stream operations. Encountering an unexpected EOF *is* an exceptional situation IMO. Other exceptional situations include unformatted operations that fail to read the desired block size (typically as a result of EOF), and formatted read operations that result in unparsable or incorrect data. Note that this includes instances where the user tries to read an integer from the keyboard and gets a letter instead. One thing I'm considering is adding overridable methods that handle these error conditions so derived classes can throw more meaningful exceptions. If it's Windows and a meaningful message can be pulled from the error code then the class should be able to include that information in the exception. Since different APIs have different ways of returning error information, it isn't really possible to put all of this in the base class. Sean
Jun 14 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...

But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not compile-time error checking is possible for a certain class of error. The difference between exceptions and return-value-checking is RUN-TIME error checking (versus crashing, which is what would happen if you forgot to check the return code).
Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a program is run be considered exceptional? Can an event that is planned for and handled be considered an error? The answer to both questions is yes. 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is better to think of an exception as meaning 'some part of the system couldn't do what it was asked to do'". Further on in the book, there is a whole section entitled Exceptions that are not Errors, in which he says: "If an exception is expected and caught so that it has no bad effects on the behavior of the program then how can it be an error? Only because the programmer thinks of it as an error and of the exception-handling mechanisms as tools for handling errors. Alternatively, one might think of the exception-handling mechanisms as simply another control structure". I have written code in which an exception signals success! And this was not because I'm completely mad (I refrain from comment on that point), but because it was a brilliantly elegant thing to do. So it is here. We've reached the end of the file - yippee - now let's roll up the stack and move onto the next bit of code. In D, we should be using exceptions even more than in C++, because the exception handling mechanism is much, much simpler and there is way less overhead. Walter has written some good stuff in the D manual about why exceptions are better than return codes in handling errors. He should probably have mentioned that this is also true for non-errors.
I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other. If I wrote an API which /required/ that callers check the return value from some funtion, how long do you think it would be before someone forgot? Exceptions are the path of safety. I think people are just afraid of using them, that's all. Maybe because of unfamilairity, or they have come to associate the exception-mechanism with error-handling. To me, refusing to employ the exception-mechanism because it isn't an error is like refusing to use for-loops because it isn't an error. I would hate to have to go back to the tedious and error-prone methods of return-value-checking. New tools give us new and powerful techniques. We should use them, not be afraid of them. Arcane Jill
Jun 11 2004
next sibling parent reply Derek <derek psyc.ward> writes:
On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:

 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...
 
But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not compile-time error checking is possible for a certain class of error. The difference between exceptions and return-value-checking is RUN-TIME error checking (versus crashing, which is what would happen if you forgot to check the return code).
Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a program is run be considered exceptional? Can an event that is planned for and handled be considered an error? The answer to both questions is yes. 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is better to think of an exception as meaning 'some part of the system couldn't do what it was asked to do'". Further on in the book, there is a whole section entitled Exceptions that are not Errors, in which he says: "If an exception is expected and caught so that it has no bad effects on the behavior of the program then how can it be an error? Only because the programmer thinks of it as an error and of the exception-handling mechanisms as tools for handling errors. Alternatively, one might think of the exception-handling mechanisms as simply another control structure". I have written code in which an exception signals success! And this was not because I'm completely mad (I refrain from comment on that point), but because it was a brilliantly elegant thing to do. So it is here. We've reached the end of the file - yippee - now let's roll up the stack and move onto the next bit of code. In D, we should be using exceptions even more than in C++, because the exception handling mechanism is much, much simpler and there is way less overhead. Walter has written some good stuff in the D manual about why exceptions are better than return codes in handling errors. He should probably have mentioned that this is also true for non-errors.
I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other. If I wrote an API which /required/ that callers check the return value from some funtion, how long do you think it would be before someone forgot? Exceptions are the path of safety. I think people are just afraid of using them, that's all. Maybe because of unfamilairity, or they have come to associate the exception-mechanism with error-handling. To me, refusing to employ the exception-mechanism because it isn't an error is like refusing to use for-loops because it isn't an error. I would hate to have to go back to the tedious and error-prone methods of return-value-checking. New tools give us new and powerful techniques. We should use them, not be afraid of them. Arcane Jill

Got it. You are using Exceptions as a fancy GOTO. -- Derek Melbourne, Australia
Jun 11 2004
parent reply "The Dr ... who?" <thedr who.com> writes:
"Derek" <derek psyc.ward> wrote in message
news:c5k4ima3f0tp.1nm5clznwf9x6.dlg 40tude.net...
 On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:

 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...

But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not


 error checking is possible for a certain class of error. The difference


 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return code).


Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times


 program is run be considered exceptional? Can an event that is planned for


 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is


 to think of an exception as meaning 'some part of the system couldn't do what


 was asked to do'".

 Further on in the book, there is a whole section entitled Exceptions that are
 not Errors, in which he says: "If an exception is expected and caught so that


 has no bad effects on the behavior of the program then how can it be an


 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively,


 might think of the exception-handling mechanisms as simply another control
 structure".

 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but


 it was a brilliantly elegant thing to do. So it is here. We've reached the


 of the file - yippee - now let's roll up the stack and move onto the next bit


 code.

 In D, we should be using exceptions even more than in C++, because the


 handling mechanism is much, much simpler and there is way less overhead.

 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have


 that this is also true for non-errors.


I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other. If I wrote an API which /required/ that callers check the return value from


 funtion, how long do you think it would be before someone forgot? Exceptions


 the path of safety.

 I think people are just afraid of using them, that's all. Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because it
 isn't an error is like refusing to use for-loops because it isn't an error.

 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking. New tools give us new and powerful techniques. We


 use them, not be afraid of them.

 Arcane Jill

Got it. You are using Exceptions as a fancy GOTO.

Well said. This is something that is little recognised by proponents of exceptions-for-all. They are a goto, since they change the flow of control. Sure, they're vastly better than goto, since resource handling is automatic. But, it's still possible for flow of control to jump to an arbitrary location. The only difference is that the destination is limited to somewhere in the call stack, whereas with goto it's anywhere the label is located. But, before anyone thinks I'm anti-exceptions, I'm not. They're the absolute right answer for a significant minority of error-handling scenarios, and the winner of an even bigger minority (if not the majority) of other scenarioe. They're just not the right answer in all cases.
Jun 12 2004
parent "The Dr ... who?" <thedr who.com> writes:
"The Dr ... who?" <thedr who.com> wrote in message
news:cafvrc$kts$1 digitaldaemon.com...
 "Derek" <derek psyc.ward> wrote in message
 news:c5k4ima3f0tp.1nm5clznwf9x6.dlg 40tude.net...
 On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:

 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...

But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not


 error checking is possible for a certain class of error. The difference


 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return



Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition"



 Bjarne Stroustrup (the creator of C++): "Can an event that happens most



 a
 program is run be considered exceptional? Can an event that is planned for


 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is


 to think of an exception as meaning 'some part of the system couldn't do



 it
 was asked to do'".

 Further on in the book, there is a whole section entitled Exceptions that



 not Errors, in which he says: "If an exception is expected and caught so



 it
 has no bad effects on the behavior of the program then how can it be an


 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively,


 might think of the exception-handling mechanisms as simply another control
 structure".

 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but


 it was a brilliantly elegant thing to do. So it is here. We've reached the


 of the file - yippee - now let's roll up the stack and move onto the next



 of
 code.

 In D, we should be using exceptions even more than in C++, because the


 handling mechanism is much, much simpler and there is way less overhead.

 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have


 that this is also true for non-errors.


I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you



 code it either ways, but one way is MUCH safer than the other.

 If I wrote an API which /required/ that callers check the return value from


 funtion, how long do you think it would be before someone forgot?



 are
 the path of safety.

 I think people are just afraid of using them, that's all. Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because



 isn't an error is like refusing to use for-loops because it isn't an error.

 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking. New tools give us new and powerful techniques. We


 use them, not be afraid of them.

 Arcane Jill

Got it. You are using Exceptions as a fancy GOTO.

Well said. This is something that is little recognised by proponents of exceptions-for-all. They are a goto, since they change the flow of control.

 they're vastly better than goto, since resource handling is automatic.

_and_ because the destination is arbitrated at runtime, and (largely) under control of client code. It's a kind of publish-subscribe mechanism. But it's still a goto!
Jun 12 2004
prev sibling next sibling parent reply Derek <derek psyc.ward> writes:
On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:

 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...
 
But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not compile-time error checking is possible for a certain class of error. The difference between exceptions and return-value-checking is RUN-TIME error checking (versus crashing, which is what would happen if you forgot to check the return code).
Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a program is run be considered exceptional? Can an event that is planned for and handled be considered an error? The answer to both questions is yes. 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is better to think of an exception as meaning 'some part of the system couldn't do what it was asked to do'". Further on in the book, there is a whole section entitled Exceptions that are not Errors, in which he says: "If an exception is expected and caught so that it has no bad effects on the behavior of the program then how can it be an error? Only because the programmer thinks of it as an error and of the exception-handling mechanisms as tools for handling errors. Alternatively, one might think of the exception-handling mechanisms as simply another control structure". I have written code in which an exception signals success! And this was not because I'm completely mad (I refrain from comment on that point), but because it was a brilliantly elegant thing to do. So it is here. We've reached the end of the file - yippee - now let's roll up the stack and move onto the next bit of code. In D, we should be using exceptions even more than in C++, because the exception handling mechanism is much, much simpler and there is way less overhead. Walter has written some good stuff in the D manual about why exceptions are better than return codes in handling errors. He should probably have mentioned that this is also true for non-errors.
I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other. If I wrote an API which /required/ that callers check the return value from some funtion, how long do you think it would be before someone forgot? Exceptions are the path of safety. I think people are just afraid of using them, that's all. Maybe because of unfamilairity, or they have come to associate the exception-mechanism with error-handling. To me, refusing to employ the exception-mechanism because it isn't an error is like refusing to use for-loops because it isn't an error. I would hate to have to go back to the tedious and error-prone methods of return-value-checking. New tools give us new and powerful techniques. We should use them, not be afraid of them. Arcane Jill

Maybe I should think of Exceptions as Event Triggers instead of of Error Conditions. This is helpful. -- Derek Melbourne, Australia
Jun 11 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <iqr9lefhimza$.1wolikdlbu2jm$.dlg 40tude.net>, Derek says...

Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.

Exceptions are the most appropriate mechanism to use if the following four conditions are simultaneously met: 1) You wish to be advised of a particular circumstance 2) More often than not, the circumstance will not happen 3) When it does happen, information that it has happened needs to be propogated back up the call stack 4) Failure to communicate information about the circumstance will result in bad or undefined behavior These conditions are completely met in the case of end-of-file. You wish to be advised of it; more often than not, a read won't result in eof; you need to propogate eof up the call stack; if you don't, undefined behaviour - possibly bang. Ergo - there is mechanism that is just right - the exception mechanism. An example might help. Imagine you're writing a compiler/parser for some new programming langage you've just invented. Your main loop might look something like this:
   void parseModule(Stream s)
   {
       for (;;)
       {
           parseNextDeclaration(s);
       }
   }

Within parseNextDeclaration() you're going to have other similiar loops, things like parseNextStatement(), parseNextExpression(), parseNextIdentifier(), and so on. And way, way, WAY down the call-stack, you're going to want to read raw fixed size blocks of data. WIthout the exception mechanism, you'd have to change every single one of those functions from void to bool. You would have to check the return value of every single one of them, every single time, and make sure to return false if some lower-level function returned false. Apart from being tedious and error-prone, this is also going to slow your program down, because of all that overhead of checking return values over and over again. You shouldn't need to do it, because eof is a rare circumstance (rare in the sense that most reads will succeed - I don't mean to imply that most files don't end, obviously). By contrast, to execute a throw statement, the code which the D compiler generates for you merely has to (1) copy a value into the stack pointer, (2) switch, based on the type of the exception thrown, (3) execute a jump instruction. Now tell me which has the greater overhead? (Okay, if there are nested try blocks and/or auto class declarations, you may have to repeat that procedure a few times, once for each exception frame, but generally there will be WAY more stack frames than exception frames). (The efficiency argument doesn't hold true in C++, by the way. In C++, simply copying a new value into the stack pointer would be disastrous in C++ because there may be objects on the stack whose destructors need to be called. D is more sensible). So you've got the best of all worlds - efficiency; safety; clean code. My one complaint about D's exceptions is that there realy SHOULD be some standard ones in Phobos which we could reuse - things like EndOfFile, OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module. They should be organized into a heirarchy which makes sense, so it becomes obvious which one to use/derive from. Constructors for these should be simple. You shouldn't need to supply a string to construct an EndOfFile, and so on. Arcane Jill
Jun 12 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
In article <caeaju$162q$1 digitaldaemon.com>, Arcane Jill says...
My one complaint about D's exceptions is that there realy SHOULD be some
standard ones in Phobos which we could reuse - things like EndOfFile,
OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
They should be organized into a heirarchy which makes sense, so it becomes
obvious which one to use/derive from. Constructors for these should be simple.
You shouldn't need to supply a string to construct an EndOfFile, and so on.

Exception and Error are a good base, and we probably need BadCast, OutOfMemory, DivideByZero and some others if they don't already exist. Beyond that I think it's up to each module to define a set of errors that makes sense. I think I may change the Stream errors around a bit so they can include platform-specific information in the cases where it's applicable. Sean
Jun 12 2004
prev sibling next sibling parent Derek <derek psyc.ward> writes:
On Sat, 12 Jun 2004 07:16:14 +0000 (UTC), Arcane Jill wrote:

 In article <iqr9lefhimza$.1wolikdlbu2jm$.dlg 40tude.net>, Derek says...
 
Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.

Exceptions are the most appropriate mechanism to use if the following four conditions are simultaneously met: 1) You wish to be advised of a particular circumstance 2) More often than not, the circumstance will not happen 3) When it does happen, information that it has happened needs to be propogated back up the call stack 4) Failure to communicate information about the circumstance will result in bad or undefined behavior

Thanks. This is quite a neat explanation. Maybe #2 could be reworded as ... 2) The circumstance has a low frequency of occurance, or is not expected to occur. Because it is quite possible that sometimes, *not* reaching EOF is an error. [snip]
 
 So you've got the best of all worlds - efficiency; safety; clean code.
 
 My one complaint about D's exceptions is that there realy SHOULD be some
 standard ones in Phobos which we could reuse - things like EndOfFile,
 OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
 They should be organized into a heirarchy which makes sense, so it becomes
 obvious which one to use/derive from. Constructors for these should be simple.
 You shouldn't need to supply a string to construct an EndOfFile, and so on.

Hmmm... I tend to agree. -- Derek Melbourne, Australia
Jun 12 2004
prev sibling next sibling parent "Bent Rasmussen" <exo bent-rasmussen.info> writes:
Is it an exception that the tenth element of a list does not exist when the
previous nine did? This is case for contracts over exceptions: To catch bugs
use contracts, to recover from exceptional state use exceptions.

- It is predictable that the file will end, it is possible to check when it
does.
- It is predictable that at some time a user will type in an email address
without the at-sign, it is possible to check when s/he does.
- It is predictable that a programmer will create bugs, its often possible
to check when s/he does.

But my experience with streams in general is very limited, so I must be
missing something very trivial because this use of exceptions practically
lights up all warning lights to me.
Jun 13 2004
prev sibling parent reply Antti =?iso-8859-1?Q?Syk=E4ri?= <jsykari gamma.hut.fi> writes:
In article <caeaju$162q$1 digitaldaemon.com>, Arcane Jill wrote:
 (The efficiency argument doesn't hold true in C++, by the way. In C++,
 simply copying a new value into the stack pointer would be disastrous
 in C++ because there may be objects on the stack whose destructors
 need to be called. D is more sensible).

I'd have thought that exception handling in D and C++ have the same overhead, because both have RAII classes and try-catch statements. (In typical well-written C++ program, though, unwinding the stack takes a bit longer, since RAII is also used for memory management.) Of course, that wouldn't matter, since typical exception handling time is still negligible if exception handling is used properly.
 My one complaint about D's exceptions is that there realy SHOULD be some
 standard ones in Phobos which we could reuse - things like EndOfFile,
 OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
 They should be organized into a heirarchy which makes sense, so it becomes
 obvious which one to use/derive from. Constructors for these should be simple.
 You shouldn't need to supply a string to construct an EndOfFile, and so on.

Gets my vote. I think Matthew Wilson had something in the works a few months ago? (I somehow find it amusing to have a language with standard exception called "Success". Certainly requires some guts to admit that success is an exceptional condition ;) No committee-designed language would ever be able to pull that off...) -Antti -- I will not be using Plan 9 in the creation of weapons of mass destruction to be used by nations other than the US.
Jun 14 2004
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Antti Sykäri" <jsykari gamma.hut.fi> wrote in message
news:slrnccrqj5.3eo.jsykari pulu.hut.fi...
 In article <caeaju$162q$1 digitaldaemon.com>, Arcane Jill wrote:
 (The efficiency argument doesn't hold true in C++, by the way. In C++,
 simply copying a new value into the stack pointer would be disastrous
 in C++ because there may be objects on the stack whose destructors
 need to be called. D is more sensible).

I'd have thought that exception handling in D and C++ have the same overhead, because both have RAII classes and try-catch statements. (In typical well-written C++ program, though, unwinding the stack takes a bit longer, since RAII is also used for memory management.) Of course, that wouldn't matter, since typical exception handling time is still negligible if exception handling is used properly.
 My one complaint about D's exceptions is that there realy SHOULD be some
 standard ones in Phobos which we could reuse - things like EndOfFile,
 OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
 They should be organized into a heirarchy which makes sense, so it becomes
 obvious which one to use/derive from. Constructors for these should be


 You shouldn't need to supply a string to construct an EndOfFile, and so on.

Gets my vote. I think Matthew Wilson had something in the works a few months ago?

Yes, we did come to an accord on the naming conventions. I have to admit, somewhat sheepishly, that I didn't write it down anywhere, trusting the newgroup archive. If someone with more time (or good form) than me wants to search last year's posts for "Exception", or maybe "Win32Exception", then I'm sure it'll be there. Perhaps once located it could be placed up there with all JC's other good stuff on the Wiki? Lenny Lazybones
Jun 14 2004
parent J C Calvarese <jcc7 cox.net> writes:
Matthew wrote:
 "Antti Sykäri" <jsykari gamma.hut.fi> wrote in message
 news:slrnccrqj5.3eo.jsykari pulu.hut.fi...
 

My one complaint about D's exceptions is that there realy SHOULD be some
standard ones in Phobos which we could reuse - things like EndOfFile,
OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
They should be organized into a heirarchy which makes sense, so it becomes
obvious which one to use/derive from. Constructors for these should be


simple.
You shouldn't need to supply a string to construct an EndOfFile, and so on.

Gets my vote. I think Matthew Wilson had something in the works a few months ago?

Yes, we did come to an accord on the naming conventions. I have to admit, somewhat sheepishly, that I didn't write it down anywhere, trusting the newgroup archive. If someone with more time (or good form) than me wants to search last year's posts for "Exception", or maybe "Win32Exception", then I'm sure it'll be there. Perhaps once located it could be placed up there with all JC's other good stuff on the Wiki? Lenny Lazybones

FYI, I think I found the thread you were remembering. It was called "Exception naming conventions - Phobos, and third party libraries" (yay, for helpful thread subjects). D/15869 -- Justin (a/k/a jcc7) http://jcc_7.tripod.com/d/
Jun 21 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <iqr9lefhimza$.1wolikdlbu2jm$.dlg 40tude.net>, Derek says...
Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.

They're an alternate means of flow control. Some argue that exceptions can make a program difficult to read, but I think so long as everything is well-documented it's not an issue. They certainly beat wrapping every function call with error checking routines, and exception performance has gotten quite good even in C++ these past few years. Sean
Jun 12 2004
prev sibling parent reply "The Dr ... who?" <thedr who.com> writes:
 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...

But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.

Not exactly. The difference between bool and bit is whether or not compile-time error checking is possible for a certain class of error. The difference between exceptions and return-value-checking is RUN-TIME error checking (versus crashing, which is what would happen if you forgot to check the return code).
Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.

Let me quote you something from "The C++ Programming Language 3rd Edition" by Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a program is run be considered exceptional? Can an event that is planned for and handled be considered an error? The answer to both questions is yes. 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is

 to think of an exception as meaning 'some part of the system couldn't do what

 was asked to do'".

 Further on in the book, there is a whole section entitled Exceptions that are
 not Errors, in which he says: "If an exception is expected and caught so that

 has no bad effects on the behavior of the program then how can it be an error?
 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively, one
 might think of the exception-handling mechanisms as simply another control
 structure".

 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but because
 it was a brilliantly elegant thing to do.

Maybe so. Can't see as how it's particularly apposite to this issue, other than it's reflective of your opinion on the subject.
 So it is here. We've reached the end
 of the file - yippee - now let's roll up the stack and move onto the next bit

 code.

 In D, we should be using exceptions even more than in C++, because the

 handling mechanism is much, much simpler and there is way less overhead.

No. D is not Java, which equally accords with your given criterion. D is a multi-paradigm language, just like C++. That, above all the nifty little syntax tricks and GC, is what will make it succeed. Exceptions are the absolute right answer in many circumstances. Exceptions are the absolute wrong answer in many other circumstances. And sometimes, there's justified equivocation. And sometimes - this time! - people are coming at the issue from the wrong angle. Reading objects, or even fundamentals whose .size >1, requires exception handling. Manipulating byte (or, I guess, ANSI char) streams does _not_ require exception handling, and in many instances is better off without it. So rather than trying to do something hideous like have the user supply a bExceptionsPlease parameter, why not simply do the obvious thing, and build an object-aware exception-throwing arbitrary type manipulating layer on top of a super simple byte-stream layer? It's not exactly like it's a new idea, and the benefits are legion. For a start, we can plug in different byte streams, sharing a common interface, when appropriate.
 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have mentioned
 that this is also true for non-errors.

It's not black, and it's not white. Such statements as this are positively unhelpful to the community, and especially to neophyte/junior programmers. The world is rich, and there are few richer parts of SE than error-handling.
I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

I think you'll find it's the other way round. On the bit/bool question, non-arithmetic bool would have been the safer option, but we have not been offered them. On the exception versus check-the-return code question, you can code it either ways, but one way is MUCH safer than the other. If I wrote an API which /required/ that callers check the return value from

 funtion, how long do you think it would be before someone forgot? Exceptions

 the path of safety.

 I think people are just afraid of using them, that's all.

That may be for some. For others it's the opposite: exceptions are touted as appropriate to all error-handling / return-indicating scenarios. But there are many experienced and erudite players that do not see exceptions as a panacea. They are the answer in most cases, but they are _not_ the answer in all.
 Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because it
 isn't an error is like refusing to use for-loops because it isn't an error.

That's true.
 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking.

Horses for courses.
 New tools give us new and powerful techniques. We should
 use them, not be afraid of them.

True. But we should also be confident to point out when the emporer's undies are visible. Anyway, back to the topic: let's have a two/multi-layer streaming architecture, and this whole issue is rendered moot. The added advantage is that we'll actually have a more elegant and extensible architecture to boot!
Jun 12 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
The Dr ... who? wrote:
 No. D is not Java...

How about this? abstract class Stream { // reads a byte and stores it in dest, returns false // (and stores -1?) on eof abstract bool read(out ubyte dest); // reads up to len bytes, returns number read, or -1 on eof int read(ubyte[] dest,int off,int len); int read(ubyte[] dest) { return read(dest,0,dest.length); } // Reads a formatted value and throws on eof // Anyone got a good short name for this? void readData(out ubyte b); void readData(out int i); ... // reads exactly len bytes, throws on eof void readFully(byte[] dest,int off,int len); void readFully(byte[] dest) { readFully(dest,0,dest.length); } } All methods throw an IOException of some sort when a "real" error occurs, EOFException (or just EOF) could be a subclass of this or not. Java puts the formatted value reading in a subclass DataInputStream that wraps an instance of InputStream, I'm not sure about the rationale behind this. Sam
Jun 13 2004
parent Sam McCall <tunah.d tunah.net> writes:
Sam McCall wrote:
     // Reads a formatted value and throws on eof
     // Anyone got a good short name for this?
     void readData(out ubyte b);
     void readData(out int i);
     ...

binary format rather than just treating it as a stream of bytes. The point being that you expect there to be data in this format, so EOF *is* exceptional. Sam
Jun 13 2004
prev sibling parent "The Dr ... who?" <thedr who.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cad277$2bun$1 digitaldaemon.com...
 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...

What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be


when eof() is reached.  In my opinion, end of file is an expected rather than


exceptional condition.  If I had to wrap all my stream operations in try


I'd go nuts :)

I strongly disagree. Exceptions aren't just for errors, you know. There are all SORTS of non-error uses for exceptions. Just because something is expected, doesn't mean that an exception is not the best mechanism. Maybe I'm just more comfortable working with exceptions than some.

LOL! Glad to see someone else taking on the mantle of humility.
 To my mind,
 best practice behavior is to throw an exception on end-of-file, and a
 /different/ exception on error.

What about all the other minds, and their ways of conceptualising errors, return values + stream manipulation? Wrong, irrelevant, jejune?
 I like this way of working. It beats the hell out of having to test the return
 value all the time.

So it does, most of the time. But "all"? Big statement that. Humbly mine -- The Dr. da-da-da-dum, da-da-da-dum, da-da-da-dum, daaah da-da-da-dum, da-da-da-dum, da-da-da-dum, daaah woo-ooooo
Jun 12 2004
prev sibling parent "Walter" <newshound digitalmars.com> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:caar8o$28ba$1 digitaldaemon.com...
 One question about character output.  The docs say dchar is a 32-bit

 character but I'm pretty sure that it's currently implemented as wchar_t

 Windows right now (which would make it compatible with wchar).  Is there

 reason there shouldn't be methods to operate on all three char types:

 wchar, and dchar?

dchar is 32 bits wchar is 16 bits char is 8 bits The stream methods should all operate on chars as UTF-8 streams. The conversion to/from Windows 'W' functions should come as the last step.
Jun 14 2004
prev sibling next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca82sq$u9n$1 digitaldaemon.com>, Sean Kelly says...

As things stand, the basic D stream framework has an InputStream and
OutputStream interface and then a single Stream class that implements both of
these.  File, MemoryStream, SliceStream, and BufferedStream all inherit from
Stream, while BufferedFile inherits from BufferedStream.  Stream and File do not
buffer by default, BufferedStream acts as an adaptor class, and BufferedFile is
the buffered version of the File stream class.  Currently, all formatted i/o
goes through the printf method, with the read/write methods offering unformatted
i/o.  There is also an OutBuffer class (though no corresponding InBuffer) which
seems functionally similar to MemoryStream.

As things stand, the Phobos documentation mentions almost none of this. I didn't even know that file streams were unbuffered, or that the class BufferedStream even existed, until a few days ago when someone told me. Anyway, I'd want a function available() to be available in input streams. (Same as Java's java.io.InputStream). The implementation of this is /childs play/. It goes like this: In InputStream and Stream
    abstract uint available();

In unbuffered streams:
   override uint available() { return 0; }

In buffered streams:
   override uint available()
   {
       return numBytesRemainingInBuffer + s.available();
   }

where s is the underlying stream. (This is likely to be itself unbuffered, and so it's available() function is likely to return zero - but there will always be special cases, so it's best to assume not). As one special case, for example a string input stream could have available() return the remaining length of the string. In any case, the most important thing about available() is that it MUST NOT BLOCK. It doesn't matter if it underestimates, but it must never overestimate. Arcane Jill
Jun 09 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <ca90hu$2bgd$1 digitaldaemon.com>, Arcane Jill says...
As things stand, the Phobos documentation mentions almost none of this. I didn't
even know that file streams were unbuffered, or that the class BufferedStream
even existed, until a few days ago when someone told me.

At this point, I think it's still better to just look at the Phobos source code. There's a lot of stuff in there that's largely undocumented.
Anyway, I'd want a function available() to be available in input streams. (Same
as Java's java.io.InputStream). The implementation of this is /childs play/. It
goes like this:

In InputStream and Stream
    abstract uint available();

In unbuffered streams:
   override uint available() { return 0; }

In buffered streams:
   override uint available()
   {
       return numBytesRemainingInBuffer + s.available();
   }

where s is the underlying stream. (This is likely to be itself unbuffered, and so it's available() function is likely to return zero - but there will always be special cases, so it's best to assume not). As one special case, for example a string input stream could have available() return the remaining length of the string. In any case, the most important thing about available() is that it MUST NOT BLOCK. It doesn't matter if it underestimates, but it must never overestimate.

I was planning on adding available() to BufferedStream only, as it doesn't seem to make sense for an unbuffered stream class. I guess the alternative would be to document rules that unbuffered classes could follow, like return 0 if they don't know and -1 if at the end of file. Sean
Jun 10 2004
parent Sam McCall <tunah.d tunah.net> writes:
Sean Kelly wrote:

 In article <ca90hu$2bgd$1 digitaldaemon.com>, Arcane Jill says...
 
As things stand, the Phobos documentation mentions almost none of this. I didn't
even know that file streams were unbuffered, or that the class BufferedStream
even existed, until a few days ago when someone told me.

At this point, I think it's still better to just look at the Phobos source code. There's a lot of stuff in there that's largely undocumented.

for the ability to get a clean, readable reference, even if there's no actual description text for things (I've got the source).
 I was planning on adding available() to BufferedStream only, as it doesn't seem
 to make sense for an unbuffered stream class.  I guess the alternative would be
 to document rules that unbuffered classes could follow, like return 0 if they
 don't know and -1 if at the end of file.

known to be available, which may be an underestimate, or -1 if we're known to be at end of file (which isn't guaranteed even if we are?), then returning zero is _always_ safe, and returning nonzero values is a performance thing that shouldn't be relied on in general. (Of course if some class like BufferedStream is documented to always return positive when not at EOF, and -1 when at EOF, then you can rely on this when you know you've got a BufferedStream) Sam
Jun 13 2004
prev sibling next sibling parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Whoops! Posted this to the another thread, but I really meant to post here
instead ...

Perhaps you might consider checking out mango.io over at
http://www.dsource.org/forums/viewtopic.php?t=148 ? Mango.io attempts to
provide seamless integration of the following:

-Buffered I/O for Files, Sockets etc
-A variety of Readers and Writers for formatted IO, endian conversion, etc
-Multiple reader/writers upon the same buffer
-Tokens and Tokenizers for loosely formatted input (text lines, words,
numbers, etc)
-RegExp token wrapper
-Chained operations
-EOS checking is almost completely redundant (you rarely, if ever, need to
check)
-Both put/get and <</>> syntax
-Bidi equivalent of std.outbuffer (can use Readers/Writers per usual)
-Memory-mapped IO seamlessly overloads buffered IO
-Class serialization (to file, network, etc)
-CompositeIO framework for bracketing IO (such as commit semantics)
-Simply mechanism to bind your own classes into Reader/Writer framework
-Exceptions thrown for exceptional conditions (per discussion)
-Printf wrapper
-Mango.io has very little overhead, so it's *fast*. You can optionally
enable array-slicing when using CompositeReader. Tokens are always sliced,
but you can .dup them
-Includes typical file management tools (FileProxy, FileConduit), file path
manipulation (FilePath), etc
-Includes an RFC 2396 compliant URI specification
-Includes a simple Properties file reader
-Usual stdio singletons
-some <gasp> documentation!

There's a bunch more stuff in there also. Mango.io has been pretty well
beaten-up by three great guys since March, so it's quite stable and robust
now. Sure, there's things in there that could use improvement, but it's a
reasonable start. The design is somewhat reminiscent of Java NIO, but
without all the hideous IO warts that Java IO was originally saddled with. I
would contend that mango.io is dramatically cleaner than the Java design;
but that wouldn't be too hard, now would it? <g>

Mango.io is part of the Mango Tree, which currently includes these other
modules:

-exceptionally fast HTTP server (thanks mainly to D array slicing)
-Java like servlet engine (mango.io was built with this in mind)
-Log4J clone, including an HTML monitor/console for inspecting and setting
log/debug status at runtime (plus Chainsaw integration
-An HTTP client
-some simple caching mechanisms, including virtual caching to disk.

I invite you to take a look, and comment. It'd be great if we could turn
mango.io into a truly excellent IO platform for D ...

- Kris


"Sean Kelly" <sean f4.ca> wrote in message
news:ca82sq$u9n$1 digitaldaemon.com...
 I think the D streams provide a good base but are missing some of the more
 complex features that I miss from C++, mostly to do with formatted i/o.

 raises the rather large issue of localization which I'm going to ignore

 moment.

 As things stand, the basic D stream framework has an InputStream and
 OutputStream interface and then a single Stream class that implements both

 these.  File, MemoryStream, SliceStream, and BufferedStream all inherit

 Stream, while BufferedFile inherits from BufferedStream.  Stream and File

 buffer by default, BufferedStream acts as an adaptor class, and

 the buffered version of the File stream class.  Currently, all formatted

 goes through the printf method, with the read/write methods offering

 i/o.  There is also an OutBuffer class (though no corresponding InBuffer)

 seems functionally similar to MemoryStream.

 I'd like separate InputStream, OutputStream, and Stream base classes to

 support for streams where only input or output makes sense.  To make it

 these are base classes and prevent naming problems with the interfaces I

 these names: InputStreamBase, OutputStreamBase, and StreamBase (or
 InputOutputStreamBase, though that's pretty darn verbose).  Because of the
 single inheritance rule, StreamBase won't be able to inherit from
 InputStreamBase and OutputStreamBase which will liekly result in some

 with the design to avoid code duplication.  These classes will be

 abstract base classes.  If I wanted to nit-pick I might suggest renaming

 FileStream for consistency.

 In addition to the existing functionality I propose new formatted i/o

 for all types that already have unformatted methods.  Apart from a few

 these can forward most of the work to the existing printf method.  The

 methods would be "get" and "put" with the possibility of adding overloaded
 operator support in the future.  I'd also like to see something similar to

 callback mechanism in C++ streams so arbitrary classes can hook in

 information.  And Jill had mentioned wanting something similar to

 from C++ streams so I suspect this list will grow a bit.

 As for open issues, someone with more experience than myself should

 address localization.  At the very least, there should be some support for
 proper numeric formatting, but that raises the issue of how formatting

 will be supplied to the stream classes and what else the localization

 should cover.

 I'm willing to go and do this myself, but as it's a modification of

 Phobos stuff I wanted to get some feedback.  Does anyone have any

 suggestions?  Assuming the design doesn't suck will it have a chance for
 inclusion?


 Sean

Jun 15 2004
prev sibling parent reply surferjeff excite.com writes:
The interface for InputStream and OutputStream is huge.  I often have to create
my own streambufs in C++, which only requires implementing about 5 methods.  If
I want to write my own class that implements InputStream, then I have to
implement about 50 methods, and another 50 if I want to implement OutputStream
too!

It would useful if there were smaller interfaces.  For example, a read() method
and eof() for input, and that's it.  Then have a util class that takes one of
these small interfaces and implements the full 50 methods in the bigger
interface.

Yes, I could do this in my own code, but I think many people will have to
re-invent this wheel in D, and each re-invention will be different and not
interchangeable.
Feb 25 2006
next sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
surferjeff excite.com wrote:
 The interface for InputStream and OutputStream is huge.  I often have to create
 my own streambufs in C++, which only requires implementing about 5 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement OutputStream
 too!
 
 It would useful if there were smaller interfaces.  For example, a read() method
 and eof() for input, and that's it.  Then have a util class that takes one of
 these small interfaces and implements the full 50 methods in the bigger
 interface.
 
 Yes, I could do this in my own code, but I think many people will have to
 re-invent this wheel in D, and each re-invention will be different and not
 interchangeable.
 
 

Phobos sucks. There I said it. :-) In other news, there is a library called Mango on www.dsource.org which you may find useful.
Feb 25 2006
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
<surferjeff excite.com> wrote in message 
news:dtribg$eui$1 digitaldaemon.com...
 The interface for InputStream and OutputStream is huge.  I often have to 
 create
 my own streambufs in C++, which only requires implementing about 5 
 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement 
 OutputStream
 too!

 It would useful if there were smaller interfaces.  For example, a read() 
 method
 and eof() for input, and that's it.  Then have a util class that takes one 
 of
 these small interfaces and implements the full 50 methods in the bigger
 interface.

 Yes, I could do this in my own code, but I think many people will have to
 re-invent this wheel in D, and each re-invention will be different and not
 interchangeable.

Subclass Stream or FilterStream. You'll probably have to override 3 methods: readBlock, writeBlock and seek. The interfaces are primarily for users to restrict their API - not for library writers to shoe-horn their classes into. Look at any of the small stream classes in std.stream to get an idea how easy it is to do.
Feb 26 2006
parent reply surferjeff excite.com writes:
In article <dtsbc3$1c8g$1 digitaldaemon.com>, Ben Hinkle says...
<surferjeff excite.com> wrote in message 
news:dtribg$eui$1 digitaldaemon.com...
 The interface for InputStream and OutputStream is huge.  I often have to 
 create
 my own streambufs in C++, which only requires implementing about 5 
 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement 
 OutputStream
 too!

 It would useful if there were smaller interfaces.  For example, a read() 
 method
 and eof() for input, and that's it.  Then have a util class that takes one 
 of
 these small interfaces and implements the full 50 methods in the bigger
 interface.

 Yes, I could do this in my own code, but I think many people will have to
 re-invent this wheel in D, and each re-invention will be different and not
 interchangeable.

Subclass Stream or FilterStream. You'll probably have to override 3 methods: readBlock, writeBlock and seek. The interfaces are primarily for users to restrict their API - not for library writers to shoe-horn their classes into. Look at any of the small stream classes in std.stream to get an idea how easy it is to do.

I see. Not so bad. Thanks. Instead of the "Inherit 20 methods, implement 3 pure virtual methods" approach, how about "implement the same 3 pure virtual methods in an interface, pass the interface to a class that uses it to implement the 20 methods." I've used the former pattern a lot in C++, and have found it difficult to reuse. I recently learned the latter pattern in Objective-C and Cocoa, and have found it much easier to reuse.
Mar 15 2006
next sibling parent Sean Kelly <sean f4.ca> writes:
surferjeff excite.com wrote:
 
 I see. Not so bad.  Thanks.
 
 Instead of the "Inherit 20 methods, implement 3 pure virtual methods" approach,
 how about "implement the same 3 pure virtual methods in an interface, pass the
 interface to a class that uses it to implement the 20 methods."  I've used the
 former pattern a lot in C++, and have found it difficult to reuse.  I recently
 learned the latter pattern in Objective-C and Cocoa, and have found it much
 easier to reuse.

Something like: class MyStreamImpl() { void readBlock(); void writeBlock(); seek(); } class Stream( Impl ) { mixin Impl!() impl; alias impl.readBlock readBlock; alias impl.writeBlock writeBlock; alias impl.seek seek; ... } alias Stream!(MyStreamImpl) MyStream; Sean
Mar 15 2006
prev sibling parent "Ben Hinkle" <bhinkle mathworks.com> writes:
<surferjeff excite.com> wrote in message 
news:dvaupb$1ok0$1 digitaldaemon.com...
 In article <dtsbc3$1c8g$1 digitaldaemon.com>, Ben Hinkle says...
<surferjeff excite.com> wrote in message
news:dtribg$eui$1 digitaldaemon.com...
 The interface for InputStream and OutputStream is huge.  I often have to
 create
 my own streambufs in C++, which only requires implementing about 5
 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement
 OutputStream
 too!

 It would useful if there were smaller interfaces.  For example, a read()
 method
 and eof() for input, and that's it.  Then have a util class that takes 
 one
 of
 these small interfaces and implements the full 50 methods in the bigger
 interface.

 Yes, I could do this in my own code, but I think many people will have 
 to
 re-invent this wheel in D, and each re-invention will be different and 
 not
 interchangeable.

Subclass Stream or FilterStream. You'll probably have to override 3 methods: readBlock, writeBlock and seek. The interfaces are primarily for users to restrict their API - not for library writers to shoe-horn their classes into. Look at any of the small stream classes in std.stream to get an idea how easy it is to do.

I see. Not so bad. Thanks. Instead of the "Inherit 20 methods, implement 3 pure virtual methods" approach, how about "implement the same 3 pure virtual methods in an interface, pass the interface to a class that uses it to implement the 20 methods." I've used the former pattern a lot in C++, and have found it difficult to reuse. I recently learned the latter pattern in Objective-C and Cocoa, and have found it much easier to reuse.

Sure, one can trade-off flexibility and ease-of-use. For example std.stream.File could just implement the 3 pure virtual methods and then user code would write Stream file = new Stream(new File("foo.txt")); So the flexibility of multiple classes and interfaces results in user code having to do more to hook things up. There are many many ways to slice things up. At one point I posted a suggestion to have a Stream subclass that accepted delegates for the 3 methods and then one wouldn't have to write a class at all - one can just write 3 nested functions and pass those to the constructor. For example void foo() { char buf[100]; size_t readBlock(void* buffer, size_t size) { ... do something with buf ... } .. 2 more nested functions for writeBlock and seek ... Stream s = new DelegateStream(&readBlock, &writeBlock, &seek); ... use s as any other stream ... } but I don't recall that suggestion getting any lovin' so I dropped it. I think if you have concrete suggestions go ahead and post them, see what kind of response you get and based on the response make changes and submit them to Walter and see how things go. -Ben
Mar 16 2006