www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Streams: an open discussion

reply Sean Kelly <sean f4.ca> writes:
I think the D streams provide a good base but are missing some of the more
complex features that I miss from C++, mostly to do with formatted i/o.  This
raises the rather large issue of localization which I'm going to ignore for the
moment.

As things stand, the basic D stream framework has an InputStream and
OutputStream interface and then a single Stream class that implements both of
these.  File, MemoryStream, SliceStream, and BufferedStream all inherit from
Stream, while BufferedFile inherits from BufferedStream.  Stream and File do not
buffer by default, BufferedStream acts as an adaptor class, and BufferedFile is
the buffered version of the File stream class.  Currently, all formatted i/o
goes through the printf method, with the read/write methods offering unformatted
i/o.  There is also an OutBuffer class (though no corresponding InBuffer) which
seems functionally similar to MemoryStream.

I'd like separate InputStream, OutputStream, and Stream base classes to allow
support for streams where only input or output makes sense.  To make it clear
these are base classes and prevent naming problems with the interfaces I propose
these names: InputStreamBase, OutputStreamBase, and StreamBase (or
InputOutputStreamBase, though that's pretty darn verbose).  Because of the
single inheritance rule, StreamBase won't be able to inherit from
InputStreamBase and OutputStreamBase which will liekly result in some toying
with the design to avoid code duplication.  These classes will be obviously be
abstract base classes.  If I wanted to nit-pick I might suggest renaming File to
FileStream for consistency.

In addition to the existing functionality I propose new formatted i/o methods
for all types that already have unformatted methods.  Apart from a few instances
these can forward most of the work to the existing printf method.  The default
methods would be "get" and "put" with the possibility of adding overloaded
operator support in the future.  I'd also like to see something similar to the
callback mechanism in C++ streams so arbitrary classes can hook in formatting
information.  And Jill had mentioned wanting something similar to readsome()
from C++ streams so I suspect this list will grow a bit.

As for open issues, someone with more experience than myself should probably
address localization.  At the very least, there should be some support for
proper numeric formatting, but that raises the issue of how formatting rules
will be supplied to the stream classes and what else the localization rules
should cover.

I'm willing to go and do this myself, but as it's a modification of existing
Phobos stuff I wanted to get some feedback.  Does anyone have any objections or
suggestions?  Assuming the design doesn't suck will it have a chance for
inclusion?


Sean
Jun 09 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Sean Kelly wrote:


 I think the D streams provide a good base but are missing some of the more
 complex features that I miss from C++, mostly to do with formatted i/o. 
 This raises the rather large issue of localization which I'm going to
 ignore for the moment.
 
 As things stand, the basic D stream framework has an InputStream and
 OutputStream interface and then a single Stream class that implements both
 of
 these.  File, MemoryStream, SliceStream, and BufferedStream all inherit
 from
 Stream, while BufferedFile inherits from BufferedStream.  Stream and File
 do not buffer by default, BufferedStream acts as an adaptor class, and
 BufferedFile is
 the buffered version of the File stream class.  Currently, all formatted
 i/o goes through the printf method, with the read/write methods offering
 unformatted
 i/o.  There is also an OutBuffer class (though no corresponding InBuffer)
 which seems functionally similar to MemoryStream.
 
 I'd like separate InputStream, OutputStream, and Stream base classes to
 allow
 support for streams where only input or output makes sense.  To make it
 clear these are base classes and prevent naming problems with the
 interfaces I propose these names: InputStreamBase, OutputStreamBase, and
 StreamBase (or
 InputOutputStreamBase, though that's pretty darn verbose).  Because of the
 single inheritance rule, StreamBase won't be able to inherit from
 InputStreamBase and OutputStreamBase which will liekly result in some
 toying
 with the design to avoid code duplication.  These classes will be
 obviously be
 abstract base classes.  If I wanted to nit-pick I might suggest renaming
 File to FileStream for consistency.
 
 In addition to the existing functionality I propose new formatted i/o
 methods
 for all types that already have unformatted methods.  Apart from a few
 instances
 these can forward most of the work to the existing printf method.  The
 default methods would be "get" and "put" with the possibility of adding
 overloaded
 operator support in the future.  I'd also like to see something similar to
 the callback mechanism in C++ streams so arbitrary classes can hook in
 formatting
 information.  And Jill had mentioned wanting something similar to
 readsome() from C++ streams so I suspect this list will grow a bit.
 
 As for open issues, someone with more experience than myself should
 probably
 address localization.  At the very least, there should be some support for
 proper numeric formatting, but that raises the issue of how formatting
 rules will be supplied to the stream classes and what else the
 localization rules should cover.
 
 I'm willing to go and do this myself, but as it's a modification of
 existing
 Phobos stuff I wanted to get some feedback.  Does anyone have any
 objections or
 suggestions?  Assuming the design doesn't suck will it have a chance for
 inclusion?
 
 
 Sean


It would be nice to have mixins DefaultInputStream and DefaultOutputStream
that would contain the default implementations from Stream and then Stream
would be a mixin of those and the seeking and other odds and ends. That
would make it easier to implement InputStream and OutputStream since users
would just have to supply readBlock and writeBlock, respectively. So for
example to define a FooInputStream
class FooInputStream : InputStream {
  mixin DefaultInputStream; // ultimately call readBlock
  uint readBlock(...) {...}
  ... whatever else ...
}
Jun 09 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <ca8kcl$1nvb$1 digitaldaemon.com>, Ben Hinkle says...

It would be nice to have mixins DefaultInputStream and DefaultOutputStream
that would contain the default implementations from Stream and then Stream
would be a mixin of those and the seeking and other odds and ends.


What a wonderful idea :)  I knew those mixins would come in handy for something!

I tend to develop ideas by coding so I've gone ahead and started the
modifications.  I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.  At the moment I'm going with
having the streams castable to bit as an easy way to check state, much the same
way C++ streams work.  Otherwise the existing methods are staying pretty much
as-is.  I'll post more in a few days once I get more of the grunt work done.

One question about character output.  The docs say dchar is a 32-bit unicode
character but I'm pretty sure that it's currently implemented as wchar_t in
Windows right now (which would make it compatible with wchar).  Is there any
reason there shouldn't be methods to operate on all three char types: char,
wchar, and dchar?


Sean
Jun 10 2004
next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
On Thu, 10 Jun 2004 23:35:52 +0000 (UTC), Sean Kelly <sean f4.ca>
wrote:


In article <ca8kcl$1nvb$1 digitaldaemon.com>, Ben Hinkle says...

It would be nice to have mixins DefaultInputStream and DefaultOutputStream
that would contain the default implementations from Stream and then Stream
would be a mixin of those and the seeking and other odds and ends.


What a wonderful idea :)  I knew those mixins would come in handy for something!

I tend to develop ideas by coding so I've gone ahead and started the
modifications.  


cool. That's usually the best way to gives ideas focus.


I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.  


Are you talking about the ReadError in readExact? I think that error
should stay since readBlock returns the number of bytes read if there
isn't enough data. If one asks to read exactly 10 bytes and that is
impossible it should throw an error.


At the moment I'm going with
having the streams castable to bit as an easy way to check state, much the same
way C++ streams work.  


Two things come to mind: 
1) Can opCast be implicit? The DMD spec says "Overloading the cast
operator does not affect implicit casts, it only applies to explicit
casts." Since writing cast(bit)foo isn't more readable than foo.eof()
I'd stay away from opCast for bit.
2) if (foo) ... for any other object checks that foo is non-null.
Making it anything else would be wierd.


Otherwise the existing methods are staying pretty much
as-is.  I'll post more in a few days once I get more of the grunt work done.


If you have time and/or desire check out the MmFile wrapper for
streams that I mentioned on digitalmars.D.bugs available at
http://home.comcast.net/~benhinkle/stream.d
Down near the bottom there is ArrayStream(Buffer) that wraps
"array-like" types with a stream API.


One question about character output.  The docs say dchar is a 32-bit unicode
character but I'm pretty sure that it's currently implemented as wchar_t in
Windows right now (which would make it compatible with wchar).  Is there any
reason there shouldn't be methods to operate on all three char types: char,
wchar, and dchar?


I wondered that, too, when I was poking around. It could be that
stream.d came before dchar? I don't know. I haven't thought very much
about it but adding dchar support seems reasonable to me. I can't
think of any big reason not to.


Sean
Jun 10 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <3i7ic0dlu2mdmb8q47bmvro8n4og30jvad 4ax.com>, Ben Hinkle says...

On Thu, 10 Jun 2004 23:35:52 +0000 (UTC), Sean Kelly <sean f4.ca>
wrote:


I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.  


Are you talking about the ReadError in readExact? I think that error
should stay since readBlock returns the number of bytes read if there
isn't enough data. If one asks to read exactly 10 bytes and that is
impossible it should throw an error.


What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be throw
when eof() is reached.  In my opinion, end of file is an expected rather than an
exceptional condition.  If I had to wrap all my stream operations in try blocks
I'd go nuts :)


At the moment I'm going with
having the streams castable to bit as an easy way to check state, much the same
way C++ streams work.  


Two things come to mind: 
1) Can opCast be implicit? The DMD spec says "Overloading the cast
operator does not affect implicit casts, it only applies to explicit
casts." Since writing cast(bit)foo isn't more readable than foo.eof()
I'd stay away from opCast for bit.


Darn.  I'd forgotten about this.  And I so liked the C++ semantics:

while( istr.get(x) );

This would work if all operations returned a fail state, but it's much nicer to
have them return the stream itself so operations can be chained.  I suppose this
works however:

while( !istr.get(x).fail() );

and it is perhaps more clear.


If you have time and/or desire check out the MmFile wrapper for
streams that I mentioned on digitalmars.D.bugs available at
http://home.comcast.net/~benhinkle/stream.d
Down near the bottom there is ArrayStream(Buffer) that wraps
"array-like" types with a stream API.


Will do.

I have to say, this lack of multiple inheritance has been interesting to deal
with.  As both InputStream and OutputStream share some common features (state
tracking) it made sense to have them both inherit from a common interface.  But
as the IOStream class was to incorporate stuff from both of them this meant a
diamond-shaped inheritance tree, which I think tosses the idea of having only
two mixins.  I haven't tested it, but assuming this code:

class CommonBase
{
int x;
}


class DerivedA
{
mixin CommonBase;
}

template DerivedB
{
mixin CommonBase;
}

template Derived
{
mixin DerivedA;
mixin DerivedB;
}

Derived will end up with two instances of x defined, won't it?  Or perhaps a
compiler error?  Or is the code intelligent enough to gurantee that the stuff in
CommonBase is only imported once?


Sean
Jun 11 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
err... replace "template" with "class" in my example.

Sean
Jun 11 2004
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:cacidg$1jn3$1 digitaldaemon.com...

 In article <3i7ic0dlu2mdmb8q47bmvro8n4og30jvad 4ax.com>, Ben Hinkle


says...

On Thu, 10 Jun 2004 23:35:52 +0000 (UTC), Sean Kelly <sean f4.ca>
wrote:


I decided to refactor the design a bit so the streams won't
throw exceptions on eof() or on read errors.


Are you talking about the ReadError in readExact? I think that error
should stay since readBlock returns the number of bytes read if there
isn't enough data. If one asks to read exactly 10 bytes and that is
impossible it should throw an error.


 What bothered me about the exception throwing in Stream was that since all
 methods are implemented in terms of readExact, an exception will always be


throw

 when eof() is reached.  In my opinion, end of file is an expected rather


than an

 exceptional condition.  If I had to wrap all my stream operations in try


blocks

 I'd go nuts :)


Eof in the middle of reading an int (for example) shouldn't be expected,
though. If one is trying to read an int and you only get half an int and a
return value of false then every call to read(out int) will have to be
wrapped in "if" tests instead of try-catches. Reading multiple values would
look like
 if (!s.read(x1) || !s.read(x2) || !s.read(x3)) {evasive action}
instead of
 try{
  s.read(x1);
  s.read(x2);
  s.read(x3);
 } catch (ReadError re) { evasive action }

Either way I think eof should be the "usual" way of checking if any data
remains.

[snip]


 I have to say, this lack of multiple inheritance has been interesting to


deal

 with.  As both InputStream and OutputStream share some common features


(state

 tracking) it made sense to have them both inherit from a common interface.


But

 as the IOStream class was to incorporate stuff from both of them this


meant a

 diamond-shaped inheritance tree, which I think tosses the idea of having


only

 two mixins.  I haven't tested it, but assuming this code:

 class CommonBase
 {
 int x;
 }


 class DerivedA
 {
 mixin CommonBase;
 }

 template DerivedB
 {
 mixin CommonBase;
 }

 template Derived
 {
 mixin DerivedA;
 mixin DerivedB;
 }

 Derived will end up with two instances of x defined, won't it?  Or perhaps


a

 compiler error?  Or is the code intelligent enough to gurantee that the


stuff in

 CommonBase is only imported once?


Is it possible to just have the mixins be the non-overlapping parts? I'm
looking at Stream class and the routines that implement InputStream only
call readBlock and eof to do their thing. I can't see any overlap with the
implementations of OutputStream. In other words, lines 219-796 (plus line
209 for "readable") form DefaultInputStream and lines 802-932 (plus 210)
form DefaultOutputStream. So I'm picturing
 interface InputStream{...}
 interface OutputStream {...}
 template DefaultInputStream() {...}
 template DefaultOutputStream() {...}
 class Stream {
    mixin DefaultInputStream;
    mixin DefaultOutputStream;
    ...
 }
That way everything is backwards compatible with the existing stream.d.
Users can't instantiate a DefaultInputStream by itself but that's ok.



 Sean
Jun 11 2004
next sibling parent "Ben Hinkle" <bhinkle mathworks.com> writes:
  interface InputStream{...}
  interface OutputStream {...}
  template DefaultInputStream() {...}
  template DefaultOutputStream() {...}
  class Stream {
     mixin DefaultInputStream;
     mixin DefaultOutputStream;
     ...
  }


oops - I forgot that Stream wants to implement InputStream and OutputStream
  class Stream : InputStream, OutputStream {
Jun 11 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <cact25$2406$1 digitaldaemon.com>, Ben Hinkle says...

Eof in the middle of reading an int (for example) shouldn't be expected,
though.


Agreed.  But what about Eof as a result of reading a char?


 If one is trying to read an int and you only get half an int and a
return value of false then every call to read(out int) will have to be
wrapped in "if" tests instead of try-catches. Reading multiple values would
look like
 if (!s.read(x1) || !s.read(x2) || !s.read(x3)) {evasive action}
instead of
 try{
  s.read(x1);
  s.read(x2);
  s.read(x3);
 } catch (ReadError re) { evasive action }


It doesn't have to be quite so bad.  I would probably do something like this:

if( s.read(x1).read(x2).read(x3).fail() ) {evasive action}

I admit this is a matter of preference, and there are certainly times when I
want to throw an exception on read failure, but the fact that I don't *always*
want an exception thrown makes me leery of building it into Stream.  An
alternative would be to allow the user to specify that he wants exceptions to be
thrown by setting a flag.


Either way I think eof should be the "usual" way of checking if any data
remains.


I agree.  Part of my problem was that I didn't investigate how eof() worked
until just now.  Here's the call:

bit eof() { return position() == size(); }

This does allow for foreknowledge of eof() location (a feature I wasn't aware
of), but it does this by scanning to the end of the stream and then resetting
the read position.  In some cases this operation can be quite expensive or just
plain impossible.  An alternative might be to specialize such classes to
read-ahead by one byte on every read to check for eof(), but this will wreak
havoc with streams that may block.  I'm personally used to the method of the C
read() function where it will return 0 on eof(), in which case the stream class
would set an eof() flag and return a failure (or throw an exception I suppose).
But this obviously eliminates any foreknowledge of when eof() has been reached,
moving detection back into the read method.

But again this is all my personal style and there may be a better way to do it
(not to mention Mango, which follows yet another approach).  The other
consequence of my approach is that it isn't compatible with code written against
the existing stream model, which is something I was hoping to avoid.


Is it possible to just have the mixins be the non-overlapping parts? I'm
looking at Stream class and the routines that implement InputStream only
call readBlock and eof to do their thing. I can't see any overlap with the
implementations of OutputStream.


Definately.  The overlap would only be new code and that could all be placed in
a separate mixin.  My question was as much academic as practical.  I'm truly
curious whether such a design would result in multiply defined data or not.  In
C/C++ I would say yes, but with D's module-level linkage model I'm not so sure.


That way everything is backwards compatible with the existing stream.d.
Users can't instantiate a DefaultInputStream by itself but that's ok.


I'm kind of torn so far as backwards compatibility is concerned.  There are
things I don't like about the original design, but at the same time I don't want
to propose yet another API format.  However, the differences truly aren't that
great so I'm going to go with them for now and see how they turn out.  I admit
I'm using this as an excuse to become more familiar with the language as much as
to add features to the stream lib, so whatever happens I won't consider it a
wasted effort.


Sean
Jun 11 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...


What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be throw
when eof() is reached.  In my opinion, end of file is an expected rather than an
exceptional condition.  If I had to wrap all my stream operations in try blocks
I'd go nuts :)


I strongly disagree. Exceptions aren't just for errors, you know. There are all
SORTS of non-error uses for exceptions. Just because something is expected,
doesn't mean that an exception is not the best mechanism.

Maybe I'm just more comfortable working with exceptions than some. To my mind,
best practice behavior is to throw an exception on end-of-file, and a
/different/ exception on error.

I like this way of working. It beats the hell out of having to test the return
value all the time.

Arcane Jill
Jun 11 2004
next sibling parent reply Derek <derek psyc.ward> writes:
On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:


 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...
 

What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be throw
when eof() is reached.  In my opinion, end of file is an expected rather than an
exceptional condition.  If I had to wrap all my stream operations in try blocks
I'd go nuts :)


 
 I strongly disagree. Exceptions aren't just for errors, you know. There are all
 SORTS of non-error uses for exceptions. Just because something is expected,
 doesn't mean that an exception is not the best mechanism.
 
 Maybe I'm just more comfortable working with exceptions than some. To my mind,
 best practice behavior is to throw an exception on end-of-file, and a
 /different/ exception on error.
 
 I like this way of working. It beats the hell out of having to test the return
 value all the time.
 
 Arcane Jill


But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'. Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files. So although one can use the D
exception mechanism to handle eof, the eof event is not really an
exception. I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?

-- 
Derek
Melbourne, Australia
Jun 11 2004
next sibling parent Andy Friesen <andy ikagames.com> writes:
Derek wrote:


 But isn't this sort of like the 'bit' verses 'bool' thing. They are
 different animals but a 'bit' can be used to simulate a 'bool'. Likewise,
 reaching the end of file is not an exceptional event, in fact it is quite
 normal in most cases involving stream files. So although one can use the D
 exception mechanism to handle eof, the eof event is not really an
 exception. I know this just semantics, but ... must we be forced into using
 exceptions to handle eof, just like we are forced to use 'bit' to handle
 'bool'?


If you think about it in terms of how many times the method is called 
and what fraction of those calls will be a read past the end of the 
stream, it doesn't seem too much of a stretch to call hitting EOF 
"exceptional".

Further, this model ensures that, unlike C stdio, forgetting to check 
EOF causes reliable, reproducable behaviour.

  -- andy
Jun 11 2004
prev sibling next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Derek wrote:


 On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:
 

 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...
 

What bothered me about the exception throwing in Stream was that since
all methods are implemented in terms of readExact, an exception will
always be throw
when eof() is reached.  In my opinion, end of file is an expected rather
than an
exceptional condition.  If I had to wrap all my stream operations in try
blocks I'd go nuts :)


 
 I strongly disagree. Exceptions aren't just for errors, you know. There
 are all SORTS of non-error uses for exceptions. Just because something is
 expected, doesn't mean that an exception is not the best mechanism.
 
 Maybe I'm just more comfortable working with exceptions than some. To my
 mind, best practice behavior is to throw an exception on end-of-file, and
 a /different/ exception on error.
 
 I like this way of working. It beats the hell out of having to test the
 return value all the time.
 
 Arcane Jill


 
 But isn't this sort of like the 'bit' verses 'bool' thing. They are
 different animals but a 'bit' can be used to simulate a 'bool'. Likewise,
 reaching the end of file is not an exceptional event, in fact it is quite
 normal in most cases involving stream files. So although one can use the D
 exception mechanism to handle eof, the eof event is not really an
 exception. I know this just semantics, but ... must we be forced into
 using exceptions to handle eof, just like we are forced to use 'bit' to
 handle 'bool'?


Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.
Jun 11 2004
next sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <caddo3$2spa$1 digitaldaemon.com>, Ben Hinkle says...

Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.


If the user calls a function that expects a 4-byte chunk of data and the
function hits eof() after 2 bytes then the stream did not behave in an expected
manner.  Whether this is sufficiently bad to be worth an exception is really a
matter of opinion, and no one agrees on how exceptions should be used anyway.
Between what you and Andy said however, I'm inclined to agree that an exception
may be warranted to protect the uncautious user from weird errors, even if I
might not like the behavior myself :)

How about this: if the stream expects a fixed-size chunk >1 byte and hits eof()
before reading the expected number of bytes then it throws an exception.  If the
stream expects only 1 byte and hits eof() (ie. the read failed entirely) then it
sets a failure state and returns without throwing.  The only thing about this
tactic that worries me is that it seems inconsistent and thus confusing.

Here's an alternate: the unformatted read functions will throw an exception on
eof() while the formatted read functions will not.  The assumption being that if
a programmer is using the unformatted functions then he's likely reading a data
file with a known format and any violations of that format would be exceptional.
If the programmer really wants to check for eof() to avoid this problem he could
do:

if( istr.peek().eof() )


Sean
Jun 11 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
Okay, I've been thinking about this and you're slowly winning me over.  I do
write a lot of code like this:

if( !istr.read( x ) ) throw new MyError();

and it would make a lot more sense to just expect the stream itself to do it.
But I'm still on the fence with the end of file issue.  If eof() can always be
evaluated proactively (ie. so I can test for it when I need to and thus avoid an
exception) then I'd say let's go ahead and throw excpetions on eof() during a
read.  The C functions have the eof() function, and I think in Windows I can
call ReadFile with a read length of 0.  Sockets can return closed status.  What
about other streams like USB ports and such?  Darn, I think I've just been
convinced ;)

Sean
Jun 11 2004
prev sibling next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Sean Kelly wrote:


 In article <caddo3$2spa$1 digitaldaemon.com>, Ben Hinkle says...

Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.


 
 If the user calls a function that expects a 4-byte chunk of data and the
 function hits eof() after 2 bytes then the stream did not behave in an
 expected
 manner.  Whether this is sufficiently bad to be worth an exception is
 really a matter of opinion, and no one agrees on how exceptions should be
 used anyway. Between what you and Andy said however, I'm inclined to agree
 that an exception may be warranted to protect the uncautious user from
 weird errors, even if I might not like the behavior myself :)
 
 How about this: if the stream expects a fixed-size chunk >1 byte and hits
 eof()
 before reading the expected number of bytes then it throws an exception. 
 If the stream expects only 1 byte and hits eof() (ie. the read failed
 entirely) then it
 sets a failure state and returns without throwing.  The only thing about
 this tactic that worries me is that it seems inconsistent and thus
 confusing.


readBlock is an option - though not at convenient.


 Here's an alternate: the unformatted read functions will throw an
 exception on
 eof() while the formatted read functions will not.  The assumption being
 that if a programmer is using the unformatted functions then he's likely
 reading a data file with a known format and any violations of that format
 would be exceptional. If the programmer really wants to check for eof() to
 avoid this problem he could do:
 
 if( istr.peek().eof() )


Looking at Stream.scanf it assumes one can read one character after the
field being read. For example, the code to parse "%d" is
  while (isdigit(c) && width)
  {
    n = n * 10 + (c - '0');
    width--;
    c = getc();
    count++;
  }
and getc() will throw an exception at eof(). That looks like a bug in scanf. 
I haven't tried any example, though, so I could just be reading the code
incorrectly. It would be a little more involved to check for eof but the
code would be more robust. Do you think that is a reasonable set of changes
to make? 


 
 Sean
Jun 11 2004
parent Sean Kelly <sean f4.ca> writes:
In article <cadj30$2b2$1 digitaldaemon.com>, Ben Hinkle says...

Looking at Stream.scanf it assumes one can read one character after the
field being read. For example, the code to parse "%d" is
  while (isdigit(c) && width)
  {
    n = n * 10 + (c - '0');
    width--;
    c = getc();
    count++;
  }
and getc() will throw an exception at eof(). That looks like a bug in scanf. 
I haven't tried any example, though, so I could just be reading the code
incorrectly. It would be a little more involved to check for eof but the
code would be more robust. Do you think that is a reasonable set of changes
to make? 


Definately.  I already made a similar change in getLine.

Sean
Jun 12 2004
prev sibling parent "The Dr ... who?" <thedr who.com> writes:
All this stems from the fact that what is currently implemented, or what is
being
proposed, - pardon my ignorance of both - is a stream for all seasons. The
times - all too many - I've written such things in the past I've separated an
exception-free, byte-only stream lower layer, upon which may be layered whatever
else one wants: object-based streams, streams that'll read ints/doubles/etc.,
text streams (\r\n <=> \n), etc.

Surely that's the way to go?

"Sean Kelly" <sean f4.ca> wrote in message
news:cadg1r$2vpf$1 digitaldaemon.com...

 In article <caddo3$2spa$1 digitaldaemon.com>, Ben Hinkle says...

Reaching the end of a file is not exceptional. Reaching the end of a file
before expected is exceptional.


 If the user calls a function that expects a 4-byte chunk of data and the
 function hits eof() after 2 bytes then the stream did not behave in an expected
 manner.  Whether this is sufficiently bad to be worth an exception is really a
 matter of opinion, and no one agrees on how exceptions should be used anyway.
 Between what you and Andy said however, I'm inclined to agree that an exception
 may be warranted to protect the uncautious user from weird errors, even if I
 might not like the behavior myself :)

 How about this: if the stream expects a fixed-size chunk >1 byte and hits eof()
 before reading the expected number of bytes then it throws an exception.  If


the

 stream expects only 1 byte and hits eof() (ie. the read failed entirely) then


it

 sets a failure state and returns without throwing.  The only thing about this
 tactic that worries me is that it seems inconsistent and thus confusing.

 Here's an alternate: the unformatted read functions will throw an exception on
 eof() while the formatted read functions will not.  The assumption being that


if

 a programmer is using the unformatted functions then he's likely reading a data
 file with a known format and any violations of that format would be


exceptional.

 If the programmer really wants to check for eof() to avoid this problem he


could

 do:

 if( istr.peek().eof() )


 Sean
Jun 12 2004
prev sibling parent reply Regan Heath <regan netwin.co.nz> writes:
On Fri, 11 Jun 2004 19:03:36 -0400, Ben Hinkle <bhinkle4 juno.com> wrote:

 Derek wrote:


 On Fri, 11 Jun 2004 19:46:47 +0000 (UTC), Arcane Jill wrote:


 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...


 What bothered me about the exception throwing in Stream was that since
 all methods are implemented in terms of readExact, an exception will
 always be throw
 when eof() is reached.  In my opinion, end of file is an expected 
 rather
 than an
 exceptional condition.  If I had to wrap all my stream operations in 
 try
 blocks I'd go nuts :)


 I strongly disagree. Exceptions aren't just for errors, you know. There
 are all SORTS of non-error uses for exceptions. Just because something 
 is
 expected, doesn't mean that an exception is not the best mechanism.

 Maybe I'm just more comfortable working with exceptions than some. To 
 my
 mind, best practice behavior is to throw an exception on end-of-file, 
 and
 a /different/ exception on error.

 I like this way of working. It beats the hell out of having to test the
 return value all the time.

 Arcane Jill


 But isn't this sort of like the 'bit' verses 'bool' thing. They are
 different animals but a 'bit' can be used to simulate a 'bool'. 
 Likewise,
 reaching the end of file is not an exceptional event, in fact it is 
 quite
 normal in most cases involving stream files. So although one can use 
 the D
 exception mechanism to handle eof, the eof event is not really an
 exception. I know this just semantics, but ... must we be forced into
 using exceptions to handle eof, just like we are forced to use 'bit' to
 handle 'bool'?


 Reaching the end of a file is not exceptional. Reaching the end of a file
 before expected is exceptional.


Exactly.

Here is an example of how I typically read a file in C:

f = fopen("filename","r");
if (!f) ..error..

while(fgets(bf,BFSZ,f)) {
   ..do stuff..
}

if (ferror(f)) ..error..

fclose(f);


getting to the end of the file in the above is not an exception in my 
mind, it's something that will happen every time this code is executed.

I'd only want exceptions when:
- the file open failed
- the read failed (device error etc - not end of file)

Plus the last line read may/may not have the end of line char(s) this is 
not exceptional either IMO.


Arcane Jill's compiler code example showing using them as flow control can 
still be implemented without an exception on end of file in the stream 
code. It can be done by manually throwing an exception in the compiler 
code when the read hits eof. IMO this is actually better as it explicitly 
shows you're doing it.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:


 Exactly.
 
 Here is an example of how I typically read a file in C:
 
 f = fopen("filename","r");
 if (!f) ..error..
 
 while(fgets(bf,BFSZ,f)) {
   ..do stuff..
 }
 
 if (ferror(f)) ..error..
 
 fclose(f);
 
 
 getting to the end of the file in the above is not an exception in my 
 mind, it's something that will happen every time this code is executed.


Is there a way in theory in general streams (sockets, files, anything 
else) to test whether we've read the last byte, without blocking? If so, 
I think something along the lines of
while(!f.eof) {
	bf=f.gets(BFSZ);
	...
}
is likely clearer.


 I'd only want exceptions when:
 - the file open failed
 - the read failed (device error etc - not end of file)


What if scanf was looking for a floating point number, and it got to 
6.03E+ and then hit EOF? (Maybe that's legal, it's not the point ;-)
I'm keen on the idea that in a well-formed set of data, no read call 
fails, it means (when reading data of a known format) you don't have to 
put tests for error codes in your main code, and exceptions only handle 
errors of one sort or another.


 Plus the last line read may/may not have the end of line char(s) this is 
 not exceptional either IMO.


Sure, readLine should accept EOF as an EOL without throwing.

Sam
Jun 13 2004
next sibling parent reply Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 15:37:33 +1200, Sam McCall <tunah.d tunah.net> wrote:

 Regan Heath wrote:


 Exactly.

 Here is an example of how I typically read a file in C:

 f = fopen("filename","r");
 if (!f) ..error..

 while(fgets(bf,BFSZ,f)) {
   ..do stuff..
 }

 if (ferror(f)) ..error..

 fclose(f);


 getting to the end of the file in the above is not an exception in my 
 mind, it's something that will happen every time this code is executed.





 Is there a way in theory in general streams (sockets, files, anything 
 else) to test whether we've read the last byte, without blocking?


In the case of stream sockets I reckon you're only 'at the end' if the 
socket has been closed, otherwise, there might be data to read now, or 
not, or some might come later. Same with COM ports, and probably USB ports 
also.

Files are a little different as the data is all available as soon as you 
open it. So .eof for a file simply compares the read ptr position with the 
position of the end of the file.

You can make stream sockets non-blocking in which case a read will return 
immediately. Same with COM ports and I assume USB ports also.

Sockets are weird in that when they are closed by the other end you get 
told by 'select' that you can 'read'/'write' to/from the socket, and the 
actual read/write reports that it is closed. AFAIK there is no other way 
to tell if it's been closed.


 If so, I think something along the lines of
 while(!f.eof) {
 	bf=f.gets(BFSZ);
 	...
 }
 is likely clearer.


I was only giving my C style code, not suggesting a format for the D code, 
the D could look like this...

{
	FileStream i;

	try {
		i = new FileStream(new File("a.txt",READ));
		while(!i.eof()) {
			line = i.readLine();
			..do stuff..
		}
	}
	catch (Exception e) {
             ..you can catch all exceptions here and decide what to do with 
each..
	}
	finally {
		if (i) delete i;
	}
}

or something like that.


 I'd only want exceptions when:
 - the file open failed
 - the read failed (device error etc - not end of file)





 What if scanf was looking for a floating point number, and it got to 
 6.03E+ and then hit EOF? (Maybe that's legal, it's not the point ;-)


This is different, and should throw an exception :)


 I'm keen on the idea that in a well-formed set of data, no read call 
 fails, it means (when reading data of a known format) you don't have to 
 put tests for error codes in your main code, and exceptions only handle 
 errors of one sort or another.


Agreed.


 Plus the last line read may/may not have the end of line char(s) this 
 is not exceptional either IMO.


 Sure, readLine should accept EOF as an EOL without throwing.


I only mentioned this as I imagine it would be a common mistake to make.

Regan.

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:

 In the case of stream sockets I reckon you're only 'at the end' if the 
 socket has been closed, otherwise, there might be data to read now, or 
 not, or some might come later. Same with COM ports, and probably USB 
 ports also.
 ...
 You can make stream sockets non-blocking in which case a read will 
 return immediately. Same with COM ports and I assume USB ports also.
 
 Sockets are weird in that when they are closed by the other end you get 
 told by 'select' that you can 'read'/'write' to/from the socket, and the 
 actual read/write reports that it is closed. AFAIK there is no other way 
 to tell if it's been closed.


Ouch. Presumably there are protocols where the socket gets closed with 
no advance warning in the normal course of events (HTTP?).
The tricky case seems to be:
data\n<EOF>
If you're reading data one line at a time from an unbuffered socket, and 
it ends like that, then you'll terminate by reading an EOF when you 
expected a line, and you can't predict it.
It's frustrating, because it seems in every other case you can get away 
with using exceptions without having to handle them in non-error situations.

 If so, I think something along the lines of
 while(!f.eof) {
     bf=f.gets(BFSZ);
     ...
 }
 is likely clearer.


 
 
 I was only giving my C style code, not suggesting a format for the D 
 code, the D could look like this...


Right, the point being that now hitting EOF during a read (well, at the 
beginning of a read, allowing no EOL at EOF) _is_ an exceptional 
circumstance.

Sam
Jun 13 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 16:31:31 +1200, Sam McCall <tunah.d tunah.net> wrote:

 Regan Heath wrote:

 In the case of stream sockets I reckon you're only 'at the end' if the 
 socket has been closed, otherwise, there might be data to read now, or 
 not, or some might come later. Same with COM ports, and probably USB 
 ports also.
 ...
 You can make stream sockets non-blocking in which case a read will 
 return immediately. Same with COM ports and I assume USB ports also.

 Sockets are weird in that when they are closed by the other end you get 
 told by 'select' that you can 'read'/'write' to/from the socket, and 
 the actual read/write reports that it is closed. AFAIK there is no 
 other way to tell if it's been closed.





 Ouch. Presumably there are protocols where the socket gets closed with 
 no advance warning in the normal course of events (HTTP?).


HTTP sends a Content-Length most/all? of the time.


 The tricky case seems to be:
 data\n<EOF>
 If you're reading data one line at a time from an unbuffered socket, and 
 it ends like that, then you'll terminate by reading an EOF when you 
 expected a line, and you can't predict it.


So unless the readLine function has a failure return value you have to 
throw an exception. The same would have to be true for readInt readFloat 
etc.

If it had a failure return value then..

while(!i.eof()) {
	line = i.readLine();
	if (line.length == 0) continue;
}

would work, as the readLine that read the EOF would flag eof and i.eof 
would then return true.


 It's frustrating, because it seems in every other case you can get away 
 with using exceptions without having to handle them in non-error 
 situations.


Yeah.. I have felt this frustration before as I briefly tried to write my 
own stream lib, just for fun.


 If so, I think something along the lines of
 while(!f.eof) {
     bf=f.gets(BFSZ);
     ...
 }
 is likely clearer.



 I was only giving my C style code, not suggesting a format for the D 
 code, the D could look like this...





 Right, the point being that now hitting EOF during a read (well, at the 
 beginning of a read, allowing no EOL at EOF) _is_ an exceptional 
 circumstance.


Yep.

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:

 HTTP sends a Content-Length most/all? of the time.


Not all the time :-(


 So unless the readLine function has a failure return value you have to 
 throw an exception. The same would have to be true for readInt readFloat 
 etc.
 
 If it had a failure return value then..
 
 while(!i.eof()) {
     line = i.readLine();
     if (line.length == 0) continue;
 }


But then how do you distinguish an empty line? My preferred behaviour 
would be something like
  data \n \n <EOF>
^
readLine() returns "data"

  data \n \n <EOF>
         ^
readLine() returns ""

  data \n \n <EOF>
            ^
readLine() returns ""

  data \n \n <EOF>
               ^
readLine() returns null (EOF already been hit)

Sam
Jun 13 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 17:27:52 +1200, Sam McCall <tunah.d tunah.net> wrote:

 Regan Heath wrote:

 HTTP sends a Content-Length most/all? of the time.


 Not all the time :-(


 So unless the readLine function has a failure return value you have to 
 throw an exception. The same would have to be true for readInt 
 readFloat etc.

 If it had a failure return value then..

 while(!i.eof()) {
     line = i.readLine();
     if (line.length == 0) continue;
 }


 But then how do you distinguish an empty line? My preferred behaviour 
 would be something like
   data \n \n <EOF>
 ^
 readLine() returns "data"


agreed.


   data \n \n <EOF>
          ^
 readLine() returns ""


shouldn't that be " "


   data \n \n <EOF>
             ^
 readLine() returns ""


shouldn't that be " "


   data \n \n <EOF>
                ^
 readLine() returns null (EOF already been hit)


you're right here.

Regan.

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:

 shouldn't that be " "


Sorry, those spaces were just meant to be for decoration, so it was 
clear what had and hadn't been read yet.
Sam
Jun 13 2004
parent Regan Heath <regan netwin.co.nz> writes:
On Mon, 14 Jun 2004 17:35:35 +1200, Sam McCall <tunah.d tunah.net> wrote:

 Regan Heath wrote:

 shouldn't that be " "


 Sorry, those spaces were just meant to be for decoration, so it was 
 clear what had and hadn't been read yet.
 Sam


I thought they *might* have been, just checking. :)

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jun 13 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <caj6cj$254g$1 digitaldaemon.com>, Sam McCall says...

Is there a way in theory in general streams (sockets, files, anything 
else) to test whether we've read the last byte, without blocking? If so, 
I think something along the lines of
while(!f.eof) {
	bf=f.gets(BFSZ);
	...
}
is likely clearer.


In general, yes.  I would consider a socket at EOF when the socket is closed.
The ANSI C file API has an feof() function, and other file APIs typically have a
way to determine this as well.


 I'd only want exceptions when:
 - the file open failed
 - the read failed (device error etc - not end of file)


What if scanf was looking for a floating point number, and it got to 
6.03E+ and then hit EOF?


This is what convinced me that exceptions made sense for stream operations.
Encountering an unexpected EOF *is* an exceptional situation IMO.  Other
exceptional situations include unformatted operations that fail to read the
desired block size (typically as a result of EOF), and formatted read operations
that result in unparsable or incorrect data.  Note that this includes instances
where the user tries to read an integer from the keyboard and gets a letter
instead.

One thing I'm considering is adding overridable methods that handle these error
conditions so derived classes can throw more meaningful exceptions.  If it's
Windows and a meaningful message can be pulled from the error code then the
class should be able to include that information in the exception.  Since
different APIs have different ways of returning error information, it isn't
really possible to put all of this in the base class.


Sean
Jun 14 2004
prev sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...


But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.


Not exactly. The difference between bool and bit is whether or not compile-time
error checking is possible for a certain class of error. The difference between
exceptions and return-value-checking is RUN-TIME error checking (versus
crashing, which is what would happen if you forgot to check the return code).



Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.


Let me quote you something from "The C++ Programming Language 3rd Edition" by
Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a
program is run be considered exceptional? Can an event that is planned for and
handled be considered an error? The answer to both questions is yes.
'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is better
to think of an exception as meaning 'some part of the system couldn't do what it
was asked to do'".

Further on in the book, there is a whole section entitled Exceptions that are
not Errors, in which he says: "If an exception is expected and caught so that it
has no bad effects on the behavior of the program then how can it be an error?
Only because the programmer thinks of it as an error and of the
exception-handling mechanisms as tools for handling errors. Alternatively, one
might think of the exception-handling mechanisms as simply another control
structure".

I have written code in which an exception signals success! And this was not
because I'm completely mad (I refrain from comment on that point), but because
it was a brilliantly elegant thing to do. So it is here. We've reached the end
of the file - yippee - now let's roll up the stack and move onto the next bit of
code.

In D, we should be using exceptions even more than in C++, because the exception
handling mechanism is much, much simpler and there is way less overhead.

Walter has written some good stuff in the D manual about why exceptions are
better than return codes in handling errors. He should probably have mentioned
that this is also true for non-errors.



I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?


I think you'll find it's the other way round. On the bit/bool question,
non-arithmetic bool would have been the safer option, but we have not been
offered them. On the exception versus check-the-return code question, you can
code it either ways, but one way is MUCH safer than the other.

If I wrote an API which /required/ that callers check the return value from some
funtion, how long do you think it would be before someone forgot? Exceptions are
the path of safety.

I think people are just afraid of using them, that's all. Maybe because of
unfamilairity, or they have come to associate the exception-mechanism with
error-handling. To me, refusing to employ the exception-mechanism because it
isn't an error is like refusing to use for-loops because it isn't an error.

I would hate to have to go back to the tedious and error-prone methods of
return-value-checking. New tools give us new and powerful techniques. We should
use them, not be afraid of them.

Arcane Jill
Jun 11 2004
next sibling parent reply Derek <derek psyc.ward> writes:
On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:


 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...
 

But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.


 
 Not exactly. The difference between bool and bit is whether or not compile-time
 error checking is possible for a certain class of error. The difference between
 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return code).
 
 

Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.


 
 Let me quote you something from "The C++ Programming Language 3rd Edition" by
 Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a
 program is run be considered exceptional? Can an event that is planned for and
 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is
better
 to think of an exception as meaning 'some part of the system couldn't do what
it
 was asked to do'".
 
 Further on in the book, there is a whole section entitled Exceptions that are
 not Errors, in which he says: "If an exception is expected and caught so that
it
 has no bad effects on the behavior of the program then how can it be an error?
 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively, one
 might think of the exception-handling mechanisms as simply another control
 structure".
 
 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but because
 it was a brilliantly elegant thing to do. So it is here. We've reached the end
 of the file - yippee - now let's roll up the stack and move onto the next bit
of
 code.
 
 In D, we should be using exceptions even more than in C++, because the
exception
 handling mechanism is much, much simpler and there is way less overhead.
 
 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have mentioned
 that this is also true for non-errors.
 
 

I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?


 
 I think you'll find it's the other way round. On the bit/bool question,
 non-arithmetic bool would have been the safer option, but we have not been
 offered them. On the exception versus check-the-return code question, you can
 code it either ways, but one way is MUCH safer than the other.
 
 If I wrote an API which /required/ that callers check the return value from
some
 funtion, how long do you think it would be before someone forgot? Exceptions
are
 the path of safety.
 
 I think people are just afraid of using them, that's all. Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because it
 isn't an error is like refusing to use for-loops because it isn't an error.
 
 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking. New tools give us new and powerful techniques. We should
 use them, not be afraid of them.
 
 Arcane Jill


Got it. You are using Exceptions as a fancy GOTO.

-- 
Derek
Melbourne, Australia
Jun 11 2004
parent reply "The Dr ... who?" <thedr who.com> writes:
"Derek" <derek psyc.ward> wrote in message
news:c5k4ima3f0tp.1nm5clznwf9x6.dlg 40tude.net...

 On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:


 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...


But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.


 Not exactly. The difference between bool and bit is whether or not




compile-time

 error checking is possible for a certain class of error. The difference




between

 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return code).



Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.


 Let me quote you something from "The C++ Programming Language 3rd Edition" by
 Bjarne Stroustrup (the creator of C++): "Can an event that happens most times




a

 program is run be considered exceptional? Can an event that is planned for




and

 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is




better

 to think of an exception as meaning 'some part of the system couldn't do what




it

 was asked to do'".

 Further on in the book, there is a whole section entitled Exceptions that are
 not Errors, in which he says: "If an exception is expected and caught so that




it

 has no bad effects on the behavior of the program then how can it be an




error?

 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively,




one

 might think of the exception-handling mechanisms as simply another control
 structure".

 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but




because

 it was a brilliantly elegant thing to do. So it is here. We've reached the




end

 of the file - yippee - now let's roll up the stack and move onto the next bit




of

 code.

 In D, we should be using exceptions even more than in C++, because the




exception

 handling mechanism is much, much simpler and there is way less overhead.

 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have




mentioned

 that this is also true for non-errors.



I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?


 I think you'll find it's the other way round. On the bit/bool question,
 non-arithmetic bool would have been the safer option, but we have not been
 offered them. On the exception versus check-the-return code question, you can
 code it either ways, but one way is MUCH safer than the other.

 If I wrote an API which /required/ that callers check the return value from




some

 funtion, how long do you think it would be before someone forgot? Exceptions




are

 the path of safety.

 I think people are just afraid of using them, that's all. Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because it
 isn't an error is like refusing to use for-loops because it isn't an error.

 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking. New tools give us new and powerful techniques. We




should

 use them, not be afraid of them.

 Arcane Jill


 Got it. You are using Exceptions as a fancy GOTO.


Well said. This is something that is little recognised by proponents of
exceptions-for-all. They are a goto, since they change the flow of control.
Sure,
they're vastly better than goto, since resource handling is automatic. But, it's
still possible for flow of control to jump to an arbitrary location. The only
difference is that the destination is limited to somewhere in the call stack,
whereas with goto it's anywhere the label is located.

But, before anyone thinks I'm anti-exceptions, I'm not. They're the absolute
right answer for a significant minority of error-handling scenarios, and the
winner of an even bigger minority (if not the majority) of other scenarioe.
They're just not the right answer in all cases.
Jun 12 2004
parent "The Dr ... who?" <thedr who.com> writes:
"The Dr ... who?" <thedr who.com> wrote in message
news:cafvrc$kts$1 digitaldaemon.com...

 "Derek" <derek psyc.ward> wrote in message
 news:c5k4ima3f0tp.1nm5clznwf9x6.dlg 40tude.net...

 On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:


 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...


But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.


 Not exactly. The difference between bool and bit is whether or not




 compile-time

 error checking is possible for a certain class of error. The difference




 between

 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return






code).

Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.


 Let me quote you something from "The C++ Programming Language 3rd Edition"






by

 Bjarne Stroustrup (the creator of C++): "Can an event that happens most






times

 a

 program is run be considered exceptional? Can an event that is planned for




 and

 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is




 better

 to think of an exception as meaning 'some part of the system couldn't do






what

 it

 was asked to do'".

 Further on in the book, there is a whole section entitled Exceptions that






are

 not Errors, in which he says: "If an exception is expected and caught so






that

 it

 has no bad effects on the behavior of the program then how can it be an




 error?

 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively,




 one

 might think of the exception-handling mechanisms as simply another control
 structure".

 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but




 because

 it was a brilliantly elegant thing to do. So it is here. We've reached the




 end

 of the file - yippee - now let's roll up the stack and move onto the next






bit

 of

 code.

 In D, we should be using exceptions even more than in C++, because the




 exception

 handling mechanism is much, much simpler and there is way less overhead.

 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have




 mentioned

 that this is also true for non-errors.



I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?


 I think you'll find it's the other way round. On the bit/bool question,
 non-arithmetic bool would have been the safer option, but we have not been
 offered them. On the exception versus check-the-return code question, you






can

 code it either ways, but one way is MUCH safer than the other.

 If I wrote an API which /required/ that callers check the return value from




 some

 funtion, how long do you think it would be before someone forgot?






Exceptions

 are

 the path of safety.

 I think people are just afraid of using them, that's all. Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because






it

 isn't an error is like refusing to use for-loops because it isn't an error.

 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking. New tools give us new and powerful techniques. We




 should

 use them, not be afraid of them.

 Arcane Jill


 Got it. You are using Exceptions as a fancy GOTO.


 Well said. This is something that is little recognised by proponents of
 exceptions-for-all. They are a goto, since they change the flow of control.


Sure,

 they're vastly better than goto, since resource handling is automatic.


_and_ because the destination is arbitrated at runtime, and (largely) under
control of client code. It's a kind of publish-subscribe mechanism.

But it's still a goto!
Jun 12 2004
prev sibling next sibling parent reply Derek <derek psyc.ward> writes:
On Fri, 11 Jun 2004 23:32:32 +0000 (UTC), Arcane Jill wrote:


 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...
 

But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.


 
 Not exactly. The difference between bool and bit is whether or not compile-time
 error checking is possible for a certain class of error. The difference between
 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return code).
 
 

Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.


 
 Let me quote you something from "The C++ Programming Language 3rd Edition" by
 Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a
 program is run be considered exceptional? Can an event that is planned for and
 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is
better
 to think of an exception as meaning 'some part of the system couldn't do what
it
 was asked to do'".
 
 Further on in the book, there is a whole section entitled Exceptions that are
 not Errors, in which he says: "If an exception is expected and caught so that
it
 has no bad effects on the behavior of the program then how can it be an error?
 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively, one
 might think of the exception-handling mechanisms as simply another control
 structure".
 
 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but because
 it was a brilliantly elegant thing to do. So it is here. We've reached the end
 of the file - yippee - now let's roll up the stack and move onto the next bit
of
 code.
 
 In D, we should be using exceptions even more than in C++, because the
exception
 handling mechanism is much, much simpler and there is way less overhead.
 
 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have mentioned
 that this is also true for non-errors.
 
 

I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?


 
 I think you'll find it's the other way round. On the bit/bool question,
 non-arithmetic bool would have been the safer option, but we have not been
 offered them. On the exception versus check-the-return code question, you can
 code it either ways, but one way is MUCH safer than the other.
 
 If I wrote an API which /required/ that callers check the return value from
some
 funtion, how long do you think it would be before someone forgot? Exceptions
are
 the path of safety.
 
 I think people are just afraid of using them, that's all. Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because it
 isn't an error is like refusing to use for-loops because it isn't an error.
 
 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking. New tools give us new and powerful techniques. We should
 use them, not be afraid of them.
 
 Arcane Jill


Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.

-- 
Derek
Melbourne, Australia
Jun 11 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <iqr9lefhimza$.1wolikdlbu2jm$.dlg 40tude.net>, Derek says...


Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.


Exceptions are the most appropriate mechanism to use if the following four
conditions are simultaneously met:

1) You wish to be advised of a particular circumstance
2) More often than not, the circumstance will not happen
3) When it does happen, information that it has happened needs to be propogated
back up the call stack
4) Failure to communicate information about the circumstance will result in bad
or undefined behavior

These conditions are completely met in the case of end-of-file. You wish to be
advised of it; more often than not, a read won't result in eof; you need to
propogate eof up the call stack; if you don't, undefined behaviour - possibly
bang. Ergo - there is mechanism that is just right - the exception mechanism.

An example might help. Imagine you're writing a compiler/parser for some new
programming langage you've just invented. Your main loop might look something
like this:


   void parseModule(Stream s)
   {
       for (;;)
       {
           parseNextDeclaration(s);
       }
   }



Within parseNextDeclaration() you're going to have other similiar loops, things
like parseNextStatement(), parseNextExpression(), parseNextIdentifier(), and so
on. And way, way, WAY down the call-stack, you're going to want to read raw
fixed size blocks of data.

WIthout the exception mechanism, you'd have to change every single one of those
functions from void to bool. You would have to check the return value of every
single one of them, every single time, and make sure to return false if some
lower-level function returned false.

Apart from being tedious and error-prone, this is also going to slow your
program down, because of all that overhead of checking return values over and
over again. You shouldn't need to do it, because eof is a rare circumstance
(rare in the sense that most reads will succeed - I don't mean to imply that
most files don't end, obviously).

By contrast, to execute a throw statement, the code which the D compiler
generates for you merely has to (1) copy a value into the stack pointer, (2)
switch, based on the type of the exception thrown, (3) execute a jump
instruction. Now tell me which has the greater overhead? (Okay, if there are
nested try blocks and/or auto class declarations, you may have to repeat that
procedure a few times, once for each exception frame, but generally there will
be WAY more stack frames than exception frames).

(The efficiency argument doesn't hold true in C++, by the way. In C++, simply
copying a new value into the stack pointer would be disastrous in C++ because
there may be objects on the stack whose destructors need to be called. D is more
sensible).

So you've got the best of all worlds - efficiency; safety; clean code.

My one complaint about D's exceptions is that there realy SHOULD be some
standard ones in Phobos which we could reuse - things like EndOfFile,
OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
They should be organized into a heirarchy which makes sense, so it becomes
obvious which one to use/derive from. Constructors for these should be simple.
You shouldn't need to supply a string to construct an EndOfFile, and so on.

Arcane Jill
Jun 12 2004
next sibling parent Sean Kelly <sean f4.ca> writes:
In article <caeaju$162q$1 digitaldaemon.com>, Arcane Jill says...

My one complaint about D's exceptions is that there realy SHOULD be some
standard ones in Phobos which we could reuse - things like EndOfFile,
OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
They should be organized into a heirarchy which makes sense, so it becomes
obvious which one to use/derive from. Constructors for these should be simple.
You shouldn't need to supply a string to construct an EndOfFile, and so on.


Exception and Error are a good base, and we probably need BadCast, OutOfMemory,
DivideByZero and some others if they don't already exist.  Beyond that I think
it's up to each module to define a set of errors that makes sense.  I think I
may change the Stream errors around a bit so they can include platform-specific
information in the cases where it's applicable.

Sean
Jun 12 2004
prev sibling next sibling parent Derek <derek psyc.ward> writes:
On Sat, 12 Jun 2004 07:16:14 +0000 (UTC), Arcane Jill wrote:


 In article <iqr9lefhimza$.1wolikdlbu2jm$.dlg 40tude.net>, Derek says...
 

Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.


 
 Exceptions are the most appropriate mechanism to use if the following four
 conditions are simultaneously met:
 
 1) You wish to be advised of a particular circumstance
 2) More often than not, the circumstance will not happen
 3) When it does happen, information that it has happened needs to be propogated
 back up the call stack
 4) Failure to communicate information about the circumstance will result in bad
 or undefined behavior




2) The circumstance has a low frequency of occurance, or is not expected to 
occur.

Because it is quite possible that sometimes, *not* reaching EOF is an 
error.
[snip]


 
 So you've got the best of all worlds - efficiency; safety; clean code.
 
 My one complaint about D's exceptions is that there realy SHOULD be some
 standard ones in Phobos which we could reuse - things like EndOfFile,
 OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
 They should be organized into a heirarchy which makes sense, so it becomes
 obvious which one to use/derive from. Constructors for these should be simple.
 You shouldn't need to supply a string to construct an EndOfFile, and so on.


Hmmm... I tend to agree.

-- 
Derek
Melbourne, Australia
Jun 12 2004
prev sibling next sibling parent "Bent Rasmussen" <exo bent-rasmussen.info> writes:
Is it an exception that the tenth element of a list does not exist when the
previous nine did? This is case for contracts over exceptions: To catch bugs
use contracts, to recover from exceptional state use exceptions.

- It is predictable that the file will end, it is possible to check when it
does.
- It is predictable that at some time a user will type in an email address
without the at-sign, it is possible to check when s/he does.
- It is predictable that a programmer will create bugs, its often possible
to check when s/he does.

But my experience with streams in general is very limited, so I must be
missing something very trivial because this use of exceptions practically
lights up all warning lights to me.
Jun 13 2004
prev sibling parent reply Antti =?iso-8859-1?Q?Syk=E4ri?= <jsykari gamma.hut.fi> writes:
In article <caeaju$162q$1 digitaldaemon.com>, Arcane Jill wrote:

 (The efficiency argument doesn't hold true in C++, by the way. In C++,
 simply copying a new value into the stack pointer would be disastrous
 in C++ because there may be objects on the stack whose destructors
 need to be called. D is more sensible).


I'd have thought that exception handling in D and C++ have the same
overhead, because both have RAII classes and try-catch statements.

(In typical well-written C++ program, though, unwinding the stack takes
a bit longer, since RAII is also used for memory management.)

Of course, that wouldn't matter, since typical exception handling time
is still negligible if exception handling is used properly.


 My one complaint about D's exceptions is that there realy SHOULD be some
 standard ones in Phobos which we could reuse - things like EndOfFile,
 OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
 They should be organized into a heirarchy which makes sense, so it becomes
 obvious which one to use/derive from. Constructors for these should be simple.
 You shouldn't need to supply a string to construct an EndOfFile, and so on.


Gets my vote. I think Matthew Wilson had something in the works a few
months ago?

(I somehow find it amusing to have a language with standard exception
called "Success". Certainly requires some guts to admit that success is
an exceptional condition ;) No committee-designed language would ever be
able to pull that off...)

-Antti

-- 
I will not be using Plan 9 in the creation of weapons of mass destruction
to be used by nations other than the US.
Jun 14 2004
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Antti Sykäri" <jsykari gamma.hut.fi> wrote in message
news:slrnccrqj5.3eo.jsykari pulu.hut.fi...

 In article <caeaju$162q$1 digitaldaemon.com>, Arcane Jill wrote:

 (The efficiency argument doesn't hold true in C++, by the way. In C++,
 simply copying a new value into the stack pointer would be disastrous
 in C++ because there may be objects on the stack whose destructors
 need to be called. D is more sensible).


 I'd have thought that exception handling in D and C++ have the same
 overhead, because both have RAII classes and try-catch statements.

 (In typical well-written C++ program, though, unwinding the stack takes
 a bit longer, since RAII is also used for memory management.)

 Of course, that wouldn't matter, since typical exception handling time
 is still negligible if exception handling is used properly.


 My one complaint about D's exceptions is that there realy SHOULD be some
 standard ones in Phobos which we could reuse - things like EndOfFile,
 OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
 They should be organized into a heirarchy which makes sense, so it becomes
 obvious which one to use/derive from. Constructors for these should be




simple.

 You shouldn't need to supply a string to construct an EndOfFile, and so on.


 Gets my vote. I think Matthew Wilson had something in the works a few
 months ago?


Yes, we did come to an accord on the naming conventions. I have to admit,
somewhat sheepishly, that I didn't write it down anywhere, trusting the newgroup
archive. If someone with more time (or good form) than me wants to search last
year's posts for "Exception", or maybe "Win32Exception", then I'm sure it'll be
there. Perhaps once located it could be placed up there with all JC's other good
stuff on the Wiki?

Lenny Lazybones
Jun 14 2004
parent J C Calvarese <jcc7 cox.net> writes:
Matthew wrote:

 "Antti Sykäri" <jsykari gamma.hut.fi> wrote in message
 news:slrnccrqj5.3eo.jsykari pulu.hut.fi...
 


...

My one complaint about D's exceptions is that there realy SHOULD be some
standard ones in Phobos which we could reuse - things like EndOfFile,
OutOfMemory, BadParameter, ReadError, Success, and so on, all in one module.
They should be organized into a heirarchy which makes sense, so it becomes
obvious which one to use/derive from. Constructors for these should be




 
 simple.
 

You shouldn't need to supply a string to construct an EndOfFile, and so on.


Gets my vote. I think Matthew Wilson had something in the works a few
months ago?


 
 
 Yes, we did come to an accord on the naming conventions. I have to admit,
 somewhat sheepishly, that I didn't write it down anywhere, trusting the
newgroup
 archive. If someone with more time (or good form) than me wants to search last
 year's posts for "Exception", or maybe "Win32Exception", then I'm sure it'll be
 there. Perhaps once located it could be placed up there with all JC's other
good
 stuff on the Wiki?
 
 Lenny Lazybones


FYI, I think I found the thread you were remembering. It was called 
"Exception naming conventions - Phobos, and third party libraries" (yay, 
for helpful thread subjects).

http://www.digitalmars.com/drn-bin/wwwnews?D/15869

-- 
Justin (a/k/a jcc7)
http://jcc_7.tripod.com/d/
Jun 21 2004
prev sibling parent Sean Kelly <sean f4.ca> writes:
In article <iqr9lefhimza$.1wolikdlbu2jm$.dlg 40tude.net>, Derek says...

Maybe I should think of Exceptions as Event Triggers instead of of Error
Conditions. This is helpful.


They're an alternate means of flow control.  Some argue that exceptions can make
a program difficult to read, but I think so long as everything is
well-documented it's not an issue.  They certainly beat wrapping every function
call with error checking routines, and exception performance has gotten quite
good even in C++ these past few years.

Sean
Jun 12 2004
prev sibling parent reply "The Dr ... who?" <thedr who.com> writes:
 In article <1l01dqjpufkr1$.e32a6q1rqkxy.dlg 40tude.net>, Derek says...


But isn't this sort of like the 'bit' verses 'bool' thing. They are
different animals but a 'bit' can be used to simulate a 'bool'.


 Not exactly. The difference between bool and bit is whether or not compile-time
 error checking is possible for a certain class of error. The difference between
 exceptions and return-value-checking is RUN-TIME error checking (versus
 crashing, which is what would happen if you forgot to check the return code).



Likewise,
reaching the end of file is not an exceptional event, in fact it is quite
normal in most cases involving stream files.


 Let me quote you something from "The C++ Programming Language 3rd Edition" by
 Bjarne Stroustrup (the creator of C++): "Can an event that happens most times a
 program is run be considered exceptional? Can an event that is planned for and
 handled be considered an error? The answer to both questions is yes.
 'Exceptional' does not mean 'almost never happens' or 'disastrous'. It is


better

 to think of an exception as meaning 'some part of the system couldn't do what


it

 was asked to do'".

 Further on in the book, there is a whole section entitled Exceptions that are
 not Errors, in which he says: "If an exception is expected and caught so that


it

 has no bad effects on the behavior of the program then how can it be an error?
 Only because the programmer thinks of it as an error and of the
 exception-handling mechanisms as tools for handling errors. Alternatively, one
 might think of the exception-handling mechanisms as simply another control
 structure".

 I have written code in which an exception signals success! And this was not
 because I'm completely mad (I refrain from comment on that point), but because
 it was a brilliantly elegant thing to do.


Maybe so. Can't see as how it's particularly apposite to this issue, other than
it's reflective of your opinion on the subject.


 So it is here. We've reached the end
 of the file - yippee - now let's roll up the stack and move onto the next bit


of

 code.

 In D, we should be using exceptions even more than in C++, because the


exception

 handling mechanism is much, much simpler and there is way less overhead.


No. D is not Java, which equally accords with your given criterion. D is a
multi-paradigm language, just like C++. That, above all the nifty little syntax
tricks and GC, is what will make it succeed.

Exceptions are the absolute right answer in many circumstances. Exceptions are
the absolute wrong answer in many other circumstances. And sometimes, there's
justified equivocation.

And sometimes - this time! - people are coming at the issue from the wrong
angle.

Reading objects, or even fundamentals whose .size >1, requires exception
handling.
Manipulating byte (or, I guess, ANSI char) streams does _not_ require exception
handling, and in many instances is better off without it.

So rather than trying to do something hideous like have the user supply a
bExceptionsPlease parameter, why not simply do the obvious thing, and build an
object-aware exception-throwing arbitrary type manipulating layer on top of a
super simple byte-stream layer? It's not exactly like it's a new idea, and the
benefits are legion. For a start, we can plug in different byte streams, sharing
a common interface, when appropriate.


 Walter has written some good stuff in the D manual about why exceptions are
 better than return codes in handling errors. He should probably have mentioned
 that this is also true for non-errors.


It's not black, and it's not white. Such statements as this are positively
unhelpful to the community, and especially to neophyte/junior programmers. The
world is rich, and there are few richer parts of SE than error-handling.


I know this just semantics, but ... must we be forced into using
exceptions to handle eof, just like we are forced to use 'bit' to handle
'bool'?


 I think you'll find it's the other way round. On the bit/bool question,
 non-arithmetic bool would have been the safer option, but we have not been
 offered them. On the exception versus check-the-return code question, you can
 code it either ways, but one way is MUCH safer than the other.

 If I wrote an API which /required/ that callers check the return value from


some

 funtion, how long do you think it would be before someone forgot? Exceptions


are

 the path of safety.

 I think people are just afraid of using them, that's all.


That may be for some. For others it's the opposite: exceptions are touted as
appropriate to all error-handling / return-indicating scenarios.

But there are many experienced and erudite players that do not see exceptions as
a panacea. They are the answer in most cases, but they are _not_ the answer in
all.


 Maybe because of
 unfamilairity, or they have come to associate the exception-mechanism with
 error-handling. To me, refusing to employ the exception-mechanism because it
 isn't an error is like refusing to use for-loops because it isn't an error.


That's true.


 I would hate to have to go back to the tedious and error-prone methods of
 return-value-checking.


Horses for courses.


 New tools give us new and powerful techniques. We should
 use them, not be afraid of them.


True. But we should also be confident to point out when the emporer's undies are
visible.


Anyway, back to the topic: let's have a two/multi-layer streaming architecture,
and this whole issue is rendered moot. The added advantage is that we'll
actually
have a more elegant and extensible architecture to boot!
Jun 12 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
The Dr ... who? wrote:

 No. D is not Java...


And yet the way Java does does provide both mechanisms :-)

How about this?

abstract class Stream {
	// reads a byte and stores it in dest, returns false
	// 	(and stores -1?) on eof
	abstract bool read(out ubyte dest);

	// reads up to len bytes, returns number read, or -1 on eof
	int read(ubyte[] dest,int off,int len);
	int read(ubyte[] dest) {
		return read(dest,0,dest.length);
	}

	// Reads a formatted value and throws on eof
	// Anyone got a good short name for this?
	void readData(out ubyte b);
	void readData(out int i);
	...

	// reads exactly len bytes, throws on eof
	void readFully(byte[] dest,int off,int len);
	void readFully(byte[] dest) {
		readFully(dest,0,dest.length);
	}
}

All methods throw an IOException of some sort when a "real" error 
occurs, EOFException (or just EOF) could be a subclass of this or not.
Java puts the formatted value reading in a subclass DataInputStream that 
wraps an instance of InputStream, I'm not sure about the rationale 
behind this.
Sam
Jun 13 2004
parent Sam McCall <tunah.d tunah.net> writes:
Sam McCall wrote:

     // Reads a formatted value and throws on eof
     // Anyone got a good short name for this?
     void readData(out ubyte b);
     void readData(out int i);
     ...


Of course by "formatted" I meant data where we potentially assume some 
binary format rather than just treating it as a stream of bytes. The 
point being that you expect there to be data in this format, so EOF *is* 
exceptional.
Sam
Jun 13 2004
prev sibling parent "The Dr ... who?" <thedr who.com> writes:
"Arcane Jill" <Arcane_member pathlink.com> wrote in message
news:cad277$2bun$1 digitaldaemon.com...

 In article <cacidg$1jn3$1 digitaldaemon.com>, Sean Kelly says...


What bothered me about the exception throwing in Stream was that since all
methods are implemented in terms of readExact, an exception will always be




throw

when eof() is reached.  In my opinion, end of file is an expected rather than




an

exceptional condition.  If I had to wrap all my stream operations in try




blocks

I'd go nuts :)


 I strongly disagree. Exceptions aren't just for errors, you know. There are all
 SORTS of non-error uses for exceptions. Just because something is expected,
 doesn't mean that an exception is not the best mechanism.

 Maybe I'm just more comfortable working with exceptions than some.


LOL! Glad to see someone else taking on the mantle of humility.


 To my mind,
 best practice behavior is to throw an exception on end-of-file, and a
 /different/ exception on error.


What about all the other minds, and their ways of conceptualising errors, return
values + stream manipulation? Wrong, irrelevant, jejune?


 I like this way of working. It beats the hell out of having to test the return
 value all the time.


So it does, most of the time. But "all"? Big statement that.

Humbly mine


-- 
The Dr.

da-da-da-dum, da-da-da-dum, da-da-da-dum, daaah
da-da-da-dum, da-da-da-dum, da-da-da-dum, daaah
woo-ooooo
Jun 12 2004
prev sibling parent "Walter" <newshound digitalmars.com> writes:
"Sean Kelly" <sean f4.ca> wrote in message
news:caar8o$28ba$1 digitaldaemon.com...

 One question about character output.  The docs say dchar is a 32-bit


unicode

 character but I'm pretty sure that it's currently implemented as wchar_t


in

 Windows right now (which would make it compatible with wchar).  Is there


any

 reason there shouldn't be methods to operate on all three char types:


char,

 wchar, and dchar?


dchar is 32 bits
wchar is 16 bits
char is 8 bits

The stream methods should all operate on chars as UTF-8 streams. The
conversion to/from Windows 'W' functions should come as the last step.
Jun 14 2004
prev sibling next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ca82sq$u9n$1 digitaldaemon.com>, Sean Kelly says...


As things stand, the basic D stream framework has an InputStream and
OutputStream interface and then a single Stream class that implements both of
these.  File, MemoryStream, SliceStream, and BufferedStream all inherit from
Stream, while BufferedFile inherits from BufferedStream.  Stream and File do not
buffer by default, BufferedStream acts as an adaptor class, and BufferedFile is
the buffered version of the File stream class.  Currently, all formatted i/o
goes through the printf method, with the read/write methods offering unformatted
i/o.  There is also an OutBuffer class (though no corresponding InBuffer) which
seems functionally similar to MemoryStream.


As things stand, the Phobos documentation mentions almost none of this. I didn't
even know that file streams were unbuffered, or that the class BufferedStream
even existed, until a few days ago when someone told me.


Anyway, I'd want a function available() to be available in input streams. (Same
as Java's java.io.InputStream). The implementation of this is /childs play/. It
goes like this:

In InputStream and Stream

    abstract uint available();


In unbuffered streams:

   override uint available() { return 0; }


In buffered streams:

   override uint available()
   {
       return numBytesRemainingInBuffer + s.available();
   }


where s is the underlying stream. (This is likely to be itself unbuffered, and
so it's available() function is likely to return zero - but there will always be
special cases, so it's best to assume not).

As one special case, for example a string input stream could have available()
return the remaining length of the string.

In any case, the most important thing about available() is that it MUST NOT
BLOCK. It doesn't matter if it underestimates, but it must never overestimate.


Arcane Jill
Jun 09 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <ca90hu$2bgd$1 digitaldaemon.com>, Arcane Jill says...

As things stand, the Phobos documentation mentions almost none of this. I didn't
even know that file streams were unbuffered, or that the class BufferedStream
even existed, until a few days ago when someone told me.


At this point, I think it's still better to just look at the Phobos source code.
There's a lot of stuff in there that's largely undocumented.


Anyway, I'd want a function available() to be available in input streams. (Same
as Java's java.io.InputStream). The implementation of this is /childs play/. It
goes like this:

In InputStream and Stream

    abstract uint available();


In unbuffered streams:

   override uint available() { return 0; }


In buffered streams:

   override uint available()
   {
       return numBytesRemainingInBuffer + s.available();
   }


where s is the underlying stream. (This is likely to be itself unbuffered, and
so it's available() function is likely to return zero - but there will always be
special cases, so it's best to assume not).

As one special case, for example a string input stream could have available()
return the remaining length of the string.

In any case, the most important thing about available() is that it MUST NOT
BLOCK. It doesn't matter if it underestimates, but it must never overestimate.


I was planning on adding available() to BufferedStream only, as it doesn't seem
to make sense for an unbuffered stream class.  I guess the alternative would be
to document rules that unbuffered classes could follow, like return 0 if they
don't know and -1 if at the end of file.

Sean
Jun 10 2004
parent Sam McCall <tunah.d tunah.net> writes:
Sean Kelly wrote:


 In article <ca90hu$2bgd$1 digitaldaemon.com>, Arcane Jill says...
 

As things stand, the Phobos documentation mentions almost none of this. I didn't
even know that file streams were unbuffered, or that the class BufferedStream
even existed, until a few days ago when someone told me.


 
 
 At this point, I think it's still better to just look at the Phobos source
code.
 There's a lot of stuff in there that's largely undocumented.


How well does Doxygen work with D? I've never used it, but love javadoc 
for the ability to get a clean, readable reference, even if there's no 
actual description text for things (I've got the source).


 I was planning on adding available() to BufferedStream only, as it doesn't seem
 to make sense for an unbuffered stream class.  I guess the alternative would be
 to document rules that unbuffered classes could follow, like return 0 if they
 don't know and -1 if at the end of file.


If you make the contract for available() to return the number of bytes 
known to be available, which may be an underestimate, or -1 if we're 
known to be at end of file (which isn't guaranteed even if we are?), 
then returning zero is _always_ safe, and returning nonzero values is a 
performance thing that shouldn't be relied on in general.
(Of course if some class like BufferedStream is documented to always 
return positive when not at EOF, and -1 when at EOF, then you can rely 
on this when you know you've got a BufferedStream)
Sam
Jun 13 2004
prev sibling next sibling parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Whoops! Posted this to the another thread, but I really meant to post here
instead ...

Perhaps you might consider checking out mango.io over at
http://www.dsource.org/forums/viewtopic.php?t=148 ? Mango.io attempts to
provide seamless integration of the following:

-Buffered I/O for Files, Sockets etc
-A variety of Readers and Writers for formatted IO, endian conversion, etc
-Multiple reader/writers upon the same buffer
-Tokens and Tokenizers for loosely formatted input (text lines, words,
numbers, etc)
-RegExp token wrapper
-Chained operations
-EOS checking is almost completely redundant (you rarely, if ever, need to
check)
-Both put/get and <</>> syntax
-Bidi equivalent of std.outbuffer (can use Readers/Writers per usual)
-Memory-mapped IO seamlessly overloads buffered IO
-Class serialization (to file, network, etc)
-CompositeIO framework for bracketing IO (such as commit semantics)
-Simply mechanism to bind your own classes into Reader/Writer framework
-Exceptions thrown for exceptional conditions (per discussion)
-Printf wrapper
-Mango.io has very little overhead, so it's *fast*. You can optionally
enable array-slicing when using CompositeReader. Tokens are always sliced,
but you can .dup them
-Includes typical file management tools (FileProxy, FileConduit), file path
manipulation (FilePath), etc
-Includes an RFC 2396 compliant URI specification
-Includes a simple Properties file reader
-Usual stdio singletons
-some <gasp> documentation!

There's a bunch more stuff in there also. Mango.io has been pretty well
beaten-up by three great guys since March, so it's quite stable and robust
now. Sure, there's things in there that could use improvement, but it's a
reasonable start. The design is somewhat reminiscent of Java NIO, but
without all the hideous IO warts that Java IO was originally saddled with. I
would contend that mango.io is dramatically cleaner than the Java design;
but that wouldn't be too hard, now would it? <g>

Mango.io is part of the Mango Tree, which currently includes these other
modules:

-exceptionally fast HTTP server (thanks mainly to D array slicing)
-Java like servlet engine (mango.io was built with this in mind)
-Log4J clone, including an HTML monitor/console for inspecting and setting
log/debug status at runtime (plus Chainsaw integration
-An HTTP client
-some simple caching mechanisms, including virtual caching to disk.

I invite you to take a look, and comment. It'd be great if we could turn
mango.io into a truly excellent IO platform for D ...

- Kris


"Sean Kelly" <sean f4.ca> wrote in message
news:ca82sq$u9n$1 digitaldaemon.com...

 I think the D streams provide a good base but are missing some of the more
 complex features that I miss from C++, mostly to do with formatted i/o.


This

 raises the rather large issue of localization which I'm going to ignore


for the

 moment.

 As things stand, the basic D stream framework has an InputStream and
 OutputStream interface and then a single Stream class that implements both


of

 these.  File, MemoryStream, SliceStream, and BufferedStream all inherit


from

 Stream, while BufferedFile inherits from BufferedStream.  Stream and File


do not

 buffer by default, BufferedStream acts as an adaptor class, and


BufferedFile is

 the buffered version of the File stream class.  Currently, all formatted


i/o

 goes through the printf method, with the read/write methods offering


unformatted

 i/o.  There is also an OutBuffer class (though no corresponding InBuffer)


which

 seems functionally similar to MemoryStream.

 I'd like separate InputStream, OutputStream, and Stream base classes to


allow

 support for streams where only input or output makes sense.  To make it


clear

 these are base classes and prevent naming problems with the interfaces I


propose

 these names: InputStreamBase, OutputStreamBase, and StreamBase (or
 InputOutputStreamBase, though that's pretty darn verbose).  Because of the
 single inheritance rule, StreamBase won't be able to inherit from
 InputStreamBase and OutputStreamBase which will liekly result in some


toying

 with the design to avoid code duplication.  These classes will be


obviously be

 abstract base classes.  If I wanted to nit-pick I might suggest renaming


File to

 FileStream for consistency.

 In addition to the existing functionality I propose new formatted i/o


methods

 for all types that already have unformatted methods.  Apart from a few


instances

 these can forward most of the work to the existing printf method.  The


default

 methods would be "get" and "put" with the possibility of adding overloaded
 operator support in the future.  I'd also like to see something similar to


the

 callback mechanism in C++ streams so arbitrary classes can hook in


formatting

 information.  And Jill had mentioned wanting something similar to


readsome()

 from C++ streams so I suspect this list will grow a bit.

 As for open issues, someone with more experience than myself should


probably

 address localization.  At the very least, there should be some support for
 proper numeric formatting, but that raises the issue of how formatting


rules

 will be supplied to the stream classes and what else the localization


rules

 should cover.

 I'm willing to go and do this myself, but as it's a modification of


existing

 Phobos stuff I wanted to get some feedback.  Does anyone have any


objections or

 suggestions?  Assuming the design doesn't suck will it have a chance for
 inclusion?


 Sean
Jun 15 2004
prev sibling parent reply surferjeff excite.com writes:
The interface for InputStream and OutputStream is huge.  I often have to create
my own streambufs in C++, which only requires implementing about 5 methods.  If
I want to write my own class that implements InputStream, then I have to
implement about 50 methods, and another 50 if I want to implement OutputStream
too!

It would useful if there were smaller interfaces.  For example, a read() method
and eof() for input, and that's it.  Then have a util class that takes one of
these small interfaces and implements the full 50 methods in the bigger
interface.

Yes, I could do this in my own code, but I think many people will have to
re-invent this wheel in D, and each re-invention will be different and not
interchangeable.
Feb 25 2006
next sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
surferjeff excite.com wrote:

 The interface for InputStream and OutputStream is huge.  I often have to create
 my own streambufs in C++, which only requires implementing about 5 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement OutputStream
 too!
 
 It would useful if there were smaller interfaces.  For example, a read() method
 and eof() for input, and that's it.  Then have a util class that takes one of
 these small interfaces and implements the full 50 methods in the bigger
 interface.
 
 Yes, I could do this in my own code, but I think many people will have to
 re-invent this wheel in D, and each re-invention will be different and not
 interchangeable.
 
 


Phobos sucks. There I said it. :-)

In other news, there is a library called Mango on www.dsource.org which you may
find useful.
Feb 25 2006
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
<surferjeff excite.com> wrote in message 
news:dtribg$eui$1 digitaldaemon.com...

 The interface for InputStream and OutputStream is huge.  I often have to 
 create
 my own streambufs in C++, which only requires implementing about 5 
 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement 
 OutputStream
 too!

 It would useful if there were smaller interfaces.  For example, a read() 
 method
 and eof() for input, and that's it.  Then have a util class that takes one 
 of
 these small interfaces and implements the full 50 methods in the bigger
 interface.

 Yes, I could do this in my own code, but I think many people will have to
 re-invent this wheel in D, and each re-invention will be different and not
 interchangeable.



Subclass Stream or FilterStream. You'll probably have to override 3 methods: 
readBlock, writeBlock and seek. The interfaces are primarily for users to 
restrict their API - not for library writers to shoe-horn their classes 
into. Look at any of the small stream classes in std.stream to get an idea 
how easy it is to do.
Feb 26 2006
parent reply surferjeff excite.com writes:
In article <dtsbc3$1c8g$1 digitaldaemon.com>, Ben Hinkle says...

<surferjeff excite.com> wrote in message 
news:dtribg$eui$1 digitaldaemon.com...

 The interface for InputStream and OutputStream is huge.  I often have to 
 create
 my own streambufs in C++, which only requires implementing about 5 
 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement 
 OutputStream
 too!

 It would useful if there were smaller interfaces.  For example, a read() 
 method
 and eof() for input, and that's it.  Then have a util class that takes one 
 of
 these small interfaces and implements the full 50 methods in the bigger
 interface.

 Yes, I could do this in my own code, but I think many people will have to
 re-invent this wheel in D, and each re-invention will be different and not
 interchangeable.



Subclass Stream or FilterStream. You'll probably have to override 3 methods: 
readBlock, writeBlock and seek. The interfaces are primarily for users to 
restrict their API - not for library writers to shoe-horn their classes 
into. Look at any of the small stream classes in std.stream to get an idea 
how easy it is to do. 


I see. Not so bad.  Thanks.

Instead of the "Inherit 20 methods, implement 3 pure virtual methods" approach,
how about "implement the same 3 pure virtual methods in an interface, pass the
interface to a class that uses it to implement the 20 methods."  I've used the
former pattern a lot in C++, and have found it difficult to reuse.  I recently
learned the latter pattern in Objective-C and Cocoa, and have found it much
easier to reuse.
Mar 15 2006
next sibling parent Sean Kelly <sean f4.ca> writes:
surferjeff excite.com wrote:

 
 I see. Not so bad.  Thanks.
 
 Instead of the "Inherit 20 methods, implement 3 pure virtual methods" approach,
 how about "implement the same 3 pure virtual methods in an interface, pass the
 interface to a class that uses it to implement the 20 methods."  I've used the
 former pattern a lot in C++, and have found it difficult to reuse.  I recently
 learned the latter pattern in Objective-C and Cocoa, and have found it much
 easier to reuse.


Something like:

     class MyStreamImpl()
     {
         void readBlock();
         void writeBlock();
         seek();
     }

     class Stream( Impl )
     {
         mixin Impl!() impl;
         alias impl.readBlock readBlock;
         alias impl.writeBlock writeBlock;
         alias impl.seek seek;
         ...
     }

     alias Stream!(MyStreamImpl) MyStream;


Sean
Mar 15 2006
prev sibling parent "Ben Hinkle" <bhinkle mathworks.com> writes:
<surferjeff excite.com> wrote in message 
news:dvaupb$1ok0$1 digitaldaemon.com...

 In article <dtsbc3$1c8g$1 digitaldaemon.com>, Ben Hinkle says...

<surferjeff excite.com> wrote in message
news:dtribg$eui$1 digitaldaemon.com...

 The interface for InputStream and OutputStream is huge.  I often have to
 create
 my own streambufs in C++, which only requires implementing about 5
 methods.  If
 I want to write my own class that implements InputStream, then I have to
 implement about 50 methods, and another 50 if I want to implement
 OutputStream
 too!

 It would useful if there were smaller interfaces.  For example, a read()
 method
 and eof() for input, and that's it.  Then have a util class that takes 
 one
 of
 these small interfaces and implements the full 50 methods in the bigger
 interface.

 Yes, I could do this in my own code, but I think many people will have 
 to
 re-invent this wheel in D, and each re-invention will be different and 
 not
 interchangeable.



Subclass Stream or FilterStream. You'll probably have to override 3 
methods:
readBlock, writeBlock and seek. The interfaces are primarily for users to
restrict their API - not for library writers to shoe-horn their classes
into. Look at any of the small stream classes in std.stream to get an idea
how easy it is to do.


 I see. Not so bad.  Thanks.

 Instead of the "Inherit 20 methods, implement 3 pure virtual methods" 
 approach,
 how about "implement the same 3 pure virtual methods in an interface, pass 
 the
 interface to a class that uses it to implement the 20 methods."  I've used 
 the
 former pattern a lot in C++, and have found it difficult to reuse.  I 
 recently
 learned the latter pattern in Objective-C and Cocoa, and have found it 
 much
 easier to reuse.


Sure, one can trade-off flexibility and ease-of-use. For example 
std.stream.File could just implement the 3 pure virtual methods and then 
user code would write
 Stream file = new Stream(new File("foo.txt"));
So the flexibility of multiple classes and interfaces results in user code 
having to do more to hook things up. There are many many ways to slice 
things up. At one point I posted a suggestion to have a Stream subclass that 
accepted delegates for the 3 methods and then one wouldn't have to write a 
class at all - one can just write 3 nested functions and pass those to the 
constructor. For example
 void foo() {
    char buf[100];
    size_t readBlock(void* buffer, size_t size) {
       ... do something with buf ...
    }
    .. 2 more nested functions for writeBlock and seek ...
    Stream s = new DelegateStream(&readBlock, &writeBlock, &seek);
    ... use s as any other stream ...
    }
but I don't recall that suggestion getting any lovin' so I dropped it.
I think if you have concrete suggestions go ahead and post them, see what 
kind of response you get and based on the response make changes and submit 
them to Walter and see how things go.

-Ben
Mar 16 2006