www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Stream, Socket == SocketStream...

reply "Regan Heath" <regan netwin.co.nz> writes:
Ok,

I want to create a class/wrapper that lets me call Stream methods i.e.  
readLine, writeLine, etc. But, also lets me call Socket methods i.e.  
'connect' eg.

version(build) {pragma(link, wsock32);}

import std.socket;
import std.stream;
import std.stdio;

class SocketStream : Stream
{
	this(Socket s)
	{
		_socket = s;
		readable = true;
		writeable = true;
		seekable = false;
		isopen = _socket.isAlive();
	}
	
	override size_t readBlock(void* result, size_t len)
	{
		void[] tmp = result[0..len];
		return socket.receive(tmp);
	}
	
	override size_t writeBlock(void* result, size_t len)
	{
		void[] tmp = result[0..len];
		return socket.send(tmp);
	}
	
	override ulong seek(long offset, SeekPos whence)
	{
		assert(0);
	}
private:
	Socket _socket;
}

int main(char[][] args)
{
	BufferedStream s = new BufferedStream(new SocketStream(new TcpSocket()));
	
	//this is not possible
	s.connect(new InternetAddress("www.digitalmars.com",80));

	//these work.
	s.writeLine("GET / HTTP/1.0");
	s.writeLine("");
	writefln("%s",s.readLine());
}

Before anyone points it out, yes, I realise I can put the hostname and  
port in the constructor for TcpSocket. I don't want to do that.

Before anyone (Kris) directs me to another library which contains an  
existing SocketStream like class, thanks, but I'm also interested in how  
you solve the general problem.

It seems to me, that multiple inheritance would let me do this, so, what's  
the D solution?

My first thought is that if we took the Stream class methods, placed them  
in templates TInputStream and TOutputStream, then I could mix them into my  
class.

Meaning, I could derive my SocketStream above from "TcpSocket" and mixin  
the TInputStream and TOutputStream methods.

But! doesn't that mean it's not technically a "Stream" and as such so I  
wouldn't be able to use it with BufferedStream? If so, ideally I'd want a  
"Stream" interface to derive from also.

That would mean the Stream class would have to go in favour of 3  
interfaces (InputStream, OutputStream, Stream) and 2 templates  
(TInputStream, TOutputStream).

Thoughts? Ideas?

Regan
Jun 08 2005
next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opsr2zs8mt23k2f5 nrage.netwin.co.nz...
 Ok,

 I want to create a class/wrapper that lets me call Stream methods i.e. 
 readLine, writeLine, etc. But, also lets me call Socket methods i.e. 
 'connect' eg.

 version(build) {pragma(link, wsock32);}

 import std.socket;
 import std.stream;
 import std.stdio;

 class SocketStream : Stream
 {
 this(Socket s)
 {
 _socket = s;
 readable = true;
 writeable = true;
 seekable = false;
 isopen = _socket.isAlive();
 }

 override size_t readBlock(void* result, size_t len)
 {
 void[] tmp = result[0..len];
 return socket.receive(tmp);
 }

 override size_t writeBlock(void* result, size_t len)
 {
 void[] tmp = result[0..len];
 return socket.send(tmp);
 }

 override ulong seek(long offset, SeekPos whence)
 {
 assert(0);
 }
 private:
 Socket _socket;
 }

 int main(char[][] args)
 {
 BufferedStream s = new BufferedStream(new SocketStream(new TcpSocket()));

 //this is not possible
 s.connect(new InternetAddress("www.digitalmars.com",80));

 //these work.
 s.writeLine("GET / HTTP/1.0");
 s.writeLine("");
 writefln("%s",s.readLine());
 }

 Before anyone points it out, yes, I realise I can put the hostname and 
 port in the constructor for TcpSocket. I don't want to do that.

 Before anyone (Kris) directs me to another library which contains an 
 existing SocketStream like class, thanks, but I'm also interested in how 
 you solve the general problem.

 It seems to me, that multiple inheritance would let me do this, so, what's 
 the D solution?

 My first thought is that if we took the Stream class methods, placed them 
 in templates TInputStream and TOutputStream, then I could mix them into my 
 class.

I tried factoring out TInputStream and TOutputStream a while ago but the private unget buffer couldn't go into the mixin so I didn't pursue it very far (you can't mix in private members). We could just make the unget buffer public and live with it.
 Meaning, I could derive my SocketStream above from "TcpSocket" and mixin 
 the TInputStream and TOutputStream methods.

 But! doesn't that mean it's not technically a "Stream" and as such so I 
 wouldn't be able to use it with BufferedStream? If so, ideally I'd want a 
 "Stream" interface to derive from also.

 That would mean the Stream class would have to go in favour of 3 
 interfaces (InputStream, OutputStream, Stream) and 2 templates 
 (TInputStream, TOutputStream).

 Thoughts? Ideas?

 Regan

Stream should be pretty much the same as the union of InputStream and OutputStream with the missing functions being the seeking functions and read/writeBlock. You've given a good argument for making BufferedStream wrap an abbreviated interface that just supplies the readBlock, writeBlock and seek members (and maybe one or two other things I've forgotten the buffering needs). Maybe call this interface ... something like BasicStream or SimpleStream. It would look something like interface BasicStream { size_t readBlock(void* buffer, size_t size); size_t writeBlock(void* buffer, size_t size); ulong seek(long offset, SeekPos rel); } class Stream : InputStream, OutputStream, BasicStream { ... } class BufferedStream : Stream { BasicStream s; this(BasicStream s) {this.s = s; ... } } That way your class wouldn't subclass Stream but would implement some or all of the interfaces.
Jun 09 2005
parent reply "Kris" <fu bar.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote ...
 Stream should be pretty much the same as the union of InputStream and
 OutputStream with the missing functions being the seeking functions and
 read/writeBlock. You've given a good argument for making BufferedStream

 an abbreviated interface that just supplies the readBlock, writeBlock and
 seek members (and maybe one or two other things I've forgotten the

 needs). Maybe call this interface ... something like BasicStream or
 SimpleStream. It would look something like
 interface BasicStream {
   size_t readBlock(void* buffer, size_t size);
   size_t writeBlock(void* buffer, size_t size);
   ulong seek(long offset, SeekPos rel);
 }
 class Stream : InputStream, OutputStream, BasicStream { ... }
 class BufferedStream : Stream {
   BasicStream s;
   this(BasicStream s) {this.s = s; ... }
 }
 That way your class wouldn't subclass Stream but would implement some or

 of the interfaces.

BasicStream should probably not have a seek() method, since not all streams are seekable (socket, for example). This kind of issue, and the limitations implied via Decorators, is why mango.io took the route it did. It doesn't exhibit these problems, is more flexible, and is simpler to use ;)
Jun 09 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Kris" <fu bar.com> wrote in message news:d8a2n1$22pn$1 digitaldaemon.com...
 "Ben Hinkle" <ben.hinkle gmail.com> wrote ...
 Stream should be pretty much the same as the union of InputStream and
 OutputStream with the missing functions being the seeking functions and
 read/writeBlock. You've given a good argument for making BufferedStream

 an abbreviated interface that just supplies the readBlock, writeBlock and
 seek members (and maybe one or two other things I've forgotten the

 needs). Maybe call this interface ... something like BasicStream or
 SimpleStream. It would look something like
 interface BasicStream {
   size_t readBlock(void* buffer, size_t size);
   size_t writeBlock(void* buffer, size_t size);
   ulong seek(long offset, SeekPos rel);
 }
 class Stream : InputStream, OutputStream, BasicStream { ... }
 class BufferedStream : Stream {
   BasicStream s;
   this(BasicStream s) {this.s = s; ... }
 }
 That way your class wouldn't subclass Stream but would implement some or

 of the interfaces.

BasicStream should probably not have a seek() method, since not all streams are seekable (socket, for example). This kind of issue, and the limitations implied via Decorators, is why mango.io took the route it did. It doesn't exhibit these problems, is more flexible, and is simpler to use ;)

If the goal is to make an interface that can be the backing stream for a BufferedStream then seek() is needed. Streams throw if they aren't seekable - as they throw if they aren't readable or writeable (or writable...) Anyway, some interface for the backing stream for BufferedStream, EndianStream and SliceStream will probably be fine given the template mixins. The name BasicStream doesn't sound quite right so any suggestions are welcome. Glancing at those wrapper streams the interface will probably have to inherit both InputStream and OutputStream and add one or two seek members.
Jun 09 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 9 Jun 2005 17:15:02 -0400, Ben Hinkle <bhinkle mathworks.com>  
wrote:
 "Kris" <fu bar.com> wrote in message  
 news:d8a2n1$22pn$1 digitaldaemon.com...
 "Ben Hinkle" <ben.hinkle gmail.com> wrote ...
 Stream should be pretty much the same as the union of InputStream and
 OutputStream with the missing functions being the seeking functions and
 read/writeBlock. You've given a good argument for making BufferedStream

 an abbreviated interface that just supplies the readBlock, writeBlock  
 and
 seek members (and maybe one or two other things I've forgotten the

 needs). Maybe call this interface ... something like BasicStream or
 SimpleStream. It would look something like
 interface BasicStream {
   size_t readBlock(void* buffer, size_t size);
   size_t writeBlock(void* buffer, size_t size);
   ulong seek(long offset, SeekPos rel);
 }
 class Stream : InputStream, OutputStream, BasicStream { ... }
 class BufferedStream : Stream {
   BasicStream s;
   this(BasicStream s) {this.s = s; ... }
 }
 That way your class wouldn't subclass Stream but would implement some  
 or

 of the interfaces.

BasicStream should probably not have a seek() method, since not all streams are seekable (socket, for example). This kind of issue, and the limitations implied via Decorators, is why mango.io took the route it did. It doesn't exhibit these problems, is more flexible, and is simpler to use ;)

If the goal is to make an interface that can be the backing stream for a BufferedStream then seek() is needed.

I dont think that is the 'goal'. The goal is to have a logical set of classes/templates/interfaces that define how streams are to be handled in D. Some streams aren't seekable. BufferedStream can only operate on a stream which is seekable? that doesn't seem right. What if I want to buffer my input, but never plan to seek it? It seems to me we have 3 properties Readability, Writability and Seekability, a stream may have 1, 2 or all 3 of them. So we need 3 seperate interfaces, 3 seperate templates to mixin, and some way of saying "I require a stream which can read and seek" or "write and seek" or "read and write" or "read and write and seek".
 Streams throw if they aren't
 seekable - as they throw if they aren't readable or writeable (or
 writable...)

Wouldn't it be better to know at compile time that it's not seekable. As in: void foo(ISeekable a) {} SocketStream s = new SocketStream(); foo(s); //compile time error. We can _also_ throw if the seekable nature of a stream changes.
 Anyway, some interface for the backing stream for BufferedStream,
 EndianStream and SliceStream will probably be fine given the template
 mixins. The name BasicStream doesn't sound quite right so any suggestions
 are welcome. Glancing at those wrapper streams the interface will  
 probably
 have to inherit both InputStream and OutputStream and add one or two seek
 members.

It's a stream which can read, write, and seek. SeekStream might not be too bad a name. Regan
Jun 09 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 BasicStream should probably not have a seek() method, since not all
 streams
 are seekable (socket, for example). This kind of issue, and the
 limitations
 implied via Decorators, is why mango.io took the route it did. It 
 doesn't
 exhibit these problems, is more flexible, and is simpler to use ;)

If the goal is to make an interface that can be the backing stream for a BufferedStream then seek() is needed.

I dont think that is the 'goal'. The goal is to have a logical set of classes/templates/interfaces that define how streams are to be handled in D. Some streams aren't seekable. BufferedStream can only operate on a stream which is seekable? that doesn't seem right. What if I want to buffer my input, but never plan to seek it?

BufferedStream works fine with non-seekable streams. Kris's design is to make seekable a compile-time decision. std.stream makes seekable a run-time decision. It's handy to make seekable a run-time decision since that allows better integration with std.c.stdio and the OS concept of files since pipes are considered files.
 It seems to me we have 3 properties Readability, Writability and 
 Seekability, a stream may have 1, 2 or all 3 of them. So we need 3 
 seperate interfaces, 3 seperate templates to mixin, and some way of saying 
 "I require a stream which can read and seek" or "write and seek" or "read 
 and write" or "read and write and seek".

Yup that's a possiblity. The question is how many interfaces does one need? If we had Read/Write/Seek and the combinations of those then that's seven total interfaces. I propose three of those cover enough surface area: Read, Write and Read+Write+Seek. If one wants one of the other interface then the run-time decisions ala Stream are available. The Read interface is InputStream; Write is OutputStream and Read+Write+Seek is <name TDB>.
 Streams throw if they aren't
 seekable - as they throw if they aren't readable or writeable (or
 writable...)

Wouldn't it be better to know at compile time that it's not seekable. As in: void foo(ISeekable a) {} SocketStream s = new SocketStream(); foo(s); //compile time error. We can _also_ throw if the seekable nature of a stream changes.

The use cases of a function that takes ISeekable is very small (ie I can't think of any).
 Anyway, some interface for the backing stream for BufferedStream,
 EndianStream and SliceStream will probably be fine given the template
 mixins. The name BasicStream doesn't sound quite right so any suggestions
 are welcome. Glancing at those wrapper streams the interface will 
 probably
 have to inherit both InputStream and OutputStream and add one or two seek
 members.

It's a stream which can read, write, and seek. SeekStream might not be too bad a name.

To me SeekStream would just contain the seek methods and wouldn't cover the rest of streaming. So off the top of my head something more inclusive would be better.
Jun 09 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 9 Jun 2005 19:21:44 -0400, Ben Hinkle <ben.hinkle gmail.com> wrote:
 BufferedStream works fine with non-seekable streams.

Ok.
 Kris's design is to
 make seekable a compile-time decision.

Yep.
 std.stream makes seekable a run-time decision.

A runtime decision *only*.
 It's handy to make seekable a run-time decision since that allows better  
 integration with std.c.stdio and the OS concept of files since pipes are  
 considered files.

It's handier to do both (compile+runtime)
 It seems to me we have 3 properties Readability, Writability and
 Seekability, a stream may have 1, 2 or all 3 of them. So we need 3
 seperate interfaces, 3 seperate templates to mixin, and some way of  
 saying "I require a stream which can read and seek" or "write and seek"  
 or "read and write" or "read and write and seek".

Yup that's a possiblity. The question is how many interfaces does one need? If we had Read/Write/Seek and the combinations of those then that's seven total interfaces.

True. But what's wrong with that? --basic InputStream OutputStream SeekStream --combinations InputSeekStream OutputSeekStream InputOutputStream --full InputOutputSeekStream (or simply "Stream" for brevity) It does get kinda long winded, but, you'd use the combinations rarely. Q: Why include the word 'Stream' at all? Q: What is a stream anyway? It seems to me a "Stream" is anything you can read bytes from *or* anything you can write bytes to, my definition has never included seekability. So, IMO, a "Stream" should not be seekable at all. Nor would it include any sort of unget or other advanced idea. Of course, we want seekable things, we want unget, we want buffering, we'd probably never directly use something that didn't have one of those, but, my feeling is that they should be factored in outside of the basic "Stream" interface, i.e. in BufferedStream perhaps. I think perhaps I should have a go at designing something myself, at least then I'll have something concrete to bring to the table. To be honest I am relatively new to the whole Stream concept, being from a C (not C++) background.
 Streams throw if they aren't
 seekable - as they throw if they aren't readable or writeable (or
 writable...)

Wouldn't it be better to know at compile time that it's not seekable. As in: void foo(ISeekable a) {} SocketStream s = new SocketStream(); foo(s); //compile time error. We can _also_ throw if the seekable nature of a stream changes.

The use cases of a function that takes ISeekable is very small (ie I can't think of any).

True, I was assuming it would also read or write, the example would be better asking for the combination of seek + read or write. I guess the only point I'm making is that it would be nice to be able to select/check seekability at compile time as well as runtime (which we currently have). Regan
Jun 09 2005
prev sibling parent "Kris" <fu bar.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote...
 BufferedStream works fine with non-seekable streams. Kris's design is to
 make seekable a compile-time decision. std.stream makes seekable a

 decision. It's handy to make seekable a run-time decision since that

 better integration with std.c.stdio and the OS concept of files since

 are considered files.

Correction: the developer has a choice with mango.io ~ use compile-time or runtime checking ~ one is not forced down a single path. This is not true of phobos.io, but it's really not that important <g>
Jun 09 2005
prev sibling next sibling parent reply Carlos Santander <csantander619 gmail.com> writes:
Regan Heath escribió:
 Ok,
 
 I want to create a class/wrapper that lets me call Stream methods i.e.  
 readLine, writeLine, etc. But, also lets me call Socket methods i.e.  
 'connect' eg.
 

How about std.socketstream? -- Carlos Santander Bernal
Jun 09 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 09 Jun 2005 15:20:08 -0500, Carlos Santander  
<csantander619 gmail.com> wrote:
 Regan Heath escribió:
 Ok,
  I want to create a class/wrapper that lets me call Stream methods  
 i.e.  readLine, writeLine, etc. But, also lets me call Socket methods  
 i.e.  'connect' eg.

How about std.socketstream?

It's almost exactly the same as the code I provided and has the same problems (which aren't really huge problems, it's just not quite what I'd prefer). Regan
Jun 09 2005
prev sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Being dissatisfied with the current stream implementation (no offence  
intended Ben you do some great work) I've had a play with various stream  
ideas.

I keep coming back to seperating input and output, it makes the most sense  
to me, but it's not my preffered choice for usability i.e. it's nice to  
have a single object/class and both read and write to/from it. I can't  
seem to achieve this, how I'd like, without some sort of multiple  
inheritance.

What are peoples favourite stream libraries (I've poked my nose into the  
Java one and the mango one, what else should I look at for inspiration).

I found a good solution to the whole seekability problem. You define a  
seekable interface, eg.

interface ISeekable
{
   size_t seek(size_t offset, SeekPos from);
   ..etc..
}

then you can use "static if" to check for seekability (rather than having  
a boolean flag in the class. The same applies to readability and  
writability. This way you can have compile time checking (I believe).

Regan
Jun 20 2005