www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D2 and stream

reply Mafi <mafi example.org> writes:
Hi all,
what are the directions of D2's streams. I heared std.stream is going to 
get deprecated in D2. I think it's good as long as 
BufferedFile.readLine() returns char[] instead of string. Is D2 going to 
get a Java-like stream model? What's wrong with the old one except that 
it's a bit out of date?
Mafi
Aug 08 2010
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Mafi (mafi example.org)'s article
 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to
 get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to
 get a Java-like stream model? What's wrong with the old one except that
 it's a bit out of date?
 Mafi

Two things off the top of my head: 1. std.stream doesn't correctly support files > 2GB on 32-bit Linux, and possibly on other OS's as well. 2. It doesn't have a range interface. I can't tell you how useful something like this is to me: filter!"a.length > 0"( map!strip( File(myFilename).byLine() ) ); That said, please don't use Java as a reference for how to design a stream API. It just seems horribly verbose when trying to do something simple.
Aug 08 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Mafi (mafi example.org)'s article
 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to
 get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to
 get a Java-like stream model? What's wrong with the old one except that
 it's a bit out of date?
 Mafi

Two things off the top of my head: 1. std.stream doesn't correctly support files > 2GB on 32-bit Linux, and possibly on other OS's as well.

Doesn't that have to do with using C's FILE * interface? -Steve
Aug 09 2010
parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article
 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:
 == Quote from Mafi (mafi example.org)'s article
 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to
 get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to
 get a Java-like stream model? What's wrong with the old one except that
 it's a bit out of date?
 Mafi

Two things off the top of my head: 1. std.stream doesn't correctly support files > 2GB on 32-bit Linux, and possibly on other OS's as well.

-Steve

No, std.stdio uses this. std.stream uses OS-native APIs at least in some places.
Aug 09 2010
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Mon, 09 Aug 2010 08:11:54 -0400, Steven Schveighoffer wrote:

 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:
 
 == Quote from Mafi (mafi example.org)'s article
 Hi all,
 what are the directions of D2's streams. I heared std.stream is going
 to get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going
 to get a Java-like stream model? What's wrong with the old one except
 that it's a bit out of date?
 Mafi

Two things off the top of my head: 1. std.stream doesn't correctly support files > 2GB on 32-bit Linux, and possibly on other OS's as well.

Doesn't that have to do with using C's FILE * interface?

FILE* isn't used anywhere in std.stream. You're either thinking of std.stdio.File or std.cstream.CFile. -Lars
Aug 09 2010
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 09:01:33 -0400, Lars T. Kyllingstad  
<public kyllingen.nospamnet> wrote:

 On Mon, 09 Aug 2010 08:11:54 -0400, Steven Schveighoffer wrote:

 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Mafi (mafi example.org)'s article
 Hi all,
 what are the directions of D2's streams. I heared std.stream is going
 to get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going
 to get a Java-like stream model? What's wrong with the old one except
 that it's a bit out of date?
 Mafi

Two things off the top of my head: 1. std.stream doesn't correctly support files > 2GB on 32-bit Linux, and possibly on other OS's as well.

Doesn't that have to do with using C's FILE * interface?

FILE* isn't used anywhere in std.stream. You're either thinking of std.stdio.File or std.cstream.CFile.

Ah, ok. But does that mean that the problem isn't going away by deprecating std.stream? I thought FILE * doesn't support large files. If that's the case, then assertion 1 from David is incorrect, it's not a good reason to get rid of std.stream. If that's not the case, then I'm wrong :) -Steve
Aug 09 2010
prev sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Mon, 09 Aug 2010 09:11:19 -0400, Steven Schveighoffer wrote:

 On Mon, 09 Aug 2010 09:01:33 -0400, Lars T. Kyllingstad
 <public kyllingen.nospamnet> wrote:
 
 On Mon, 09 Aug 2010 08:11:54 -0400, Steven Schveighoffer wrote:

 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Mafi (mafi example.org)'s article
 Hi all,
 what are the directions of D2's streams. I heared std.stream is
 going to get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2
 going to get a Java-like stream model? What's wrong with the old one
 except that it's a bit out of date?
 Mafi

Two things off the top of my head: 1. std.stream doesn't correctly support files > 2GB on 32-bit Linux, and possibly on other OS's as well.

Doesn't that have to do with using C's FILE * interface?

FILE* isn't used anywhere in std.stream. You're either thinking of std.stdio.File or std.cstream.CFile.

Ah, ok. But does that mean that the problem isn't going away by deprecating std.stream? I thought FILE * doesn't support large files. If that's the case, then assertion 1 from David is incorrect, it's not a good reason to get rid of std.stream. If that's not the case, then I'm wrong :)

A bit of googling shows you are right. We really should get rid of the FILE* dependency... -Lars
Aug 09 2010
prev sibling parent Adrian Matoga <epi atari8.info> writes:
On 2010-08-08 14:11, Mafi wrote:
 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to 
 get deprecated in D2. I think it's good as long as 
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to 
 get a Java-like stream model? What's wrong with the old one except that 
 it's a bit out of date?
 Mafi

AIUI from the discussion under the topic "Network I/O and streaming in D2" the whole I/O will go through std.stdio.File, which looks great for a closed set of possible streams, but I'm afraid that it won't be sufficiently extensible for some cases. Like many people, I found Java stream model bloated. However, in my opinion, Stream makes it possible to extend I/O subsystem at library level with custom streams, while still having common (standard!) interface to them, and a possibility to use a fair amount of already implemented operations like reading/writing various primitive types and arrays, at very small cost of implementing only a constructor and read/write primitives. In my modest experience, I found std.stream.Stream a good solution in some cases: once the application could have a common interface for both secure and raw network stream (the first being a custom stream, which wrapped OpenSSL SSL structures, and the second being SocketStream from std.socketstream). Another case is what I'm currently developing - a library for accessing 8-bit Atari disk images. There are a few different filesystems possible inside such images, and for each one there will be a different translation between image contents and file operations, but the file operations should be hidden behind a common interface. Again, Stream seemed the best choice for me - firstly, it comes from standard library; secondly, it requires only readBlock and writeBlock to be implemented, and the rest is already done. I also wondered if things like DirIterator and DirEntry could also be interfaces (I wrote my own interfaces for this purpose), but these could be of much less usage. To sum up, I think it's worth saving Stream as a common interface for I/O, but it possibly needs some improvements in details. For example, instead of having its own File class, conflicting with std.stdio.File, it could simply wrap std.stdio.File into, let's say, FileStream or sth. Another thing would be making the set of methods resembling that of std.stdio.File. It would be nice to use std.stdio.File when you don't need extensibility, but have an option to easily move to descendants of std.stream.Stream without changing every single call. What are your opinions? Adrian
Aug 08 2010