digitalmars.D.learn - phobos: What type to use instead of File when doing I/O on streams?
- J.Frank (11/11) Nov 08 2015 Hello again,
- Alex Parrill (3/15) Nov 08 2015 Use ubyte/char input ranges. Post what you're trying to do if you
- J.Frank (28/29) Nov 08 2015 $ cat test.d
- Adam D. Ruppe (9/13) Nov 08 2015 You don't, in general. stdin is sometimes seekable and the
- J.Frank (6/15) Nov 09 2015 Hm. "Maybe the stream is seekable, maybe it is not" is not really
- J.Frank (14/14) Nov 09 2015 My question is now:
- =?UTF-8?Q?Ali_=c3=87ehreli?= (19/33) Nov 09 2015 You don't need the template constraints but it is good practice:
- J.Frank (3/20) Nov 09 2015 Uhm... no. That's not a solution to my problem. Ranges are not
- Alex Parrill (8/33) Nov 09 2015 Ranges are streams. file.byLine(Copy) and byChunk are effectively
- J.Frank (12/14) Nov 09 2015 I might be wrong, but from what I read so far I don't think that
- Adam D. Ruppe (8/9) Nov 09 2015 They are in `core.sys.posix.sys.select`
- Alex Parrill (9/21) Nov 09 2015 `myrange.take(array_size).array`
- Kagamin (4/6) Nov 10 2015 Maybe you should factor out a function that does pure data
- =?UTF-8?Q?Ali_=c3=87ehreli?= (7/32) Nov 09 2015 As far as I understand, the issue is to prevent the seek() call at
- J.Frank (4/9) Nov 09 2015 Yes and no. Again, seek() was just an example.
- Adam D. Ruppe (5/7) Nov 09 2015 Just keep using them. The deprecated note isn't really
- Adam D. Ruppe (6/10) Nov 09 2015 It just isn't known at compile time. Files, especially on Unix,
Hello again, I'm a bit puzzled by the "File" type. Looking at the implementation, it seems that all I/O functions were stuffed into a single class^H^H^H^H^Hstruct. What I expected to find is some kind of "Stream" class (with read(), write(), eof()), which is extended by a "File" class (with seek(), mmap(), etc.). So, assuming my program reads from stdin and is supposed to work on a file as well as on a pipe (which is not seekable) - how can I make the compiler bark when I accidently use stdin.seek()? Am I missing something here, too?
Nov 08 2015
On Sunday, 8 November 2015 at 20:47:08 UTC, J.Frank wrote:Hello again, I'm a bit puzzled by the "File" type. Looking at the implementation, it seems that all I/O functions were stuffed into a single class^H^H^H^H^Hstruct. What I expected to find is some kind of "Stream" class (with read(), write(), eof()), which is extended by a "File" class (with seek(), mmap(), etc.). So, assuming my program reads from stdin and is supposed to work on a file as well as on a pipe (which is not seekable) - how can I make the compiler bark when I accidently use stdin.seek()? Am I missing something here, too?Use ubyte/char input ranges. Post what you're trying to do if you want an example.
Nov 08 2015
On Sunday, 8 November 2015 at 21:57:55 UTC, Alex Parrill wrote:Post what you're trying to do if you want an example.$ cat test.d import std.stdio; //void foo(Stream in_stream, Stream out_stream) // I want something like this void foo(File in_stream, File out_stream) { in_stream.seek(3); // BUG string line; while ((line = in_stream.readln()) !is null) out_stream.write(line); } void main(string[] args) { foo(stdin, stdout); } $ cat test.txt Line 0 Line 1 Line 2 $ gdc -o test test.d $ ./test <test.txt e 0 Line 1 Line 2 $ cat test.txt | ./test std.exception.ErrnoException ../../../src/libphobos/std/stdio.d(595): Could not seek in file `' (Illegal seek) $
Nov 08 2015
On Sunday, 8 November 2015 at 20:47:08 UTC, J.Frank wrote:So, assuming my program reads from stdin and is supposed to work on a file as well as on a pipe (which is not seekable) - how can I make the compiler bark when I accidently use stdin.seek()?You don't, in general. stdin is sometimes seekable and the compiler doesn't know if it is or not until you try at runtime. You could write a wrapper struct for File though that disables the seek function. It would have a File member, alias file this;, a constructor that forwards to it (optionally, you could also construct it as a File and assign it), and then a disabled function with the same signature as the seek call in File. Then, it will no longer compile as long as you use your wrapper.
Nov 08 2015
On Sunday, 8 November 2015 at 23:50:58 UTC, Adam D. Ruppe wrote:You don't, in general. stdin is sometimes seekable and the compiler doesn't know if it is or not until you try at runtime.Hm. "Maybe the stream is seekable, maybe it is not" is not really an option for a language that is supposed to be type safe. I just found std.stream which looks very good. But it is deprecated??You could write a wrapper struct for File though that disables the seek function. It would have a File member, alias file this;, a constructor that forwards to it (optionally, you could also construct it as a File and assign it), and then a disabled function with the same signature as the seek call in File. Then, it will no longer compile as long as you use your wrapper.Thank you for your solution, but this sounds more lika a hack.
Nov 09 2015
My question is now: How can I make this work without using deprecated stuff? import std.cstream; void foo(InputStream in_stream, OutputStream out_stream) { // in_stream.seek(3); // compile error - good :) char[] line; while ((line = in_stream.readLine()) !is null) out_stream.writeLine(line); } void main(string[] args) { foo(din, dout); }
Nov 09 2015
On 11/09/2015 01:48 AM, J.Frank wrote:My question is now: How can I make this work without using deprecated stuff? import std.cstream; void foo(InputStream in_stream, OutputStream out_stream) { // in_stream.seek(3); // compile error - good :) char[] line; while ((line = in_stream.readLine()) !is null) out_stream.writeLine(line); } void main(string[] args) { foo(din, dout); }You don't need the template constraints but it is good practice: import std.stdio; import std.range; void foo(I, O)(I in_stream, O out_stream) if (isInputRange!I && isOutputRange!(O, ElementType!I)) { // in_stream.seek(3); // compile error - good :) foreach (element; in_stream) { out_stream.put(element); } } void main(string[] args) { // Also consider .byLine, which is faster and risky foo(stdin.byLineCopy, stdout.lockingTextWriter); } Ali
Nov 09 2015
On Monday, 9 November 2015 at 14:48:35 UTC, Ali Çehreli wrote:import std.stdio; import std.range; void foo(I, O)(I in_stream, O out_stream) if (isInputRange!I && isOutputRange!(O, ElementType!I)) { // in_stream.seek(3); // compile error - good :) foreach (element; in_stream) { out_stream.put(element); } } void main(string[] args) { // Also consider .byLine, which is faster and risky foo(stdin.byLineCopy, stdout.lockingTextWriter); } AliUhm... no. That's not a solution to my problem. Ranges are not (I/O-)streams.
Nov 09 2015
On Monday, 9 November 2015 at 18:18:19 UTC, J.Frank wrote:On Monday, 9 November 2015 at 14:48:35 UTC, Ali Çehreli wrote:Ranges are streams. file.byLine(Copy) and byChunk are effectively streams that are ranges. The only problem with implementing your code in ranges is that `splitter` requires a forward range, which file-backed ranges aren't. Otherwise you could do `file.byChunk(4096).joiner.drop(3).splitter('\n')` to get a "stream" of lines.import std.stdio; import std.range; void foo(I, O)(I in_stream, O out_stream) if (isInputRange!I && isOutputRange!(O, ElementType!I)) { // in_stream.seek(3); // compile error - good :) foreach (element; in_stream) { out_stream.put(element); } } void main(string[] args) { // Also consider .byLine, which is faster and risky foo(stdin.byLineCopy, stdout.lockingTextWriter); } AliUhm... no. That's not a solution to my problem. Ranges are not (I/O-)streams.
Nov 09 2015
On Monday, 9 November 2015 at 18:44:00 UTC, Alex Parrill wrote:Ranges are streams. file.byLine(Copy) and byChunk are effectively streams that are ranges.I might be wrong, but from what I read so far I don't think that "ranges are streams": - Can you read an arbitrary length of bytes from a range? (Reading by line in my code was just an example.) The only way I see is to make it a byte range. But then reading n bytes would result in n calls to popFront(), which is out of the question for performance reasons. Is this correct? - Can you flush() a range? - Can you use select() on a range? etc. (BTW. Where do I find select()/poll() in phobos?)
Nov 09 2015
On Monday, 9 November 2015 at 19:42:53 UTC, J.Frank wrote:(BTW. Where do I find select()/poll() in phobos?)They are in `core.sys.posix.sys.select` In general, OS headers for Posix in C like "sys/select.h" can be gotten by "import core.sys.posix.sys.select;" <unistd.h> is in "core.sys.posix.unistd" Linux-specific ones like sys/epoll.h are in `core.sys.linux.sys.epoll` and so on
Nov 09 2015
On Monday, 9 November 2015 at 19:42:53 UTC, J.Frank wrote:On Monday, 9 November 2015 at 18:44:00 UTC, Alex Parrill wrote:`myrange.take(array_size).array` front/popFront are very good candidates for optimization; the compiler should be able to inline them, removing all of the overhead (GCD and LDC will likely be better at this than DMD, though).Ranges are streams. file.byLine(Copy) and byChunk are effectively streams that are ranges.I might be wrong, but from what I read so far I don't think that "ranges are streams": - Can you read an arbitrary length of bytes from a range? (Reading by line in my code was just an example.) The only way I see is to make it a byte range. But then reading n bytes would result in n calls to popFront(), which is out of the question for performance reasons. Is this correct?- Can you flush() a range? - Can you use select() on a range?No, but you can't do either to anything other than file descriptors anyway (at least on Linux, dunno about Windows), so you may as well pass in a File.
Nov 09 2015
On Monday, 9 November 2015 at 19:42:53 UTC, J.Frank wrote:- Can you flush() a range? - Can you use select() on a range?Maybe you should factor out a function that does pure data processing on arbitrary ranges and manage sources of the ranges - opening, flushing and closing files - in the caller code?
Nov 10 2015
On 11/09/2015 10:18 AM, J.Frank wrote:On Monday, 9 November 2015 at 14:48:35 UTC, Ali Çehreli wrote:As far as I understand, the issue is to prevent the seek() call at compile time, right? If so, the range types that byLine() and byLineCopy() return does not have such a member function. I think the code achieves that goal because we are not dealing with File objects anymore. Aliimport std.stdio; import std.range; void foo(I, O)(I in_stream, O out_stream) if (isInputRange!I && isOutputRange!(O, ElementType!I)) { // in_stream.seek(3); // compile error - good :) foreach (element; in_stream) { out_stream.put(element); } } void main(string[] args) { // Also consider .byLine, which is faster and risky foo(stdin.byLineCopy, stdout.lockingTextWriter); } AliUhm... no. That's not a solution to my problem. Ranges are not (I/O-)streams.
Nov 09 2015
On Monday, 9 November 2015 at 19:38:06 UTC, Ali Çehreli wrote:As far as I understand, the issue is to prevent the seek() call at compile time, right? If so, the range types that byLine() and byLineCopy() return does not have such a member function. I think the code achieves that goal because we are not dealing with File objects anymore.Yes and no. Again, seek() was just an example. I'm looking for a replacement for the deprecated inferfaces InputStream and OutputStream.
Nov 09 2015
On Monday, 9 November 2015 at 19:49:15 UTC, J.Frank wrote:I'm looking for a replacement for the deprecated inferfaces InputStream and OutputStream.Just keep using them. The deprecated note isn't really important... they will still be in Phobos for another year and you can copy the file out yourself to keep using it beyond that. As of right now, there is no exact official replacement.
Nov 09 2015
On Monday, 9 November 2015 at 08:49:08 UTC, J.Frank wrote:Hm. "Maybe the stream is seekable, maybe it is not" is not really an option for a language that is supposed to be type safe.It just isn't known at compile time. Files, especially on Unix, are interchangeable at compile time but offer different capabilities at run time.Thank you for your solution, but this sounds more lika a hack.Wrapper structs with alias this are basically how D does inheritance on a struct.
Nov 09 2015