www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Ctrl-Z in windows and byLine()

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Got this from a reader:

=======================
I'm testing on Windows the code below, based on your errata changes for
p8, print 1.  It works ok if on console I enter some characters, but if I
terminate console input with ctrl-z, then there is an error exit of the
program.  So, does this code need some exception handler to handle the
immediate end of input from console?

void dict1_4()
{
	size_t[string] dict; // see errata use size_t
	foreach (line; stdin.byLine()){
		foreach (word; splitter(strip(line))){ // see errata required 
std.algorithm
			if (word in dict) continue;
			auto newID = dict.length;
			dict[word.idup] = newID; // see errata required idup
			writeln(newID, '\t', word);
		}
	}

}
=======================

I thought Ctrl-Z simply sends EOF to the reader, so this should work. 
What is byLine() doing wrong?


Thanks,

Andrei
Oct 12 2011
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 12.10.2011 20:31, Andrei Alexandrescu wrote:
 Got this from a reader:

 =======================
 I'm testing on Windows the code below, based on your errata changes for
 p8, print 1. It works ok if on console I enter some characters, but if I
 terminate console input with ctrl-z, then there is an error exit of the
 program. So, does this code need some exception handler to handle the
 immediate end of input from console?

 void dict1_4()
 {
 size_t[string] dict; // see errata use size_t
 foreach (line; stdin.byLine()){
 foreach (word; splitter(strip(line))){ // see errata required std.algorithm
 if (word in dict) continue;
 auto newID = dict.length;
 dict[word.idup] = newID; // see errata required idup
 writeln(newID, '\t', word);
 }
 }

 }
 =======================

 I thought Ctrl-Z simply sends EOF to the reader, so this should work.
 What is byLine() doing wrong?


 Thanks,

 Andrei
Works for me on Win7 x64. Yet there is a known problem with pipes in DMC runtime, but surely not console. So assuming the program was named dict, this indeed fails: echo hello | dict -- Dmitry Olshansky
Oct 12 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 12 Oct 2011 12:31:10 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 Got this from a reader:

 =======================
 I'm testing on Windows the code below, based on your errata changes for
 p8, print 1.  It works ok if on console I enter some characters, but if I
 terminate console input with ctrl-z, then there is an error exit of the
 program.  So, does this code need some exception handler to handle the
 immediate end of input from console?

 void dict1_4()
 {
 	size_t[string] dict; // see errata use size_t
 	foreach (line; stdin.byLine()){
 		foreach (word; splitter(strip(line))){ // see errata required  
 std.algorithm
 			if (word in dict) continue;
 			auto newID = dict.length;
 			dict[word.idup] = newID; // see errata required idup
 			writeln(newID, '\t', word);
 		}
 	}

 }
 =======================

 I thought Ctrl-Z simply sends EOF to the reader, so this should work.  
 What is byLine() doing wrong?
My guess would be the pipes issue with DMC. I'd have to have more information to be sure. Specifically, when an attempt to read from a closed pipe occurs, Windows flags this as an EPIPE error, but DMC incorrectly flags it as an EBADF (bad file descriptor). -Steve
Oct 12 2011
parent reply dsimcha <dsimcha yahoo.com> writes:
Probably related:

http://d.puremagic.com/issues/show_bug.cgi?id=3425

== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article
 On Wed, 12 Oct 2011 12:31:10 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:
 Got this from a reader:

 =======================
 I'm testing on Windows the code below, based on your errata changes for
 p8, print 1.  It works ok if on console I enter some characters, but if I
 terminate console input with ctrl-z, then there is an error exit of the
 program.  So, does this code need some exception handler to handle the
 immediate end of input from console?

 void dict1_4()
 {
 	size_t[string] dict; // see errata use size_t
 	foreach (line; stdin.byLine()){
 		foreach (word; splitter(strip(line))){ // see errata required
 std.algorithm
 			if (word in dict) continue;
 			auto newID = dict.length;
 			dict[word.idup] = newID; // see errata required idup
 			writeln(newID, '\t', word);
 		}
 	}

 }
 =======================

 I thought Ctrl-Z simply sends EOF to the reader, so this should work.
 What is byLine() doing wrong?
My guess would be the pipes issue with DMC. I'd have to have more information to be sure. Specifically, when an attempt to read from a closed pipe occurs, Windows flags this as an EPIPE error, but DMC incorrectly flags it as an EBADF (bad file descriptor). -Steve
Oct 12 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 12 Oct 2011 15:32:29 -0400, dsimcha <dsimcha yahoo.com> wrote:

 Probably related:

 http://d.puremagic.com/issues/show_bug.cgi?id=3425
I'd say definitely related. That's exactly the error I was seeing when testing new std.process. I can send you a preliminary dmc runtime to try that should fix the problem. I've been recently given pull-request access for DMC's runtime, but I'm having a hard time building it because it uses obscure versions of MASM which are not available anywhere. So I'm hesitant to create pull requests that I don't know if they compile. However, this change was tested by doing some library hacking. -Steve
Oct 12 2011
prev sibling parent reply Jay Norwood <jayn prismnet.com> writes:
The error came in the case of entering ctrl-z on the first line.

The error msg on WinXP 32 was
^Z
object.Exception D:\dmd\dmd2\windows\bin\..\..\src\phobos\std\stdio.d(943):
Enforcement failed
----------------
D:\dmd\dmd2\windows\bin\..\..\src\phobos\std\stdio.d(944): std


A solution that avoids the error is to add a check for stdin.eof().

void dict1_4()
{
        size_t[string] dict; // see errata use size_t
        foreach (line; stdin.byLine()){
                if (stdin.eof()) break;
                foreach (word; splitter(strip(line))){ // see errata required
std.algorithm
                        if (word in dict) continue;
                        auto newID = dict.length;
                        dict[word.idup] = newID; // see errata required idup
                        writeln(newID, '\t', word);
                }
        }

}


Andrei Alexandrescu Wrote:

 Got this from a reader:
 
 =======================
 I'm testing on Windows the code below, based on your errata changes for
 p8, print 1.  It works ok if on console I enter some characters, but if I
 terminate console input with ctrl-z, then there is an error exit of the
 program.  So, does this code need some exception handler to handle the
 immediate end of input from console?
 
 void dict1_4()
 {
 	size_t[string] dict; // see errata use size_t
 	foreach (line; stdin.byLine()){
 		foreach (word; splitter(strip(line))){ // see errata required 
 std.algorithm
 			if (word in dict) continue;
 			auto newID = dict.length;
 			dict[word.idup] = newID; // see errata required idup
 			writeln(newID, '\t', word);
 		}
 	}
 
 }
 =======================
 
 I thought Ctrl-Z simply sends EOF to the reader, so this should work. 
 What is byLine() doing wrong?
 
 
 Thanks,
 
 Andrei
Oct 12 2011
next sibling parent Jay Norwood <jayn prismnet.com> writes:
So it seems to think it has another line when processing ctrl-z from console
stdin.  I saw some comment in the eof() description about the possibility of
having to seek past the end of file on non-seekable streams in order to detect
the eof.  This might be the case when detecting ctrl-z from console, right?  
Oct 12 2011
prev sibling parent reply Jay Norwood <jayn prismnet.com> writes:
I stepped through in the debugger and here is what I noted:
1.in response to ctrl-z on the first line, stdin.byLine() call to popFront()
executes file.detach due to 0 length line returned by file.readln().

2.  surprisingly (to me), the first foreach enters the loop even though there
is a 0 length line.

3. in the inner foreach statement, strip returns 0 length line.
4. during the next iteration of  outer foreach,  during stdin.byLine()  bailOut
occurs in popFront() on the enforce(file.isOpen) ... makes sense.

So ... looks to me like the error is whatever is allowing execution to enter
the outer foreach loop when line.length is 0 and the eof is true on return from
byLine().    
Oct 12 2011
parent Jay Norwood <jayn prismnet.com> writes:
After looking at it more, I see that

1. the case of 0 length lines can't be a terminating condition for the loop
since the byLine() call removes the newline character and returns 0 length
lines for a line that has only a newline.

2. the initial test of empty(), which tests based on the file being not open,
is used as the loop terminating condition.  It doesn't catch the case of the
first line of input being ctrl-z because it is executed before the calls to
front() and popFront().  popFront() is what does the file.detach.

 
Oct 12 2011