• Dennis (24/24) Dec 11 2017 I'm on Windows and I recently got confused by how Phobos
• Steven Schveighoffer (20/45) Dec 11 2017 Please note that D uses FILE * as its underlying implementation. This
• Dennis (4/5) Dec 11 2017 Thanks for your reply, that clears it up.

Dennis <dkorpel gmail.com> writes:

I'm on Windows and I recently got confused by how Phobos 
functions handle newlines.

```
void main() {
     import std.stdio;
     import std.path : buildPath, tempDir;

     auto path = buildPath(tempDir(), "test.txt");
     auto file = new File(path, "w");

     file.write("hello there!\n");   //actually writes hello 
there!\r\n
     file.write('\n');               //actually writes \r\n
     file.rawWrite("\n");            //actually writes \n
     //file.rawWrite('\n');          //doesn't compile
     file.close();

     //byLine uses \n as default terminator, so a trailing \r is 
kept
     writeln(File(path).byLine.front~"huh?");   //prints "huh?o 
there!"
}
```

- Why do these functions behave like that?
- What is the easiest way to iterate over a file by line that 
works with both \r\n and \n line endings? I prefer not to write a 
new method for this.

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 12/11/17 10:40 AM, Dennis wrote:
 I'm on Windows and I recently got confused by how Phobos functions 
 handle newlines.
 
 ```
 void main() {
      import std.stdio;
      import std.path : buildPath, tempDir;
 
      auto path = buildPath(tempDir(), "test.txt");
      auto file = new File(path, "w");
 
      file.write("hello there!\n");   //actually writes hello there!\r\n
      file.write('\n');               //actually writes \r\n
      file.rawWrite("\n");            //actually writes \n
      //file.rawWrite('\n');          //doesn't compile
      file.close();
 
      //byLine uses \n as default terminator, so a trailing \r is kept
      writeln(File(path).byLine.front~"huh?");   //prints "huh?o there!"
 }
 ```
 
 - Why do these functions behave like that?
 - What is the easiest way to iterate over a file by line that works with 
 both \r\n and \n line endings? I prefer not to write a new method for this.

Please note that D uses FILE * as its underlying implementation. This 
means that whatever the C library does, D's library does.

In the case of Windows, there is this lovely concept that line endings 
should be \r\n, not just \n. So the C library helpfully inserts \r 
whenever it sees a \n. This is known as "text mode" as opposed to 
"binary mode".

rawWrite temporarily sets the mode to binary mode, then restores the 
original mode. This is why the \n is not translated there.

Here's the fun part: the default open mode is "rb". See the docs here: 
https://dlang.org/phobos/std_stdio.html#.File.this

So you are actually opening the file in binary mode, whereas you were 
writing it in text mode.

You can fix this in one of 3 ways:
1. Open your file for writing with "wb". This will not store the \r by 
default.
2. Open your file for reading with "r" (e.g. File(path, "r")). This will 
replace the \r\n with just \n in the underlying C library.
3. Stop using Windows ;)

-Steve

Dennis <dkorpel gmail.com> writes:

Thanks for your reply, that clears it up.

On Monday, 11 December 2017 at 21:13:11 UTC, Steven Schveighoffer 
wrote:
 3. Stop using Windows ;)

Haha, if only the rest of the userbase would follow.