www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - File needs to be closed on Windows but not on Posix, bug?

reply "Joakim" <dlang joakim.airpost.net> writes:
I ran into this when trying to fix the Phobos unit tests and have 
reduced it down to this test file:

import std.stdio, std.file;

void main() {
     auto f = File("test.txt", "w");
     //f.close();
     std.file.remove("test.txt");
}

This compiles and runs fine on linux and the autotester shows 
that it works on all the tested Posix platforms, but it fails on 
Windows with

std.file.FileException std\file.d(433): test.txt: The process 
cannot access the file because it is being used by another 
process.

Uncommenting the f.close() gets it to work fine on all supported 
platforms, no doubt it has to do with the different OS APIs that 
are being called.

This seems like inconsistent behavior: should I file a bug?
Jul 05 2014
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 5 July 2014 at 20:23:03 UTC, Joakim wrote:

 This seems like inconsistent behavior: should I file a bug?


This is because the operating systems do it differently; I think 
D is doing the right thing by being a pretty thin wrapper around 
that functionality.

If anything, I'd just say deleting an open file may not be 
permitted in the docs, implying that the user should always close 
it themselves before deleting it.
Jul 05 2014
parent "Kagamin" <spam here.lot> writes:
It can be also a bad user experience, when delete succeeds only 
pertially and doesn't free the disk space. Delete-on-close flag 
should be better in this regard.
Jul 07 2014
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On windows files are traditionally opened without delete sharing, 
such files can't be deleted until closed, because all sharing 
options are always honored.
Jul 07 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
See if stdio allows you to specify delete sharing when opening 
the file.
Jul 07 2014
parent reply "Joakim" <dlang joakim.airpost.net> writes:
On Monday, 7 July 2014 at 10:19:01 UTC, Kagamin wrote:

 See if stdio allows you to specify delete sharing when opening 
 the file.


I don't know what delete sharing is exactly, but the File 
constructor simply calls fopen and I don't see any option for the 
Windows fopen that seems to do it:

http://msdn.microsoft.com/en-us/library/yeby3zcb.aspx
Jul 07 2014
next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 07 Jul 2014 12:17:34 +0100, Joakim <dlang joakim.airpost.net>  
wrote:


 On Monday, 7 July 2014 at 10:19:01 UTC, Kagamin wrote:

 See if stdio allows you to specify delete sharing when opening the file.


 I don't know what delete sharing is exactly, but the File constructor  
 simply calls fopen and I don't see any option for the Windows fopen that  
 seems to do it:

 http://msdn.microsoft.com/en-us/library/yeby3zcb.aspx


The fopen variant that allows you to specify sharing is:
http://msdn.microsoft.com/en-us/library/8f30b0db.aspx

But it does not mention delete sharing there.

CreateFile allows sharing to be specified for delete however:
http://msdn.microsoft.com/en-gb/library/windows/desktop/aa363858(v=vs.85).aspx

So... you could:
  - Call CreateFile giving you a "handle"
  - Call _open_osfhandle to get a "file descriptor"
  - Call _fdopen on the file descriptor to get a FILE* for it

But!  I agree with Adam, leave it as a thin wrapper.  Being a windows  
programmer by trade I would expect the remove to fail, I would not expect  
all my files to be opened with delete sharing enabled by default.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Jul 07 2014
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 7 July 2014 at 12:00:48 UTC, Regan Heath wrote:

 But!  I agree with Adam, leave it as a thin wrapper.  Being a 
 windows programmer by trade I would expect the remove to fail, 
 I would not expect all my files to be opened with delete 
 sharing enabled by default.

 R


And I believe behavior is still different. In Linux an open file 
can be accessed even after a delete operation (unlink). But in 
Windows is that possible?
Jul 07 2014
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 07 Jul 2014 15:18:51 +0100, Jesse Phillips  
<Jesse.K.Phillips+D gmail.com> wrote:


 On Monday, 7 July 2014 at 12:00:48 UTC, Regan Heath wrote:

 But!  I agree with Adam, leave it as a thin wrapper.  Being a windows  
 programmer by trade I would expect the remove to fail, I would not  
 expect all my files to be opened with delete sharing enabled by default.

 R


 And I believe behavior is still different. In Linux an open file can be  
 accessed even after a delete operation (unlink). But in Windows is that  
 possible?


Not sure, I've never done this.  It's just not something you would  
typically want/try to do on windows.

If I had to guess, I would say it would still be possible to access the  
file.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Jul 07 2014
parent reply "Kagamin" <spam here.lot> writes:
On Monday, 7 July 2014 at 14:25:54 UTC, Regan Heath wrote:

 If I had to guess, I would say it would still be possible to 
 access the file.


It's documented so. I guess, linux implements file deletion with 
delete-on-close feature too, if it exists, a separate deletion 
operation is not needed.
Jul 07 2014
parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 7 July 2014 at 16:02:33 UTC, Kagamin wrote:

 On Monday, 7 July 2014 at 14:25:54 UTC, Regan Heath wrote:

 If I had to guess, I would say it would still be possible to 
 access the file.


 It's documented so. I guess, linux implements file deletion 
 with delete-on-close feature too, if it exists, a separate 
 deletion operation is not needed.


It's not really a delete on close. The file system link is 
removed, but there is still a handle open to the location. This 
ends up working more like a GC, file is not returned for reuse 
until there are no more references to it.
Jul 08 2014
prev sibling parent "Joakim" <dlang joakim.airpost.net> writes:
On Monday, 7 July 2014 at 12:00:48 UTC, Regan Heath wrote:

 On Mon, 07 Jul 2014 12:17:34 +0100, Joakim 
 <dlang joakim.airpost.net> wrote:


 On Monday, 7 July 2014 at 10:19:01 UTC, Kagamin wrote:

 See if stdio allows you to specify delete sharing when 
 opening the file.


 I don't know what delete sharing is exactly, but the File 
 constructor simply calls fopen and I don't see any option for 
 the Windows fopen that seems to do it:

 http://msdn.microsoft.com/en-us/library/yeby3zcb.aspx


 The fopen variant that allows you to specify sharing is:
 http://msdn.microsoft.com/en-us/library/8f30b0db.aspx

 But it does not mention delete sharing there.

 CreateFile allows sharing to be specified for delete however:
 http://msdn.microsoft.com/en-gb/library/windows/desktop/aa363858(v=vs.85).aspx

 So... you could:
  - Call CreateFile giving you a "handle"
  - Call _open_osfhandle to get a "file descriptor"
  - Call _fdopen on the file descriptor to get a FILE* for it

 But!  I agree with Adam, leave it as a thin wrapper.  Being a 
 windows programmer by trade I would expect the remove to fail, 
 I would not expect all my files to be opened with delete 
 sharing enabled by default.


Thanks for all the info.  I'm looking at it from the point of 
view of a beginner or someone who just writes a quick D script 
that does something like this.  They may find it confusing that 
it works on linux but doesn't work on Windows, especially given 
the confusing error message.

But it is good practice to close a file before deleting and if D 
prefers to have thin wrappers around OS APIs, that implies less 
hand-holding like this, so maybe we just tell such people to 
close the file first.
Jul 07 2014
prev sibling parent "rumbu" <rumbu rumbu.ro> writes:
On Monday, 7 July 2014 at 11:17:35 UTC, Joakim wrote:

 On Monday, 7 July 2014 at 10:19:01 UTC, Kagamin wrote:

 See if stdio allows you to specify delete sharing when opening 
 the file.




In std.stdio, File(name, mode) is forwarded to _wfopen on Windows 
and to fopen on posix.

On Windows there are many mode flags supported by the _wfopen 
function (link above):

If you intend to delete the file immediately after closing it, 
you can use the "D" mode flag when instantiating the File struct.

-or-

You can write your own wrapper around CreateFile/Ex WinAPI 
function and use FILE_SHARE_DELETE for dwSharingMode; now you can 
safely delete the file without closing it even from another 
process.
Jul 07 2014