www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Atomicity of file-copying/moving

reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
What's the status of atomicity of file-copying and -moving 
(renaming) using std.file on different platforms?
May 16
next sibling parent reply FreeSlave <freeslave93 gmail.com> writes:
On Tuesday, 16 May 2017 at 08:32:56 UTC, Nordlöw wrote:
 What's the status of atomicity of file-copying and -moving 
 (renaming) using std.file on different platforms?
Not sure about renaming but copying is not atomic on Posix because it does not handle interruption by signal. I opened issue about that https://issues.dlang.org/show_bug.cgi?id=17296
May 16
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 16 May 2017 at 10:57:08 UTC, FreeSlave wrote:
 Not sure about renaming but copying is not atomic on Posix 
 because it does not handle interruption by signal. I opened 
 issue about that https://issues.dlang.org/show_bug.cgi?id=17296
The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking the temporary to the target file. If an error occurs during copying you either retry or abort. That should be atomic.
May 17
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 17 May 2017 at 19:56:52 UTC, Per Nordlöw wrote:
 On Tuesday, 16 May 2017 at 10:57:08 UTC, FreeSlave wrote:
 Not sure about renaming but copying is not atomic on Posix 
 because it does not handle interruption by signal. I opened 
 issue about that https://issues.dlang.org/show_bug.cgi?id=17296
The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking
Correction: should be renaming.
May 17
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 17 May 2017 at 20:02:44 UTC, Per Nordlöw wrote:
 The standard way is to copy the source to a temporary file on 
 the same file system as the target file followed by hardlinking
Correction: should be renaming.
Here's an implementation in Python (3): https://github.com/nordlow/containerize/blob/1dbcf57c1240882f8492f261962df0dfaa4edecb/containerize.py#L132 I would like to have it in D too. Any advice regarding the port?
May 17
prev sibling parent "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Wed, May 17, 2017 at 07:56:52PM +0000, Per Nordlw via Digitalmars-d-learn
wrote:
 On Tuesday, 16 May 2017 at 10:57:08 UTC, FreeSlave wrote:
 
 Not sure about renaming but copying is not atomic on Posix because
 it does not handle interruption by signal. I opened issue about that
 https://issues.dlang.org/show_bug.cgi?id=17296
The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking the temporary to the target file. If an error occurs during copying you either retry or abort. That should be atomic.
Unfortunately it does suffer from the limitation that if the file is large, you may run out of space on the target filesystem where you may not have, had you overwritten the target file directly. But I suppose it's an acceptable tradeoff where atomicity of copying is desired. T -- "The number you have dialed is imaginary. Please rotate your phone 90 degrees and try again."
May 17
prev sibling next sibling parent Andrew Godfrey <X y.com> writes:
On Tuesday, 16 May 2017 at 08:32:56 UTC, Nordlöw wrote:
 What's the status of atomicity of file-copying and -moving 
 (renaming) using std.file on different platforms?
For renaming that's a good question, but for copying, no-one should make atomicity guarantees. It's inherently non-atomic, and if you try to build an atomic wrapper around it (by trying to catch failure cases and deleting the file), you'd be ignoring cases like power failure, system hang, process crash. Some of those could be achieved on some OSes, but I doubt all of them can on all OSes.
May 17
prev sibling parent Jerry <Kickupx gmail.com> writes:
On Tuesday, 16 May 2017 at 08:32:56 UTC, Nordlöw wrote:
 What's the status of atomicity of file-copying and -moving 
 (renaming) using std.file on different platforms?
Niall has a good talk about this on youtube: https://www.youtube.com/watch?v=uhRWMGBjlO8
May 17