www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Getting a safe path for a temporary file

reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
Is it currently possible to get the path to a safe temporary 
file, i.e. one that is guaranteed to be freshly created and will 
not override another existing file?

There's `std.file.tempDir`, which doesn't create a unique file. 
Then there's `std.stdio.tmpfile()`, which does, but it returns a 
`File` object, and its `name` property is `null`.

Did I miss something? IMO this is very import functionality. One 
use case is passing these names as command line arguments to an 
external program that doesn't support stdin/stdout.
Jan 17 2015
next sibling parent reply "Laeeth Isharc" <Laeeth.nospam nospam-laeeth.com> writes:
On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz wrote:
 Is it currently possible to get the path to a safe temporary 
 file, i.e. one that is guaranteed to be freshly created and 
 will not override another existing file?

 There's `std.file.tempDir`, which doesn't create a unique file. 
 Then there's `std.stdio.tmpfile()`, which does, but it returns 
 a `File` object, and its `name` property is `null`.

 Did I miss something? IMO this is very import functionality. 
 One use case is passing these names as command line arguments 
 to an external program that doesn't support stdin/stdout.
I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html
Jan 17 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 17 January 2015 at 14:37:00 UTC, Laeeth Isharc wrote:
 On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz wrote:
 Is it currently possible to get the path to a safe temporary 
 file, i.e. one that is guaranteed to be freshly created and 
 will not override another existing file?

 There's `std.file.tempDir`, which doesn't create a unique 
 file. Then there's `std.stdio.tmpfile()`, which does, but it 
 returns a `File` object, and its `name` property is `null`.

 Did I miss something? IMO this is very import functionality. 
 One use case is passing these names as command line arguments 
 to an external program that doesn't support stdin/stdout.
I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html
Nice idea, but it still allows for intentional collision attacks :-( The only really safe solution is one that generates (probably) unique names, then opens the file with O_EXCL|O_CREAT (or whatever other means the OS provides), and if it fails, retries with a different name. `std.stdio.tmpfile()` already does that (it uses `tmpfile(3)` under the hood), but doesn't allow access to the name.
Jan 17 2015
next sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 17 January 2015 at 16:55:42 UTC, Marc Schütz wrote:
 On Saturday, 17 January 2015 at 14:37:00 UTC, Laeeth Isharc 
 wrote:
 On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz 
 wrote:
 Is it currently possible to get the path to a safe temporary 
 file, i.e. one that is guaranteed to be freshly created and 
 will not override another existing file?

 There's `std.file.tempDir`, which doesn't create a unique 
 file. Then there's `std.stdio.tmpfile()`, which does, but it 
 returns a `File` object, and its `name` property is `null`.

 Did I miss something? IMO this is very import functionality. 
 One use case is passing these names as command line arguments 
 to an external program that doesn't support stdin/stdout.
I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html
Nice idea, but it still allows for intentional collision attacks :-( The only really safe solution is one that generates (probably) unique names, then opens the file with O_EXCL|O_CREAT (or whatever other means the OS provides), and if it fails, retries with a different name. `std.stdio.tmpfile()` already does that (it uses `tmpfile(3)` under the hood), but doesn't allow access to the name.
You're looking for core.sys.posix.stdlib : mkstemp. I think that should be used by std.stdio.File as well, care to create an enhancement request in bugzilla?
Jan 17 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 17 January 2015 at 17:16:41 UTC, Tobias Pankrath
wrote:
 On Saturday, 17 January 2015 at 16:55:42 UTC, Marc Schütz wrote:
 On Saturday, 17 January 2015 at 14:37:00 UTC, Laeeth Isharc 
 wrote:
 On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz 
 wrote:
 Is it currently possible to get the path to a safe temporary 
 file, i.e. one that is guaranteed to be freshly created and 
 will not override another existing file?

 There's `std.file.tempDir`, which doesn't create a unique 
 file. Then there's `std.stdio.tmpfile()`, which does, but it 
 returns a `File` object, and its `name` property is `null`.

 Did I miss something? IMO this is very import functionality. 
 One use case is passing these names as command line 
 arguments to an external program that doesn't support 
 stdin/stdout.
I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html
Nice idea, but it still allows for intentional collision attacks :-( The only really safe solution is one that generates (probably) unique names, then opens the file with O_EXCL|O_CREAT (or whatever other means the OS provides), and if it fails, retries with a different name. `std.stdio.tmpfile()` already does that (it uses `tmpfile(3)` under the hood), but doesn't allow access to the name.
You're looking for core.sys.posix.stdlib : mkstemp. I think that should be used by std.stdio.File as well, care to create an enhancement request in bugzilla?
But it's POSIX only :-(
Jan 18 2015
prev sibling parent reply "Laeeth Isharc" <Laeethnospam nospam.laeeth.com> writes:
On Saturday, 17 January 2015 at 16:55:42 UTC, Marc Schütz wrote:
 On Saturday, 17 January 2015 at 14:37:00 UTC, Laeeth Isharc 
 wrote:
 On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz 
 wrote:
 Is it currently possible to get the path to a safe temporary 
 file, i.e. one that is guaranteed to be freshly created and 
 will not override another existing file?

 There's `std.file.tempDir`, which doesn't create a unique 
 file. Then there's `std.stdio.tmpfile()`, which does, but it 
 returns a `File` object, and its `name` property is `null`.

 Did I miss something? IMO this is very import functionality. 
 One use case is passing these names as command line arguments 
 to an external program that doesn't support stdin/stdout.
I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html
Nice idea, but it still allows for intentional collision attacks :-( The only really safe solution is one that generates (probably) unique names, then opens the file with O_EXCL|O_CREAT (or whatever other means the OS provides), and if it fails, retries with a different name. `std.stdio.tmpfile()` already does that (it uses `tmpfile(3)` under the hood), but doesn't allow access to the name.
I don't follow why a collision attack is applicable in this case. Your stage 1 of generating unique names: how is this different from using a random uuid?
Jan 17 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 18 January 2015 at 00:51:37 UTC, Laeeth Isharc wrote:
 I don't follow why a collision attack is applicable in this 
 case.
  Your stage 1 of generating unique names: how is this different 
 from using a random uuid?
It's not different, and if you're still doing the O_EXCL open afterwards, it's safe. I just assumed you were going to use the generated filename without a further check. This is then unsafe, no matter how the UUID is generated, and depending on the RNG that's been used, they can be quite predictable. Granted, the risk is low, but still...
Jan 18 2015
parent "Kagamin" <spam here.lot> writes:
On Sunday, 18 January 2015 at 11:21:52 UTC, Marc Schütz wrote:
 It's not different, and if you're still doing the O_EXCL open 
 afterwards, it's safe. I just assumed you were going to use the 
 generated filename without a further check. This is then 
 unsafe, no matter how the UUID is generated, and depending on 
 the RNG that's been used, they can be quite predictable. 
 Granted, the risk is low, but still...
tmpfile is more predictable: it generates sequential file names.
Jan 18 2015
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Saturday, January 17, 2015 13:47:37 via Digitalmars-d-learn wrote:
 Is it currently possible to get the path to a safe temporary
 file, i.e. one that is guaranteed to be freshly created and will
 not override another existing file?

 There's `std.file.tempDir`, which doesn't create a unique file.
 Then there's `std.stdio.tmpfile()`, which does, but it returns a
 `File` object, and its `name` property is `null`.

 Did I miss something? IMO this is very import functionality. One
 use case is passing these names as command line arguments to an
 external program that doesn't support stdin/stdout.
The _only_ way to this write is to randomly generate a file name and then open the file with O_CREAT | O_EXCL (or more likely, O_RDWR | O_CREAT | O_EXCL), and then retry with a new name if the creation fails (good enough random name generation would likely require only two attempts at most). Simply randomly generating a file name is not enough, because it's still technically possible for the file to already exist (even if it's unlikely), and even checking for the file's existence prior to opening it isn't enough, because technically, the file could be created by another program in the small amount of time between when you checked for the file's existence and tried to create it. POSIX actually has mkstemp for doing this for you, but on some operating systems, it restricts the number of random files that it can generate to as little as 26 (at least, that's what I recall the number being). I don't think that any of the POSIX systems that we currently support have an implementation of mkstemp that's quite that bad, but all in all, I don't think that using mkstemp is a good idea. The problem is solved simply enough by randomly generating a file name (e.g. with rndGen()) and then using the correct flags with open. And I actually have code that does this that I was working on getting into Phobos, but the problem was getting access to the correct function on Windows (_wsopen_s, I believe). It wasn't available in druntime, and I didn't get around to fixing that (IIRC, because I started looking into the whole problem of how to deal with windows bindings in druntime in general and going down a rat hole that I didn't have time for). So, I never finished that pull request, and I really should get back to it. But I think that what we need is a function in std.stdio (e.g tempFile insteaf of tmpfile) which returns an open File with a randomly generated name and gives you access to its name rather than using C's tmpfile, which does not give you access to the name and deletes the file on you when it's closed. IMHO, tmpfile is pretty useless - especially when it comes to unit tests. - Jonathan M Davis
Jan 17 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 17 January 2015 at 21:32:18 UTC, Jonathan M Davis 
via Digitalmars-d-learn wrote:
 But I think that what we need is a function in std.stdio (e.g 
 tempFile
 insteaf of tmpfile) which returns an open File with a randomly 
 generated
 name and gives you access to its name rather than using C's 
 tmpfile, which
 does not give you access to the name and deletes the file on 
 you when it's
 closed.
Right - I overlooked this fact. The bad thing is that you might even be forced to close the file before another program can open it, if either of the programs wants to open it exclusively (probably most likely to happen on Windows).
Jan 18 2015