www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - iterate over a directory, dealing with permission errors

reply John Colvin <john.loughran.colvin gmail.com> writes:
Posting here instead of learn because I think it uncovers a 
design flaw

void main(string[] args)
{
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
}

Modify this program such that it will print "<file.name> access 
denied" instead of crashing with an exception whenever it hits a 
permissions problem. Remember that you might not even have 
permission to read the directory given in args[1]. Remember that 
access permissions can change at any time.

It can be done, but it is seriously ugly.
Sep 18 2015
next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> access 
 denied" instead of crashing with an exception whenever it hits 
 a permissions problem. Remember that you might not even have 
 permission to read the directory given in args[1]. Remember 
 that access permissions can change at any time.

 It can be done, but it is seriously ugly.
http://dlang.org/phobos/std_exception.html#.handle foreach(file; dirEntries(args[1], SpanMode.depth) .handle!(Exception, RangePrimitive.front, (e, r) => DirEntry())) { writeln(file.name); } change Exception to the Exception Type to handle and select the throwing range primitive (RangePrimitive). Then just supply a delegate that does the actual handling. This will not break any range chain!
Sep 18 2015
next sibling parent Adrian Matoga <epi atari8.info> writes:
On Friday, 18 September 2015 at 11:54:32 UTC, Robert burner 
Schadek wrote:
 http://dlang.org/phobos/std_exception.html#.handle

 foreach(file; dirEntries(args[1], SpanMode.depth)
         .handle!(Exception, RangePrimitive.front, (e, r) => 
 DirEntry())) {
     writeln(file.name);
 }

 change Exception to the Exception Type to handle and select the 
 throwing range primitive (RangePrimitive). Then just supply a 
 delegate that does the actual handling.
 This will not break any range chain!
Cool, I wish I had this idea back in 2012.
Sep 18 2015
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 18 September 2015 at 11:54:32 UTC, Robert burner 
Schadek wrote:
 On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> 
 access denied" instead of crashing with an exception whenever 
 it hits a permissions problem. Remember that you might not 
 even have permission to read the directory given in args[1]. 
 Remember that access permissions can change at any time.

 It can be done, but it is seriously ugly.
http://dlang.org/phobos/std_exception.html#.handle foreach(file; dirEntries(args[1], SpanMode.depth) .handle!(Exception, RangePrimitive.front, (e, r) => DirEntry())) { writeln(file.name); } change Exception to the Exception Type to handle and select the throwing range primitive (RangePrimitive). Then just supply a delegate that does the actual handling. This will not break any range chain!
That's neat, didn't know about std.exception.handle Unfortunately, I think there are two problems with your solution: 1) DirIteratorImpl will throw on popFront, not front. I had to look at source to find out. Is this a failure of documentation or is it actually an implementation detail? 2) it doesn't cover the case where args[1] itself is unreadable, because dirEntries will throw when it's created.
Sep 18 2015
parent reply Robert burner Schadek <rburners gmail.com> writes:
On Friday, 18 September 2015 at 12:17:25 UTC, John Colvin wrote:
 That's neat, didn't know about std.exception.handle
It is at least a year old. I created it because I had a range that threw, and there was nothing to keep a range going or reenter it.
 Unfortunately, I think there are two problems with your 
 solution:

 1) DirIteratorImpl will throw on popFront, not front. I had to 
 look at source to find out. Is this a failure of documentation 
 or is it actually an implementation detail?
That one is trivial RangePrimitive.popFront foreach(file; dirEntries(args[1], SpanMode.depth) .handle!(Exception, RangePrimitive.popFront, (e, r) => DirEntry())) { writeln(file.name); }
 2) it doesn't cover the case where args[1] itself is 
 unreadable, because dirEntries will throw when it's created.
well, creating a DirEntry is not a range operation. handle can't help you there.
Sep 18 2015
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 18 September 2015 at 12:27:37 UTC, Robert burner 
Schadek wrote:
 On Friday, 18 September 2015 at 12:17:25 UTC, John Colvin wrote:
 That's neat, didn't know about std.exception.handle
It is at least a year old. I created it because I had a range that threw, and there was nothing to keep a range going or reenter it.
 Unfortunately, I think there are two problems with your 
 solution:

 1) DirIteratorImpl will throw on popFront, not front. I had to 
 look at source to find out. Is this a failure of documentation 
 or is it actually an implementation detail?
That one is trivial RangePrimitive.popFront foreach(file; dirEntries(args[1], SpanMode.depth) .handle!(Exception, RangePrimitive.popFront, (e, r) => DirEntry())) { writeln(file.name); }
Yes, but implicit in this being an OK solution for people is that no-one ever reorganises the internals of DirIteratorImpl. I guess One could use handle to deal with *all* the range primitives as a defensive countermeasure.
 2) it doesn't cover the case where args[1] itself is 
 unreadable, because dirEntries will throw when it's created.
well, creating a DirEntry is not a range operation. handle can't help you there.
Yup :(
Sep 18 2015
next sibling parent Robert burner Schadek <rburners gmail.com> writes:
On Friday, 18 September 2015 at 12:42:26 UTC, John Colvin wrote:
 Yes, but implicit in this being an OK solution for people is 
 that no-one ever reorganises the internals of DirIteratorImpl. 
 I guess One could use handle to deal with *all* the range 
 primitives as a defensive countermeasure.
yes, "handle" is something you do after the building has started burning
Sep 18 2015
prev sibling parent Meta <jared771 gmail.com> writes:
On Friday, 18 September 2015 at 12:42:26 UTC, John Colvin wrote:
 well, creating a DirEntry is not a range operation. handle 
 can't help you there.
Yup :(
I think you could use std.exception.ifThrown in this case. foreach(file; dirEntries(args[1], SpanMode.depth) .ifThrown(DirEntry.init) .handle!(Exception, RangePrimitive.popFront, (e, r) => DirEntry())) { writeln(file.name); } Although I'm not exactly sure what to return for the "error" value... I'm not sure if DirEntry.init will work or not; you'll probably have to mess around with it.
Sep 18 2015
prev sibling parent reply Adrian Matoga <epi atari8.info> writes:
On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> access 
 denied" instead of crashing with an exception whenever it hits 
 a permissions problem. Remember that you might not even have 
 permission to read the directory given in args[1]. Remember 
 that access permissions can change at any time.

 It can be done, but it is seriously ugly.
https://github.com/D-Programming-Language/phobos/pull/931 I had to move to some urgent stuff instead of improving the PR and later I forgot about it. The discussion points out what not to do when solving this issue. :)
Sep 18 2015
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 18-Sep-2015 15:03, Adrian Matoga wrote:
 On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> access
 denied" instead of crashing with an exception whenever it hits a
 permissions problem. Remember that you might not even have permission
 to read the directory given in args[1]. Remember that access
 permissions can change at any time.

 It can be done, but it is seriously ugly.
https://github.com/D-Programming-Language/phobos/pull/931 I had to move to some urgent stuff instead of improving the PR and later I forgot about it. The discussion points out what not to do when solving this issue. :)
FYI https://github.com/D-Programming-Language/phobos/pull/2768 -- Dmitry Olshansky
Sep 18 2015
parent Spacen Jasset <spacen razemail.com> writes:
On Friday, 18 September 2015 at 14:35:39 UTC, Dmitry Olshansky 
wrote:
 On 18-Sep-2015 15:03, Adrian Matoga wrote:
 On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin 
 wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> 
 access
 denied" instead of crashing with an exception whenever it 
 hits a
 permissions problem. Remember that you might not even have 
 permission
 to read the directory given in args[1]. Remember that access
 permissions can change at any time.

 It can be done, but it is seriously ugly.
https://github.com/D-Programming-Language/phobos/pull/931 I had to move to some urgent stuff instead of improving the PR and later I forgot about it. The discussion points out what not to do when solving this issue. :)
FYI https://github.com/D-Programming-Language/phobos/pull/2768
I came across the same problem a few years ago. I can't remember if a bug was raised. it would be very handy to document the way to get around this in the dirEntries pages, especially if it involves a little convolution.
Sep 19 2015