digitalmars.D - iterate over a directory, dealing with permission errors

John Colvin (15/15) Sep 18 2015 Posting here instead of learn because I think it uncovers a

Robert burner Schadek (11/26) Sep 18 2015 http://dlang.org/phobos/std_exception.html#.handle

Adrian Matoga (3/13) Sep 18 2015 Cool, I wish I had this idea back in 2012.
John Colvin (9/38) Sep 18 2015 That's neat, didn't know about std.exception.handle

Robert burner Schadek (12/20) Sep 18 2015 It is at least a year old. I created it because I had a range

John Colvin (7/29) Sep 18 2015 Yes, but implicit in this being an OK solution for people is that

Robert burner Schadek (3/7) Sep 18 2015 yes, "handle" is something you do after the building has started
Meta (12/15) Sep 18 2015 I think you could use std.exception.ifThrown in this case.

xray (5/13) May 23 2021 Hello, is it something that still works today ?

Mike Parker (4/11) May 23 2021 `std.exception.ifThrown` means `ifThrown` is in the

xray (7/20) May 23 2021 Thanks Mike. Now I have a

Mike Parker (6/19) May 23 2021 It's not something I've had to look into before, so I have

Mike Parker (2/4) May 23 2021 I meant, `RangePrimitive.front` or `RangePrimitive.popFront`.

xray (4/8) May 23 2021 RangePrimitive.popFront

Adrian Matoga (5/20) Sep 18 2015 https://github.com/D-Programming-Language/phobos/pull/931

Dmitry Olshansky (5/27) Sep 18 2015 FYI

Spacen Jasset (6/40) Sep 19 2015 I came across the same problem a few years ago. I can't remember

John Colvin <john.loughran.colvin gmail.com> writes:

Posting here instead of learn because I think it uncovers a 
design flaw

void main(string[] args)
{
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
}

Modify this program such that it will print "<file.name> access 
denied" instead of crashing with an exception whenever it hits a 
permissions problem. Remember that you might not even have 
permission to read the directory given in args[1]. Remember that 
access permissions can change at any time.

It can be done, but it is seriously ugly.

Sep 18 2015

Robert burner Schadek <rburners gmail.com> writes:

On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> access 
 denied" instead of crashing with an exception whenever it hits 
 a permissions problem. Remember that you might not even have 
 permission to read the directory given in args[1]. Remember 
 that access permissions can change at any time.

 It can be done, but it is seriously ugly.



foreach(file; dirEntries(args[1], SpanMode.depth)
         .handle!(Exception, RangePrimitive.front, (e, r) => 
DirEntry())) {
     writeln(file.name);
}

change Exception to the Exception Type to handle and select the 
throwing range primitive (RangePrimitive). Then just supply a 
delegate that does the actual handling.
This will not break any range chain!

Sep 18 2015

Adrian Matoga <epi atari8.info> writes:

On Friday, 18 September 2015 at 11:54:32 UTC, Robert burner 
Schadek wrote:


 foreach(file; dirEntries(args[1], SpanMode.depth)
         .handle!(Exception, RangePrimitive.front, (e, r) => 
 DirEntry())) {
     writeln(file.name);
 }

 change Exception to the Exception Type to handle and select the 
 throwing range primitive (RangePrimitive). Then just supply a 
 delegate that does the actual handling.
 This will not break any range chain!

Cool, I wish I had this idea back in 2012.

Sep 18 2015

John Colvin <john.loughran.colvin gmail.com> writes:

On Friday, 18 September 2015 at 11:54:32 UTC, Robert burner 
Schadek wrote:
 On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> 
 access denied" instead of crashing with an exception whenever 
 it hits a permissions problem. Remember that you might not 
 even have permission to read the directory given in args[1]. 
 Remember that access permissions can change at any time.

 It can be done, but it is seriously ugly.



 foreach(file; dirEntries(args[1], SpanMode.depth)
         .handle!(Exception, RangePrimitive.front, (e, r) => 
 DirEntry())) {
     writeln(file.name);
 }

 change Exception to the Exception Type to handle and select the 
 throwing range primitive (RangePrimitive). Then just supply a 
 delegate that does the actual handling.
 This will not break any range chain!

That's neat, didn't know about std.exception.handle

Unfortunately, I think there are two problems with your solution:

1) DirIteratorImpl will throw on popFront, not front. I had to 
look at source to find out. Is this a failure of documentation or 
is it actually an implementation detail?

2) it doesn't cover the case where args[1] itself is unreadable, 
because dirEntries will throw when it's created.

Sep 18 2015

Robert burner Schadek <rburners gmail.com> writes:

On Friday, 18 September 2015 at 12:17:25 UTC, John Colvin wrote:
 That's neat, didn't know about std.exception.handle

It is at least a year old. I created it because I had a range 
that threw, and there was nothing to keep a range going or 
reenter it.

 Unfortunately, I think there are two problems with your 
 solution:

 1) DirIteratorImpl will throw on popFront, not front. I had to 
 look at source to find out. Is this a failure of documentation 
 or is it actually an implementation detail?

That one is trivial RangePrimitive.popFront

foreach(file; dirEntries(args[1], SpanMode.depth)
          .handle!(Exception, RangePrimitive.popFront, (e, r) =>  
DirEntry())) {
      writeln(file.name);
  }

 2) it doesn't cover the case where args[1] itself is 
 unreadable, because dirEntries will throw when it's created.

well, creating a DirEntry is not a range operation. handle can't 
help you there.

Sep 18 2015

John Colvin <john.loughran.colvin gmail.com> writes:

On Friday, 18 September 2015 at 12:27:37 UTC, Robert burner 
Schadek wrote:
 On Friday, 18 September 2015 at 12:17:25 UTC, John Colvin wrote:
 That's neat, didn't know about std.exception.handle

 It is at least a year old. I created it because I had a range 
 that threw, and there was nothing to keep a range going or 
 reenter it.

 Unfortunately, I think there are two problems with your 
 solution:

 1) DirIteratorImpl will throw on popFront, not front. I had to 
 look at source to find out. Is this a failure of documentation 
 or is it actually an implementation detail?

 That one is trivial RangePrimitive.popFront

 foreach(file; dirEntries(args[1], SpanMode.depth)
          .handle!(Exception, RangePrimitive.popFront, (e, r) =>
  DirEntry())) {
      writeln(file.name);
  }

Yes, but implicit in this being an OK solution for people is that 
no-one ever reorganises the internals of DirIteratorImpl. I guess 
One could use handle to deal with *all* the range primitives as a 
defensive countermeasure.

 2) it doesn't cover the case where args[1] itself is 
 unreadable, because dirEntries will throw when it's created.

 well, creating a DirEntry is not a range operation. handle 
 can't help you there.

Yup :(

Sep 18 2015

Robert burner Schadek <rburners gmail.com> writes:

On Friday, 18 September 2015 at 12:42:26 UTC, John Colvin wrote:
 Yes, but implicit in this being an OK solution for people is 
 that no-one ever reorganises the internals of DirIteratorImpl. 
 I guess One could use handle to deal with *all* the range 
 primitives as a defensive countermeasure.

yes, "handle" is something you do after the building has started 
burning

Sep 18 2015

Meta <jared771 gmail.com> writes:

On Friday, 18 September 2015 at 12:42:26 UTC, John Colvin wrote:
 well, creating a DirEntry is not a range operation. handle 
 can't help you there.

 Yup :(

I think you could use std.exception.ifThrown in this case.

foreach(file; dirEntries(args[1], SpanMode.depth)
                   .ifThrown(DirEntry.init)
                   .handle!(Exception, RangePrimitive.popFront, 
(e, r) =>  DirEntry()))
{
      writeln(file.name);
}

Although I'm not exactly sure what to return for the "error" 
value... I'm not sure if DirEntry.init will work or not; you'll 
probably have to mess around with it.

Sep 18 2015

xray <xray isd.lu> writes:

On Friday, 18 September 2015 at 13:53:45 UTC, Meta wrote:
 I think you could use std.exception.ifThrown in this case.

 foreach(file; dirEntries(args[1], SpanMode.depth)
                   .ifThrown(DirEntry.init)
                   .handle!(Exception, RangePrimitive.popFront, 
 (e, r) =>  DirEntry()))
 {
      writeln(file.name);
 }

Hello, is it something that still works today ?
because I have an error :

"no property `ifThrown` for type `std.file.DirIterator"

same if I use handle! directly. Do I need to import some modules ?

May 23 2021

Mike Parker <aldacron gmail.com> writes:

On Sunday, 23 May 2021 at 13:29:26 UTC, xray wrote:
 On Friday, 18 September 2015 at 13:53:45 UTC, Meta wrote:
 I think you could use std.exception.ifThrown in this case.


 Hello, is it something that still works today ?
 because I have an error :

 "no property `ifThrown` for type `std.file.DirIterator"

 same if I use handle! directly. Do I need to import some 
 modules ?

`std.exception.ifThrown` means `ifThrown` is in the 
`std.exception` module, so that's what you need to import.

https://dlang.org/phobos/std_exception.html#ifThrown

May 23 2021

xray <xray isd.lu> writes:

On Sunday, 23 May 2021 at 14:03:19 UTC, Mike Parker wrote:
 On Sunday, 23 May 2021 at 13:29:26 UTC, xray wrote:
 On Friday, 18 September 2015 at 13:53:45 UTC, Meta wrote:
 I think you could use std.exception.ifThrown in this case.


 Hello, is it something that still works today ?
 because I have an error :

 "no property `ifThrown` for type `std.file.DirIterator"

 same if I use handle! directly. Do I need to import some 
 modules ?

 `std.exception.ifThrown` means `ifThrown` is in the 
 `std.exception` module, so that's what you need to import.

 https://dlang.org/phobos/std_exception.html#ifThrown

Thanks Mike. Now I have a
"The error handler's return value(DirEntry) does not have a 
common type with the expression(DirIterator)."

So I removed the ifThrown for while and it compiles....but still 
doesn't work because of an access denied file (FileException).

I thought the exception will be caught be the closure.

May 23 2021

Mike Parker <aldacron gmail.com> writes:

On Sunday, 23 May 2021 at 14:33:10 UTC, xray wrote:
 On Sunday, 23 May 2021 at 14:03:19 UTC, Mike Parker wrote:
 On Sunday, 23 May 2021 at 13:29:26 UTC, xray wrote:
 On Friday, 18 September 2015 at 13:53:45 UTC, Meta wrote:



 Thanks Mike. Now I have a
 "The error handler's return value(DirEntry) does not have a 
 common type with the expression(DirIterator)."

Well, the post you initially replied to did say this:

 Although I'm not exactly sure what to return for the "error" 
 value... I'm not sure if DirEntry.init will work or not; you'll 
 probably have to mess around with it.

It's not something I've had to look into before, so I have 
nothing useful for you at the moment.

 So I removed the ifThrown for while and it compiles....but 
 still doesn't work because of an access denied file 
 (FileException).

 I thought the exception will be caught be the closure.

Are you calling it with `RangePrimitive.pop` or 
`RangePrimitive.popFront`?

May 23 2021

Mike Parker <aldacron gmail.com> writes:

On Sunday, 23 May 2021 at 15:12:22 UTC, Mike Parker wrote:

 Are you calling it with `RangePrimitive.pop` or 
 `RangePrimitive.popFront`?

I meant, `RangePrimitive.front` or `RangePrimitive.popFront`.

May 23 2021

xray <xray isd.lu> writes:

On Sunday, 23 May 2021 at 15:13:31 UTC, Mike Parker wrote:
 On Sunday, 23 May 2021 at 15:12:22 UTC, Mike Parker wrote:

 Are you calling it with `RangePrimitive.pop` or 
 `RangePrimitive.popFront`?

 I meant, `RangePrimitive.front` or `RangePrimitive.popFront`.

RangePrimitive.popFront

In SpanMode.depth  I still get the FileException and in 
SpanMode.breadth, the program hangs for ever.

May 23 2021

Adrian Matoga <epi atari8.info> writes:

On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> access 
 denied" instead of crashing with an exception whenever it hits 
 a permissions problem. Remember that you might not even have 
 permission to read the directory given in args[1]. Remember 
 that access permissions can change at any time.

 It can be done, but it is seriously ugly.

https://github.com/D-Programming-Language/phobos/pull/931

I had to move to some urgent stuff instead of improving the PR 
and later I forgot about it. The discussion points out what not 
to do when solving this issue. :)

Sep 18 2015

Dmitry Olshansky <dmitry.olsh gmail.com> writes:

On 18-Sep-2015 15:03, Adrian Matoga wrote:
 On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin wrote:
 Posting here instead of learn because I think it uncovers a design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> access
 denied" instead of crashing with an exception whenever it hits a
 permissions problem. Remember that you might not even have permission
 to read the directory given in args[1]. Remember that access
 permissions can change at any time.

 It can be done, but it is seriously ugly.

 https://github.com/D-Programming-Language/phobos/pull/931

 I had to move to some urgent stuff instead of improving the PR and later
 I forgot about it. The discussion points out what not to do when solving
 this issue. :)

FYI

https://github.com/D-Programming-Language/phobos/pull/2768

-- 
Dmitry Olshansky

Sep 18 2015

Spacen Jasset <spacen razemail.com> writes:

On Friday, 18 September 2015 at 14:35:39 UTC, Dmitry Olshansky 
wrote:
 On 18-Sep-2015 15:03, Adrian Matoga wrote:
 On Friday, 18 September 2015 at 11:35:45 UTC, John Colvin 
 wrote:
 Posting here instead of learn because I think it uncovers a 
 design flaw

 void main(string[] args)
 {
     import std.file : dirEntries, SpanMode;
     import std.stdio : writeln;
     foreach(file; dirEntries(args[1], SpanMode.depth))
         writeln(file.name);
 }

 Modify this program such that it will print "<file.name> 
 access
 denied" instead of crashing with an exception whenever it 
 hits a
 permissions problem. Remember that you might not even have 
 permission
 to read the directory given in args[1]. Remember that access
 permissions can change at any time.

 It can be done, but it is seriously ugly.

 https://github.com/D-Programming-Language/phobos/pull/931

 I had to move to some urgent stuff instead of improving the PR 
 and later
 I forgot about it. The discussion points out what not to do 
 when solving
 this issue. :)

 FYI

 https://github.com/D-Programming-Language/phobos/pull/2768

I came across the same problem a few years ago. I can't remember 
if a bug was raised. it would be very handy to document the way 
to get around this in the dirEntries pages, especially if it 
involves a little convolution.

Sep 19 2015

D Programming

C/C++ Programming

Other

digitalmars.D - iterate over a directory, dealing with permission errors