www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Passing private functions to template algorithms

reply FreeSlave <freeslave93 gmail.com> writes:
I want to bring up this unpleasant topic for discussion, because 
I'm really tired of hacks I need to do in my code.

Look at this snippet:

private bool isFileNothrow(string path)
{
     import std.file : isFile;
     import std.exception : collectException;
     bool ok;
     collectException(path.isFile, ok);
     return ok;
}

void main(string[] args)
{
     import std.algorithm : filter;
     auto r = args.filter!(isFileNothrow);
}

Looks good, but it won't compile because isFileNothrow is 
invisible to std.algorithm.

Technically it's right: function is private, so no access from 
other modules. But it makes me to do hacks like

auto r = args.filter!(p => p.isFileNothrow);

I don't see a way how can I reuse my functions while keeping it 
private without ugly hacks.

How about allowing templates from other modules to have access to 
private functions? I think this would be pretty legal behavior, 
since template is instantiated in this module anyway.
Jun 07 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07.06.2016 20:45, FreeSlave wrote:

 I want to bring up this unpleasant topic for discussion, because I'm
 really tired of hacks I need to do in my code.

 Look at this snippet:

 private bool isFileNothrow(string path)
 {
      import std.file : isFile;
      import std.exception : collectException;
      bool ok;
      collectException(path.isFile, ok);
      return ok;
 }

 void main(string[] args)
 {
      import std.algorithm : filter;
      auto r = args.filter!(isFileNothrow);
 }

 Looks good, but it won't compile because isFileNothrow is invisible to
 std.algorithm.

 Technically it's right: function is private, so no access from other
 modules. But it makes me to do hacks like

 auto r = args.filter!(p => p.isFileNothrow);

 I don't see a way how can I reuse my functions while keeping it private
 without ugly hacks.

 How about allowing templates from other modules to have access to
 private functions? I think this would be pretty legal behavior, since
 template is instantiated in this module anyway.


I think it is obvious that this should work. Visibility checks need to 
happen during identifier lookup. This lookup is happening in the module 
where isFileNothrow is visible.
Jun 07 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 7 June 2016 at 18:59:03 UTC, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks 
 need to happen during identifier lookup. This lookup is 
 happening in the module where isFileNothrow is visible.


I agree with you.
However something in the back of my mind tells me that we'll be 
hitting nasty corner cases if we change the behavior.
Jun 07 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 00:48:00 UTC, Stefan Koch wrote:

 On Tuesday, 7 June 2016 at 18:59:03 UTC, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks 
 need to happen during identifier lookup. This lookup is 
 happening in the module where isFileNothrow is visible.


 I agree with you.
 However something in the back of my mind tells me that we'll be 
 hitting nasty corner cases if we change the behavior.


Unless you can provide such corner case, this post is 
contributing nothing.
Jun 07 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 8 June 2016 at 00:58:41 UTC, deadalnix wrote:

 On Wednesday, 8 June 2016 at 00:48:00 UTC, Stefan Koch wrote:

 I agree with you.
 However something in the back of my mind tells me that we'll 
 be hitting nasty corner cases if we change the behavior.


 Unless you can provide such corner case, this post is 
 contributing nothing.


If I had a specific case in mind I would have provided it.
Jun 07 2016
parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 05:51:38 UTC, Stefan Koch wrote:

 If I had a specific case in mind I would have provided it.


Then is is FUD.
Jun 08 2016
prev sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Wednesday, 8 June 2016 at 00:48:00 UTC, Stefan Koch wrote:

 On Tuesday, 7 June 2016 at 18:59:03 UTC, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks 
 need to happen during identifier lookup. This lookup is 
 happening in the module where isFileNothrow is visible.


 I agree with you.
 However something in the back of my mind tells me that we'll be 
 hitting nasty corner cases if we change the behavior.


Something in the back of mine tells that no corner cases can be 
worse than the present situation. It is like disallowing private 
variables to be passed to imported functions.
Jun 07 2016
prev sibling next sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Tuesday, 7 June 2016 at 18:59:03 UTC, Timon Gehr wrote:


 I think it is obvious that this should work. Visibility checks 
 need to happen during identifier lookup. This lookup is 
 happening in the module where isFileNothrow is visible.


There is also this inconsistency:

----
module a;

private int _x;
alias x = _x;

private void _foo() {
}
alias foo = _foo;

----
module main;
import a;

void main(string[] args)
{
     x = 1; // ok
     foo(); // a._foo is not accessible
}


What's the reason for this? Should both aliases be private? 
Public?
Jun 08 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
On 06/07/2016 09:59 PM, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks need to
 happen during identifier lookup. This lookup is happening in the module
 where isFileNothrow is visible.


My understanding is that right now template alias argument means
transparent symbol lookup. It acts as if doesn't access alias symbol in
template but aliased one directly.

I agree such semantics are sub-optimal but changing that can break quite
some existing idioms. Consider this snippet for example:

enum name ( alias sym ) = sym.stringof;

Right now it evaluates to name of passed symbol. If we change lookup
semantics to be non-transparent, it must always return `sym` for
consistency.
Jun 12 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12.06.2016 10:28, Dicebot wrote:

 On 06/07/2016 09:59 PM, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks need to
 happen during identifier lookup. This lookup is happening in the module
 where isFileNothrow is visible.


 My understanding is that right now template alias argument means
 transparent symbol lookup. It acts as if doesn't access alias symbol in
 template but aliased one directly.
 ...


The lookup accesses the alias and is immediately rewritten to refer to 
the aliased symbol. Visibility checks should be applied during the 
lookup, but not after the rewrite.


 I agree such semantics are sub-optimal but changing that can break quite
 some existing idioms. Consider this snippet for example:

 enum name ( alias sym ) = sym.stringof;

 Right now it evaluates to name of passed symbol.  If we change lookup
 semantics to be non-transparent, it must always return `sym` for
 consistency.


I completely disagree that this would need to happen. E.g. 
fullyQualifiedName should work with private symbols just as well as with 
public ones.

Maybe this is helpful: https://en.wikipedia.org/wiki/Information_hiding
Jun 12 2016
parent reply Dicebot <public dicebot.lv> writes:
On 06/12/2016 01:55 PM, Timon Gehr wrote:

 On 12.06.2016 10:28, Dicebot wrote:

 On 06/07/2016 09:59 PM, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks need to
 happen during identifier lookup. This lookup is happening in the module
 where isFileNothrow is visible.


 My understanding is that right now template alias argument means
 transparent symbol lookup. It acts as if doesn't access alias symbol in
 template but aliased one directly.
 ...


 
 The lookup accesses the alias and is immediately rewritten to refer to
 the aliased symbol. Visibility checks should be applied during the
 lookup, but not after the rewrite.


Yes, that is natural solution but it makes alias itself first-class
symbol which leads to my example snippet question.


 I agree such semantics are sub-optimal but changing that can break quite
 some existing idioms. Consider this snippet for example:

 enum name ( alias sym ) = sym.stringof;

 Right now it evaluates to name of passed symbol.  If we change lookup
 semantics to be non-transparent, it must always return `sym` for
 consistency.


 
 I completely disagree that this would need to happen. E.g.
 fullyQualifiedName should work with private symbols just as well as with
 public ones.
 
 Maybe this is helpful: https://en.wikipedia.org/wiki/Information_hiding


Private or public is irrelevant here, it seems you have misunderstood my
concern. The essential question is "what the alias is?", not how it is
accessed. There must be no special cases for aliases regarding
private/public symbol access, it is matter of defining actual alias
semantics so that generic access rules start being useful.
Jun 12 2016
next sibling parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
The language does not prevent taking the address of a private
symbol and escaping it.
The alias case isn't (ie shouldn't be) different.

artur
Jun 12 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12.06.2016 13:05, Dicebot wrote:

 On 06/12/2016 01:55 PM, Timon Gehr wrote:

 On 12.06.2016 10:28, Dicebot wrote:

 On 06/07/2016 09:59 PM, Timon Gehr wrote:

 I think it is obvious that this should work. Visibility checks need to
 happen during identifier lookup. This lookup is happening in the module
 where isFileNothrow is visible.


 My understanding is that right now template alias argument means
 transparent symbol lookup. It acts as if doesn't access alias symbol in
 template but aliased one directly.
 ...


 The lookup accesses the alias and is immediately rewritten to refer to
 the aliased symbol. Visibility checks should be applied during the
 lookup, but not after the rewrite.


 Yes, that is natural solution but it makes alias itself first-class
 symbol which leads to my example snippet question.
 ...


In my mind it already is a first-class symbol (it can even be part of an 
overload set), but after you look it up, it is immediately rewritten to 
another symbol.

https://github.com/tgehr/d-compiler/blob/master/semantic.d#L4670

Does DMD special-case it during lookup instead?


 I agree such semantics are sub-optimal but changing that can break quite
 some existing idioms. Consider this snippet for example:

 enum name ( alias sym ) = sym.stringof;

 Right now it evaluates to name of passed symbol.  If we change lookup
 semantics to be non-transparent, it must always return `sym` for
 consistency.


 I completely disagree that this would need to happen. E.g.
 fullyQualifiedName should work with private symbols just as well as with
 public ones.

 Maybe this is helpful: https://en.wikipedia.org/wiki/Information_hiding


 Private or public is irrelevant here, it seems you have misunderstood my
 concern.


Probably I indeed misunderstood your concern, but what I disagree with 
is the notion that breaking code not directly checking for visibility is 
somehow a prerequisite for having the correct visibility checks in 
place. You may very well be right that it would be more consistent to 
return the name of the alias, but I think what .stringof returns is an 
independent concern. (Last time I checked, it was completely unspecified 
anyway btw.)


 The essential question is "what the alias is?", not how it is
 accessed. There must be no special cases for aliases regarding
 private/public symbol access,


There are no special cases in what I propose. The alias rewrite has 
nothing to do with lookup:

https://github.com/tgehr/d-compiler/blob/master/scope_.d#L185

I guess this is precisely your proposal too?


 it is matter of defining actual alias
 semantics so that generic access rules start being useful.


I agree. (Or maybe rather, it is a matter of defining both such that the 
resulting semantics are useful, I don't know precisely where DMD goes 
astray here.)
Jun 12 2016
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12.06.2016 15:52, Timon Gehr wrote:

 semantics are useful


is useful
Jun 12 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12.06.2016 15:52, Timon Gehr wrote:

 You may very well be right that it would be more consistent to return
 the name of the alias, but I think what .stringof returns is an
 independent concern. (Last time I checked, it was completely unspecified
 anyway btw.)


What do you think about the following?

enum e=3;
static assert(e.stringof=="3");
Jun 12 2016
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Tuesday, 7 June 2016 at 18:45:23 UTC, FreeSlave wrote:

 I want to bring up this unpleasant topic for discussion, 
 because I'm really tired of hacks I need to do in my code.

 [...]

 How about allowing templates from other modules to have access 
 to private functions? I think this would be pretty legal 
 behavior, since template is instantiated in this module anyway.


Yes, I agree. This is also a concern for me. I've encountered a 
more specific case when using traits 
(https://issues.dlang.org/show_bug.cgi?id=15371). One possible 
fix would be to allow templates to be instantiated in the module 
that uses them.
But I have no idea on the fisability.

The consequence now is that a lot of good templates can't be 
added to std.traits (for example to simplify the __traits() code) 
because introspection would only work on public symbols.
Jun 07 2016