www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Should templates have the instantiating scope's protection access

reply Tofu Ninja <joeyemmons yahoo.com> writes:
Sorry if this has been posted before.

One thing that constantly bugs me about D is how the protection 
system works when using templates. Currently the a template 
instantiation has the protection rights from the declaration 
scope, but this is not really congruent with how a lot of 
templates are used.

For example in phobos it is common to see something like:
      module bar;
      void foo(alias pred, R)(R range) { /* somehow uses pred on 
the range elements */ }


Its pretty common throughout std.algorithm. However, because of 
the way the protection system works, the pred that is passed in 
MUST be public. For example the following code won't work:
      private void mypred(RangeElementType x){ /* do something 
with x */ }
      ...
      import bar : foo;
      foo!mypred(someRange);

Because foo is declared in another module, mypred MUST be public. 
I am passing mypred explicitly, there is no way I didn't intend 
for foo to have access to it. This causes lots of unnecessary 
public functions.

An even more annoying consequence of this is that emplace can 
never be used with private constructors, this fundamentally puts 
it in a weaker position than new. This is actually a serious 
problem for std.experimental.allocator. You can't use an 
allocator in the same way you use new.

One possible solution could be that templates have the access 
rights of the instantiating scope. This is honestly what I expect 
most of the time but there are probably cases where this would 
muck things up. Maybe a more case by case solution would be 
better.

Thoughts?
Jul 05 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/5/16 1:46 PM, Tofu Ninja wrote:
 Sorry if this has been posted before.
Pretty recently actually: https://forum.dlang.org/post/rduvpfuzfzmzhoknsrkr forum.dlang.org
 One possible solution could be that templates have the access rights of
 the instantiating scope. This is honestly what I expect most of the time
 but there are probably cases where this would muck things up. Maybe a
 more case by case solution would be better.
The clear problem with this solution is that this means you must use the instantiating module as part of the template definition. A template instantiation with exactly the same parameters must behave exactly the same, no matter where it was instantiated from. What this means, is that each instantiation is now tied to the module instantiating it, with no code reuse. I think at the very least, this should be opt-in if it even makes sense to do. -Steve
Jul 05 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Tuesday, 5 July 2016 at 18:04:31 UTC, Steven Schveighoffer 
wrote:
 The clear problem with this solution is that this means you 
 must use the instantiating module as part of the template 
 definition. A template instantiation with exactly the same 
 parameters must behave exactly the same, no matter where it was 
 instantiated from.

 What this means, is that each instantiation is now tied to the 
 module instantiating it, with no code reuse. I think at the 
 very least, this should be opt-in if it even makes sense to do.
Ah that is a really good point, I knew it would muck something up. But I suppose in a lot of cases this doesn't really matter, if the template is accessing private things then it already couldn't be reused outside of the module. I think it would make a difference in cases where the template does some kind of reflection or conditional compilation based on if it has access or not. Still opt-in I think. Also there could be code re-use wherever the access rights match. So if an argument is marked as opt-in, only the instantiation scope's access to that argument would need to be tied to the template instantiation. I suppose that means there would be 4 possible instantiations, one for private, package, protected, and public access.
Jul 05 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Tuesday, 5 July 2016 at 18:48:05 UTC, Tofu Ninja wrote:
 Also there could be code re-use wherever the access rights 
 match. So if an argument is marked as opt-in, only the 
 instantiation scope's access to that argument would need to be 
 tied to the template instantiation. I suppose that means there 
 would be 4 possible instantiations, one for private, package, 
 protected, and public access.
Actually scratch that, it would have to match on access rights to the argument and any symbol accessible through the argument. It would get even crazier on recursive instantiations, so I think you are right, the template would have to be tied to the original instantiation module. If the template instantiated any other templates that had this behavior it would also have to be tied to the original module. Sounds complicated.
Jul 05 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Tuesday, 5 July 2016 at 19:00:14 UTC, Tofu Ninja wrote:
 On Tuesday, 5 July 2016 at 18:48:05 UTC, Tofu Ninja wrote:
 Also there could be code re-use wherever the access rights 
 match. So if an argument is marked as opt-in, only the 
 instantiation scope's access to that argument would need to be 
 tied to the template instantiation. I suppose that means there 
 would be 4 possible instantiations, one for private, package, 
 protected, and public access.
Actually scratch that, it would have to match on access rights to the argument and any symbol accessible through the argument. It would get even crazier on recursive instantiations, so I think you are right, the template would have to be tied to the original instantiation module. If the template instantiated any other templates that had this behavior it would also have to be tied to the original module. Sounds complicated.
Using mixin templates you can get the behavior you want to some extent, with exception that you need to type "mixin" in front when you're instantiating them. I wonder if reusing the mixin statement for normal templates would be a good idea to reuse existing code while allowing access to private members.
Jul 05 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Wednesday, 6 July 2016 at 06:43:13 UTC, ZombineDev wrote:
 Using mixin templates you can get the behavior you want to some 
 extent, with exception that you need to type "mixin" in front 
 when you're instantiating them.

 I wonder if reusing the mixin statement for normal templates 
 would be a good idea to reuse existing code while allowing 
 access to private members.
I thought about that, but the problem affects things like emplace as well and I somehow doubt that we are going to turn emplace into a mixin template. I think there needs to be some kind of opt-in attribute to indicate that the template should use the instantiating module's access rights. There are three things it would need to do... 1) It would need to cause the template instantiation to have the same access rights as the instantiating module. 2) It would need to tie the instantiation to the instantiating module, different module will have different access rights so the templates won't match. 3) It would need to transmit these access rights through other templates if they also have the opt-in attribute. All the templates would have the instantiating scopes access rights. It would be needed in cases like allocator.make which ends up calling emplace eventually. The access rights would need to be transmitted all the way to emplace. The attribute could be argument specific or applied to the whole template, I am not sure which would be better but they could both work.
Jul 06 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Wednesday, 6 July 2016 at 07:47:18 UTC, Tofu Ninja wrote:
 I think there needs to be some kind of opt-in attribute to 
 indicate that the template should use the instantiating 
 module's access rights. There are three things it would need to 
 do...

 1) It would need to cause the template instantiation to have 
 the same access rights as the instantiating module.

 2) It would need to tie the instantiation to the instantiating 
 module, different module will have different access rights so 
 the templates won't match.

 3) It would need to transmit these access rights through other 
 templates if they also have the opt-in attribute. All the 
 templates would have the instantiating scopes access rights. It 
 would be needed in cases like allocator.make which ends up 
 calling emplace eventually. The access rights would need to be 
 transmitted all the way to emplace.

 The attribute could be argument specific or applied to the 
 whole template, I am not sure which would be better but they 
 could both work.
Thinking about this, maybe the choice about attributes should be made by the user, so that if I want to give emplace access to private ctors of my structs, I have to explicitly declare inside my module that I'm going to give emplace those rights (which in turn are propagated to whatever templates emplace uses internally). This way the template writer doesn't have to bother whether his template needs those rights, it is clear which templates have them without looking at their declarations, and users can decide not to grant these rights (bonus point: this way nothing changes behaviour unless the user wants so).
Jul 06 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Wednesday, 6 July 2016 at 08:05:38 UTC, Lodovico Giaretta 
wrote:
 Thinking about this, maybe the choice about attributes should 
 be made by the user, so that if I want to give emplace access 
 to private ctors of my structs, I have to explicitly declare 
 inside my module that I'm going to give emplace those rights 
 (which in turn are propagated to whatever templates emplace 
 uses internally).
 This way the template writer doesn't have to bother whether his 
 template needs those rights, it is clear which templates have 
 them without looking at their declarations, and users can 
 decide not to grant these rights (bonus point: this way nothing 
 changes behaviour unless the user wants so).
That is an option too. However there would still need to be some way for the rights to get carried through multiple levels of templates. I suppose retransmission could be done in two ways, explicitly or implicitly. I am not sure which way would be better. If it was explicit, things like allocator.make would need to explicitly retransmit the access rights down to emplace. Something like : ###################################### auto make(T, A, ARGS...)(A alloc, ARGS args) { // ... emplace!( ShareAccess T)(/* ... */); //... } // Some other module class myClass{ private this(int x){} } // ... auto x = Mallocator.make!( ShareAccess myClass)(5); // Access rights get transmitted into make and then make retransmits them into emplace ###################################### If it was implicit the ShareAccess in make would not be necessary, the access rights would carry through to emplace implicitly. I am not sure it would be a good option though because access might get accidentally granted somewhere it shouldn't. Having it be opt-in by the user also resolves the range function example I had earlier where it needed access to mypred. In this case I think the implicit retransmission is probably better, as it would allow any range function to not care about retransmitting the rights if they pass the argument to other templates.
Jul 06 2016
parent ZombineDev <petar.p.kirov gmail.com> writes:
On Wednesday, 6 July 2016 at 08:45:25 UTC, Tofu Ninja wrote:
 On Wednesday, 6 July 2016 at 08:05:38 UTC, Lodovico Giaretta 
 wrote:
 Thinking about this, maybe the choice about attributes should 
 be made by the user, so that if I want to give emplace access 
 to private ctors of my structs, I have to explicitly declare 
 inside my module that I'm going to give emplace those rights 
 (which in turn are propagated to whatever templates emplace 
 uses internally).
 This way the template writer doesn't have to bother whether 
 his template needs those rights, it is clear which templates 
 have them without looking at their declarations, and users can 
 decide not to grant these rights (bonus point: this way 
 nothing changes behaviour unless the user wants so).
That is an option too. However there would still need to be some way for the rights to get carried through multiple levels of templates. I suppose retransmission could be done in two ways, explicitly or implicitly. I am not sure which way would be better. If it was explicit, things like allocator.make would need to explicitly retransmit the access rights down to emplace. Something like : ###################################### auto make(T, A, ARGS...)(A alloc, ARGS args) { // ... emplace!( ShareAccess T)(/* ... */); //... } // Some other module class myClass{ private this(int x){} } // ... auto x = Mallocator.make!( ShareAccess myClass)(5); // Access rights get transmitted into make and then make retransmits them into emplace ###################################### If it was implicit the ShareAccess in make would not be necessary, the access rights would carry through to emplace implicitly. I am not sure it would be a good option though because access might get accidentally granted somewhere it shouldn't. Having it be opt-in by the user also resolves the range function example I had earlier where it needed access to mypred. In this case I think the implicit retransmission is probably better, as it would allow any range function to not care about retransmitting the rights if they pass the argument to other templates.
If mixin could work with normal templates, as well as mixin templates, the following would work: auto make(T, A, ARGS...)(A alloc, ARGS args) { // ... mixin emplace!T(/* ... */); //... } // Some other module class myClass{ private this(int x){} } // ... auto x = mixin Mallocator.make!myClass(5);
Jul 06 2016