www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Returning const? -- A potential solution

reply Jason House <jason.james.house gmail.com> writes:
The ugly const thread got me thinking about the old problem of returning an 
input while preserving const safety.  I have an idea that seems 
reasonable...

In a nutshell, I'm thinking that const(T) should be a base type for T, 
immutable(T) and return(T).  return(T) is treated in a read-only fashion, 
just like const(T) and immutable(T). Here are some of the key things I think 
this achieves:
  * Input arguments are never mutated
  * Use of input parameters in calls to functions as const(T) is 100% legal.
  * Temporary variables can legally be defined and used
  * Calling other functions that return return(T) is allowed
  * No code duplication
  * No code bloat (compiler only needs to generate one version of the code)
  * Functions can be virtual

Let's take a relatively simple example: max

return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }

When max is called, the compiler would examine the inputs for a and b to 
determine what the true type for return(T) is from the callee's 
perspective...  So a call with T and immutable(T) would use const(T) as the 
perceived return type while an argument of T and T would use T as the return 
type.

At the call site, the compiler would ensure type safety of how the return 
type is used.  Within max, the compiler would ensure that the arguments are 
either treated as const(T) in function calls but not mixed with with types 
T, const(T), or immutable(T)

PS: The return(T) notation is an arbitrary one for the purposes of this 
post.  We need a technical solution before worrying about the color of the 
bicycle shed
Mar 07 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Jason House wrote:
 The ugly const thread got me thinking about the old problem of returning an 
 input while preserving const safety.  I have an idea that seems 
 reasonable...
 
 In a nutshell, I'm thinking that const(T) should be a base type for T, 
 immutable(T) and return(T).

Wasn't there something like typeof(return) for this? I do agree that something like this needs to be done: I shouldn't have to muck about with mixins and/or templates just because I've written a function that will work irrespective of const-ness, and thus should work with any const-ness on the arguments.
 return(T) is treated in a read-only fashion, 
 just like const(T) and immutable(T). Here are some of the key things I think 
 this achieves:
   * Input arguments are never mutated

From whose perspective?  This is one of the weird things about

I assume you meant "Input arguments are never mutated by the callee."
   * Use of input parameters in calls to functions as const(T) is 100% legal.

Well, one would hope so. Given that this would mean that arguments being T, const(T) or immutable(T) would be valid, this seems to be in the same boat as: "Features of water: it's got hydrogen in it!"
   * Temporary variables can legally be defined and used

I don't get this. In what context? I wasn't aware that using const(T) in your code prevented you from having temporary variables...
   * Calling other functions that return return(T) is allowed

I'd bloody well hope so!
   * No code duplication

THIS right here should be the #1 reason. Having to create multiple versions of a function just to correctly propogate const-ness is a right PITA. Yes, templates, mixins, etc.: I shouldn't HAVE to resort to them.
   * No code bloat (compiler only needs to generate one version of the code)

Perhaps it would be worth adding this: "Compilers may optionally, as a quality of implementation feature, generate an overload of the function which specifically handles immutable arguments. This should only be done for optimised builds. Any user code which disallows this optimisation may be reported as a warning." Considering that one of the reasons for having transitive immutability is to aid in optimisation, it seems a shame to toss it overboard with a pair of lead shoes.
   * Functions can be virtual

Well, they can be virtual now: you just have to manually instantiate the function 3 times.
 Let's take a relatively simple example: max
 
 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }
 
 When max is called, the compiler would examine the inputs for a and b to 
 determine what the true type for return(T) is from the callee's 
 perspective...  So a call with T and immutable(T) would use const(T) as the 
 perceived return type while an argument of T and T would use T as the return 
 type.
 
 At the call site, the compiler would ensure type safety of how the return 
 type is used.  Within max, the compiler would ensure that the arguments are 
 either treated as const(T) in function calls but not mixed with with types 
 T, const(T), or immutable(T)

One would hope the following would work: static if( isConst!(typeof(a)) ) { // Slow, safe way } else { // Exploit mutability }
 PS: The return(T) notation is an arbitrary one for the purposes of this 
 post.  We need a technical solution before worrying about the color of the 
 bicycle shed

It should be red, obviously. :P On a more serious note, the return(T) syntax worries me because it looks like there's a template at play here, but I can't see it. But as you say, we should worry about getting Walter to agree this is needed first. :)
Mar 07 2009
next sibling parent reply "Tim M" <a b.com> writes:
On Sun, 08 Mar 2009 17:56:09 +1300, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:

 Jason House wrote:
 The ugly const thread got me thinking about the old problem of  
 returning an
 input while preserving const safety.  I have an idea that seems
 reasonable...

 In a nutshell, I'm thinking that const(T) should be a base type for T,
 immutable(T) and return(T).

Wasn't there something like typeof(return) for this? I do agree that something like this needs to be done: I shouldn't have to muck about with mixins and/or templates just because I've written a function that will work irrespective of const-ness, and thus should work with any const-ness on the arguments.
 return(T) is treated in a read-only fashion,
 just like const(T) and immutable(T). Here are some of the key things I  
 think
 this achieves:
   * Input arguments are never mutated

 From whose perspective?  This is one of the weird things about

I assume you meant "Input arguments are never mutated by the callee."
   * Use of input parameters in calls to functions as const(T) is 100%  
 legal.

Well, one would hope so. Given that this would mean that arguments being T, const(T) or immutable(T) would be valid, this seems to be in the same boat as: "Features of water: it's got hydrogen in it!"
   * Temporary variables can legally be defined and used

I don't get this. In what context? I wasn't aware that using const(T) in your code prevented you from having temporary variables...
   * Calling other functions that return return(T) is allowed

I'd bloody well hope so!
   * No code duplication

THIS right here should be the #1 reason. Having to create multiple versions of a function just to correctly propogate const-ness is a right PITA. Yes, templates, mixins, etc.: I shouldn't HAVE to resort to them.
   * No code bloat (compiler only needs to generate one version of the  
 code)

Perhaps it would be worth adding this: "Compilers may optionally, as a quality of implementation feature, generate an overload of the function which specifically handles immutable arguments. This should only be done for optimised builds. Any user code which disallows this optimisation may be reported as a warning." Considering that one of the reasons for having transitive immutability is to aid in optimisation, it seems a shame to toss it overboard with a pair of lead shoes.
   * Functions can be virtual

Well, they can be virtual now: you just have to manually instantiate the function 3 times.
 Let's take a relatively simple example: max

 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }

 When max is called, the compiler would examine the inputs for a and b to
 determine what the true type for return(T) is from the callee's
 perspective...  So a call with T and immutable(T) would use const(T) as  
 the
 perceived return type while an argument of T and T would use T as the  
 return
 type.

 At the call site, the compiler would ensure type safety of how the  
 return
 type is used.  Within max, the compiler would ensure that the arguments  
 are
 either treated as const(T) in function calls but not mixed with with  
 types
 T, const(T), or immutable(T)

One would hope the following would work: static if( isConst!(typeof(a)) ) { // Slow, safe way } else { // Exploit mutability }
 PS: The return(T) notation is an arbitrary one for the purposes of this
 post.  We need a technical solution before worrying about the color of  
 the
 bicycle shed

It should be red, obviously. :P On a more serious note, the return(T) syntax worries me because it looks like there's a template at play here, but I can't see it. But as you say, we should worry about getting Walter to agree this is needed first. :)

What does this mean: module tconst; import std.stdio; invariant(char)[] func() { invariant(char)[] s = "hello"; return s; } void main() { auto s = func(); s[0] = 'm'; //error } I thought we already have returning const/invariant? That code ^ works fine for me.
Mar 07 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
If you're not actually responding to a post, please don't quote the
entire thing in your message.

Tim M wrote:
 What does this mean:
 
 module tconst;
 
 import std.stdio;
 
 invariant(char)[] func()
 {
       invariant(char)[] s = "hello";
       return s;
 }
 
 void main()
 {
       auto s = func();
       s[0] = 'm'; //error
 }
 
 I thought we already have returning const/invariant? That code ^ works
 fine for me.

You missed the point. This has nothing to do with returning invariant types. Jason is proposing a way to create a function which maintains the const-ness of its arguments without having to implement multiple versions. In other words, return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; } Would be similar to the following: T max(T a, T b){ return (a>b)?a:b; } const(T) max(const(T) a, const(T) b){ return (a>b)?a:b; } invariant(T) max(invariant(T) a, invariant(T) b){ return (a>b)?a:b; } Except that each would share a single implementation. -- Daniel
Mar 07 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Tim M wrote:
 
 If you're not actually responding to a post, please don't quote the
 entire thing in your message.


The subject should have been more accurate then but yes preserving constness is very usefull though I prefer the inferred template way:

One issue with this is that template functions can't be virtual. You can work around it, but it's really putting barriers up to easy use of the new const system, which I think is a bad thing. -- Daniel
Mar 07 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Tim M wrote:
 On Sun, 08 Mar 2009 20:12:20 +1300, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:
 
 One issue with this is that template functions can't be virtual.

 You can work around it, but it's really putting barriers up to easy use
 of the new const system, which I think is a bad thing.

   -- Daniel

I'd rather have virtual template functions, even if it means I have to re-compile all subclasses when the vtbls get modified. Is there any technicality causing it to be impossible to implement or just something rather tricky?

Tricky?! /gobsmacked
 class C
 {
     void fn(T);
 }

T can be anything, and can do anything. The ONLY way for this to work, as far as I know, is for one of two things to happen: 1. The compiler has to know every single invocation of fn that takes place, and instantiate the template for that type for every class that derives from C. This, of course, means that you can't ever load code at runtime through, say, a dynamic library, because it might try to call a method that doesn't exist, but is actually defined. This also means that the D compilers have to be changed to no longer use single-file compilation. Additionally, you still can't ever have a compiled library with virtual template members, since there's no way for the library to know what types it should be compiled with: there's an infinite number! 2. The runtime has to be able to instantiate the template at run-time. This means you have to store the source for the template in the executable, along with a full symbol table. The D runtime then has to contain a full D compiler, including all libraries and probably a linker, too. Remember that the template could contain string mixins, so you really can't be "clever": you have to include the whole thing. Saying this is "tricky" is a tremendous understatement. -- Daniel
Mar 07 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Tim M wrote:
 Firstly option 2 was just crazy and I don't know why you said that.

Because it's how .NET does it. Granted, .NET doesn't really have templates; just deferred type erasure. Still, it's basically the same idea. .NET has it a lot easier since its binaries include considerable reflection information, plus the standard library has a platform-independent assembler.
 For
 option 1 you identifying the limitations and are just trying to apply as
 many problems as you can though the main barriers can easily be lowered.

That's how I think: it's easy to come up with simple cases where a design works. The trick is to come up with a design that works even for the complex cases.
 As you already know templates are a compile time feature, so that
 statement about the compiler needs to every possible call doesn't make
 that much sense, the template inference works only for compile time type
 recognition and that is all that my sample code I posted used:

 ...
 
 This code when run will print: B- A.max()
 
 It is still a "B" but compiler can only see the "A.(T)max(T,T)" being
 called. What can actually be computed by the compiler is that another
 member function with the same exact signature exists in one or more
 subclasses. With this knowledge it can compile the "max" template as it
 did with "A" for all it's subclasses and insert the runtime type
 checking to decide which of them to call.

It's somewhat hypocritical to say "the compiler [needing to know] every possible call doesn't make that much sense" and then require the compiler know of every possible subclass of a given class. It's the same issue, dressed in different clothes and a fake beard. Avoiding dynamic instantiation, you HAVE to know what template instances are going to be needed at compile time. You can't wave your hands and make it go away.
 So just to help clarify the template is not instantiated in different
 ways but the same instantiation is done for each compatible class function.
 
 The example currently compiles in:
 
 A.max(invariant int)()
 
 I am suggesting to compile in:
 
 A.max(invariant int)()
 B.max(invariant int)()
 //any other subclasses
 
 + runtime type checking

OK: an instance has a pointer to its vtable. Virtual method resolution works by assigning each method an offset in this table at compile time. When called, the method's slot is found and the method's address is read out. So in order to have virtual templated members, you have to stuff the instantiations into the vtable, or something like the vtable, somewhere. Let's take your example: the compiler knows all subclasses. This requires all-at-once compiling to be mandated by the language; any given file will depend not only on the modules it directly references, but all modules that subclass any classes it references. It also means you cannot ever have classes subclassed by an external library, since these will be unavailable at compile time. You still need to find a vtable for these methods. Since you don't want to know all function calls, we can't extend the existing vtable because otherwise we can't predict the order in which extra entries will show up. So let's add a new hidden field for each instance that points to a hashtable of methods, indexed by mangled template name, for that class. The compiler then generates a static ctor that fills in the hashtable at program startup. Calling a virtual template member now requires a hashtable lookup, plus another 4 bytes for every object instance. Knowing every function invocation isn't much better: the one advantage of that method is that the vtable can be fixed, so we don't need the hashtable any more. But in either case, I don't think the tradeoff is anywhere near worth it. The only solution that I think would be acceptable would be the second I mentioned; and that's obviously a tremendous amount of work. But I could be wrong. It could be that there's a really easy way of implementing this. That said, I have no idea what it is, and I've never seen anyone suggest anything feasible. -- Daniel
Mar 08 2009
next sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Tim M wrote:
 On Sun, 08 Mar 2009 23:12:31 +1300, Daniel Keep
 <daniel.keep.lists gmail.com> wrote:
 It's somewhat hypocritical to say "the compiler [needing to know] every
 possible call doesn't make that much sense" and then require the
 compiler know of every possible subclass of a given class.

That argument has no validity as non templated functions have been working as virtual for years.

That doesn't have any bearing on this.
 Avoiding dynamic instantiation, you HAVE to know what template instances
 are going to be needed at compile time.  You can't wave your hands and
 make it go away.

I'm not saying guess the template instance. I'm saying use the same exact types for instance in class A to instantiate template in class B where the signature is exactly the same. There is absolutely no guess work as everything is KNOWN.

Virtual functions do NOT require the compiler to know about all subclasses. Subclasses override the entries in the vtable, which only requires the subclass to know about the superclass, not vice versa. In order to be able to instantiate the template member for B, the compiler MUST KNOW ALL SUBCLASSES OF A. It cannot know all subclasses of A unless you feed it the entire program at compile time. It cannot know about subclasses which are loaded at run time. This means a host application cannot use subclasses in a library, and a library cannot use virtual template members in any class in the host application unless it lucks out and they're compiled.
 OK: an instance has a pointer to its vtable.  Virtual method resolution
 works by assigning each method an offset in this table at compile time.
  When called, the method's slot is found and the method's address is
 read out.

 So in order to have virtual templated members, you have to stuff the
 instantiations into the vtable, or something like the vtable, somewhere.

Don't worry the class won't feel a thing as long as the subclasses are recompiled. This is required for non templated functions anyway but a different subject for another day.

Changing a line in an unrelated module does NOT require an entire class hierarchy to be recompiled.
 Let's take your example: the compiler knows all subclasses.  This
 requires all-at-once compiling to be mandated by the language; any given
 file will depend not only on the modules it directly references, but all
 modules that subclass any classes it references.  It also means you
 cannot ever have classes subclassed by an external library, since these
 will be unavailable at compile time.

There is really no extra complexities that non template functions already deal with.

If you can't see that, yes, there are extra complexities, then there's no point in continuing this thread as we're obviously just talking past each other. -- Daniel
Mar 08 2009
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-03-08 06:38:13 -0400, "Tim M" <a b.com> said:

 Avoiding dynamic instantiation, you HAVE to know what template instances
 are going to be needed at compile time.  You can't wave your hands and
 make it go away.

I'm not saying guess the template instance. I'm saying use the same exact types for instance in class A to instantiate template in class B where the signature is exactly the same. There is absolutely no guess work as everything is KNOWN.

If you introduce a way to limit templates to what generics can do in Java and C#, you can have virtual template functions. Java and C# generics can do only do a subset of what templates can do, but this ensure there's only one compiled code instanciation. So perhaps non-final non-static member template functions could be constrained to generic-like operations and thus could become virtual. I remembrer myself proposing this a few months ago, but it didn't caught on. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Mar 08 2009
next sibling parent Christopher Wright <dhasenan gmail.com> writes:
Tim M wrote:
 On Mon, 09 Mar 2009 00:58:14 +1300, Michel Fortin 
 <michel.fortin michelf.com> wrote:
 
 If you introduce a way to limit templates to what generics can do in 
 Java and C#, you can have virtual template functions. Java and C# 
 generics can do only do a subset of what templates can do, but this 
 ensure there's only one compiled code instanciation. So perhaps 
 non-final non-static member template functions could be constrained to 
 generic-like operations and thus could become virtual.

 I remembrer myself proposing this a few months ago, but it didn't 
 caught on.

I remember you blogging about a way of compiling base classes with new methods and not needing to recompile the sub classes, I will read up on those genrics in C# and java later. If that doesn't work out, what if the compiler could check for all sub class functions within the same module and allowing a sort of limited virtual template functions, so no work through external libraries. I would prefer limited virtual over no virtual.

People would constantly complain and file bug reports about the limitations. It's all or nothing. Nothing is the superior choice here.
Mar 08 2009
prev sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-03-08 08:16:42 -0400, "Tim M" <a b.com> said:

 I remember you blogging about a way of compiling base classes with new  
 methods and not needing to recompile the sub classes, I will read up on 
  those genrics in C# and java later. If that doesn't work out, what if 
 the  compiler could check for all sub class functions within the same 
 module  and allowing a sort of limited virtual template functions, so 
 no work  through external libraries. I would prefer limited virtual 
 over no virtual.

Well, if you add a way to dynamically insert methods into objects, as I was indeed proposing on my blog, the only missing step is to compile the template at first use and insert it into the virtual table. But that would require a compiler in the runtime and preservation of the template source and its related types (perhaps as bytecode). What I was proposing in the post you quoted doesn't require anything special in the runtime or the language other than the ability to impose restrictions about what a template can do in a virtual function to make sure that, whatever the template arguments, it always emit the same code. That'd work for return values using the same constness as an argument; it'd also work for making return values being of the same class as an argument when your function expect a base class. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Mar 08 2009
prev sibling parent Christopher Wright <dhasenan gmail.com> writes:
Daniel Keep wrote:
 
 Tim M wrote:
 Firstly option 2 was just crazy and I don't know why you said that.

Because it's how .NET does it. Granted, .NET doesn't really have templates; just deferred type erasure. Still, it's basically the same idea.

This is a much easier feature to support, assuming you have sufficient reflection. You'd output some type constraints for the metadata on the generic element. On instantiation of a generic method, the runtime would check the given type against those constraints. Local variables of the generic type would be stored in an array on the heap, perhaps, or you could extend the stack as necessary. Methods and properties would all be accessed via reflection. Dog slow, and it only works out as syntactic sugar. It isn't worthwhile.
Mar 08 2009
prev sibling parent Christopher Wright <dhasenan gmail.com> writes:
Tim M wrote:
 On Sun, 08 Mar 2009 20:12:20 +1300, Daniel Keep 
 <daniel.keep.lists gmail.com> wrote:
 
 One issue with this is that template functions can't be virtual.

 You can work around it, but it's really putting barriers up to easy use
 of the new const system, which I think is a bad thing.

   -- Daniel

I'd rather have virtual template functions, even if it means I have to re-compile all subclasses when the vtbls get modified. Is there any technicality causing it to be impossible to implement or just something rather tricky?

You can have virtual template functions, but you have to compile your entire application and all its libraries at once, or record the instantiated virtual template functions somewhere and handle all affected classes in a different phase of compilation. If Walter merely forbid creating object files, it would be relatively easy. It'd be a much larger change if you wanted to support compiling files one at a time or creating libraries. Still possible, though.
Mar 08 2009
prev sibling parent "Joel C. Salomon" <joelcsalomon gmail.com> writes:
Daniel Keep wrote:
 In other words,
 
 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }
 
 Would be similar to the following:
 
 T max(T a, T b){ return (a>b)?a:b; }
 const(T) max(const(T) a, const(T) b){ return (a>b)?a:b; }
 invariant(T) max(invariant(T) a, invariant(T) b){ return (a>b)?a:b; }
 
 Except that each would share a single implementation.

Sounds like you want to declare a template along with a list of types for which it should be instantiated. If the list is defined along with the template, there’s no reason the set of functions couldn’t be made virtual. —Joel Salomon
Mar 08 2009
prev sibling next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Daniel Keep wrote:

 
 
 Jason House wrote:
 The ugly const thread got me thinking about the old problem of returning
 an
 input while preserving const safety.  I have an idea that seems
 reasonable...
 
 In a nutshell, I'm thinking that const(T) should be a base type for T,
 immutable(T) and return(T).

Wasn't there something like typeof(return) for this? I do agree that something like this needs to be done: I shouldn't have to muck about with mixins and/or templates just because I've written a function that will work irrespective of const-ness, and thus should work with any const-ness on the arguments.
 return(T) is treated in a read-only fashion,
 just like const(T) and immutable(T). Here are some of the key things I
 think this achieves:
   * Input arguments are never mutated

From whose perspective?  This is one of the weird things about

I assume you meant "Input arguments are never mutated by the callee."

Yes. The callee never mutates it.
   * Use of input parameters in calls to functions as const(T) is 100%
   legal.

Well, one would hope so. Given that this would mean that arguments being T, const(T) or immutable(T) would be valid, this seems to be in the same boat as: "Features of water: it's got hydrogen in it!"
   * Temporary variables can legally be defined and used

I don't get this. In what context? I wasn't aware that using const(T) in your code prevented you from having temporary variables...
   * Calling other functions that return return(T) is allowed

I'd bloody well hope so!
   * No code duplication

THIS right here should be the #1 reason. Having to create multiple versions of a function just to correctly propogate const-ness is a right PITA. Yes, templates, mixins, etc.: I shouldn't HAVE to resort to them.
   * No code bloat (compiler only needs to generate one version of the
   code)

Perhaps it would be worth adding this: "Compilers may optionally, as a quality of implementation feature, generate an overload of the function which specifically handles immutable arguments. This should only be done for optimised builds. Any user code which disallows this optimisation may be reported as a warning." Considering that one of the reasons for having transitive immutability is to aid in optimisation, it seems a shame to toss it overboard with a pair of lead shoes.

That's an important point that should be addressed after the basic solution is found.
   * Functions can be virtual

Well, they can be virtual now: you just have to manually instantiate the function 3 times.

I was thinking of the contrast to template-based solutions that avoid code duplication, but cause both code bloat and are not virtual. (Virtual template functions is a whole thread in itself that I hope to avoid at the moment)
 Let's take a relatively simple example: max
 
 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }
 
 When max is called, the compiler would examine the inputs for a and b to
 determine what the true type for return(T) is from the callee's
 perspective...  So a call with T and immutable(T) would use const(T) as
 the perceived return type while an argument of T and T would use T as the
 return type.
 
 At the call site, the compiler would ensure type safety of how the return
 type is used.  Within max, the compiler would ensure that the arguments
 are either treated as const(T) in function calls but not mixed with with
 types T, const(T), or immutable(T)

One would hope the following would work: static if( isConst!(typeof(a)) ) { // Slow, safe way } else { // Exploit mutability }

In such scenarios, people are not complaining about code duplication and will just code an overloaded function... I hope that can stay out of this thread since it's really a different problem.
 PS: The return(T) notation is an arbitrary one for the purposes of this
 post.  We need a technical solution before worrying about the color of
 the bicycle shed

It should be red, obviously. :P On a more serious note, the return(T) syntax worries me because it looks like there's a template at play here, but I can't see it. But as you say, we should worry about getting Walter to agree this is needed first. :)

Mar 07 2009
parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Jason House wrote:
 Daniel Keep wrote:
 Perhaps it would be worth adding this:

 "Compilers may optionally, as a quality of implementation feature,
 generate an overload of the function which specifically handles
 immutable arguments.  This should only be done for optimised builds.
 Any user code which disallows this optimisation may be reported as a
 warning."

 Considering that one of the reasons for having transitive immutability
 is to aid in optimisation, it seems a shame to toss it overboard with a
 pair of lead shoes.

That's an important point that should be addressed after the basic solution is found.

Why? If the solution is designed without thought to this, you could end up in a situation where it's no longer possible. I'm not saying this has to be done from the get-go; I'm saying it should be kept in mind so we don't lock ourselves out from it.
   * Functions can be virtual

function 3 times.

I was thinking of the contrast to template-based solutions that avoid code duplication, but cause both code bloat and are not virtual. (Virtual template functions is a whole thread in itself that I hope to avoid at the moment)

You can define a templated function, then mix it in with three different type arguments. Yes, there's two functions in there which probably have the same implementation, but they're still virtual. As for virtual templates, that's impossible unless the D runtime grows a compiler.
 One would hope the following would work:

 static if( isConst!(typeof(a)) )
 {
     // Slow, safe way
 }
 else
 {
     // Exploit mutability
 }

In such scenarios, people are not complaining about code duplication and will just code an overloaded function... I hope that can stay out of this thread since it's really a different problem.

What if it's a one-line difference in a 20-line function? And it's very related to this: you either allow the above construct or you don't. Let's say the compiler takes the method, generates the single implementation, and off it goes. How does it parse the above construct? You've got to define THAT either way. -- Daniel
Mar 07 2009
prev sibling next sibling parent "Tim M" <a b.com> writes:
On Sun, 08 Mar 2009 19:18:52 +1300, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:

 If you're not actually responding to a post, please don't quote the
 entire thing in your message.

 Tim M wrote:
 What does this mean:

 module tconst;

 import std.stdio;

 invariant(char)[] func()
 {
       invariant(char)[] s = "hello";
       return s;
 }

 void main()
 {
       auto s = func();
       s[0] = 'm'; //error
 }

 I thought we already have returning const/invariant? That code ^ works
 fine for me.

You missed the point. This has nothing to do with returning invariant types. Jason is proposing a way to create a function which maintains the const-ness of its arguments without having to implement multiple versions. In other words, return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; } Would be similar to the following: T max(T a, T b){ return (a>b)?a:b; } const(T) max(const(T) a, const(T) b){ return (a>b)?a:b; } invariant(T) max(invariant(T) a, invariant(T) b){ return (a>b)?a:b; } Except that each would share a single implementation. -- Daniel

Mar 07 2009
prev sibling next sibling parent "Tim M" <a b.com> writes:
 If you're not actually responding to a post, please don't quote the
 entire thing in your message.


The subject should have been more accurate then but yes preserving constness is very usefull though I prefer the inferred template way: module tconst; import std.stdio; T max(T)(T a, T b) { if(a > b) return a; else return b; } void main() { invariant int a = 2; invariant int b = 4; auto c = max(a,b); //c becomes an invariant int c = 5; //should fail as c is also invariant }
Mar 07 2009
prev sibling next sibling parent "Tim M" <a b.com> writes:
On Sun, 08 Mar 2009 20:12:20 +1300, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:

 One issue with this is that template functions can't be virtual.

 You can work around it, but it's really putting barriers up to easy use
 of the new const system, which I think is a bad thing.

   -- Daniel

I'd rather have virtual template functions, even if it means I have to re-compile all subclasses when the vtbls get modified. Is there any technicality causing it to be impossible to implement or just something rather tricky?
Mar 07 2009
prev sibling next sibling parent "Tim M" <a b.com> writes:
Firstly option 2 was just crazy and I don't know why you said that. For  
option 1 you identifying the limitations and are just trying to apply as  
many problems as you can though the main barriers can easily be lowered.

As you already know templates are a compile time feature, so that  
statement about the compiler needs to every possible call doesn't make  
that much sense, the template inference works only for compile time type  
recognition and that is all that my sample code I posted used:



import std.stdio;

class A
{
       char[] name;
       this()
       {
             name = "A".dup;
       }
       T max(T)(T a, T b)
       {
             writefln(name ~ "- A.max()");
             return a > b ? a : b;
       }

}

class B : A
{
       this()
       {
             name = "B".dup;
       }
       T max(T)(T a, T b)
       {
             writefln(name ~ "- B.max()");
             return a > b ? a : b;
       }
}


void main()
{
       A b = new B();
       invariant int i = 2;
       invariant int j = 4;
       auto k = b.max(i,j); //invariant int is the only template compiled in

}

This code when run will print: B- A.max()

It is still a "B" but compiler can only see the "A.(T)max(T,T)" being  
called. What can actually be computed by the compiler is that another  
member function with the same exact signature exists in one or more  
subclasses. With this knowledge it can compile the "max" template as it  
did with "A" for all it's subclasses and insert the runtime type checking  
to decide which of them to call.

So just to help clarify the template is not instantiated in different ways  
but the same instantiation is done for each compatible class function.

The example currently compiles in:

A.max(invariant int)()

I am suggesting to compile in:

A.max(invariant int)()
B.max(invariant int)()
//any other subclasses

+ runtime type checking
Mar 08 2009
prev sibling next sibling parent "Tim M" <a b.com> writes:
On Sun, 08 Mar 2009 23:12:31 +1300, Daniel Keep  
<daniel.keep.lists gmail.com> wrote:


 That's how I think: it's easy to come up with simple cases where a
 design works.  The trick is to come up with a design that works even for
 the complex cases.

What I am suggesting has strong importance without any changes to syntax, or any changes that may brake existing code.
 It's somewhat hypocritical to say "the compiler [needing to know] every
 possible call doesn't make that much sense" and then require the
 compiler know of every possible subclass of a given class.

That argument has no validity as non templated functions have been working as virtual for years.
 Avoiding dynamic instantiation, you HAVE to know what template instances
 are going to be needed at compile time.  You can't wave your hands and
 make it go away.

I'm not saying guess the template instance. I'm saying use the same exact types for instance in class A to instantiate template in class B where the signature is exactly the same. There is absolutely no guess work as everything is KNOWN.
 OK: an instance has a pointer to its vtable.  Virtual method resolution
 works by assigning each method an offset in this table at compile time.
  When called, the method's slot is found and the method's address is
 read out.

 So in order to have virtual templated members, you have to stuff the
 instantiations into the vtable, or something like the vtable, somewhere.

Don't worry the class won't feel a thing as long as the subclasses are recompiled. This is required for non templated functions anyway but a different subject for another day.
 Let's take your example: the compiler knows all subclasses.  This
 requires all-at-once compiling to be mandated by the language; any given
 file will depend not only on the modules it directly references, but all
 modules that subclass any classes it references.  It also means you
 cannot ever have classes subclassed by an external library, since these
 will be unavailable at compile time.

There is really no extra complexities that non template functions already deal with. PS: .net is in a whole different ball game and is storred in a common intermediate language not source but that kind of a high level is still not required to have a virtual templated function.
Mar 08 2009
prev sibling parent "Tim M" <a b.com> writes:
On Mon, 09 Mar 2009 00:58:14 +1300, Michel Fortin  
<michel.fortin michelf.com> wrote:

 If you introduce a way to limit templates to what generics can do in  
 Java and C#, you can have virtual template functions. Java and C#  
 generics can do only do a subset of what templates can do, but this  
 ensure there's only one compiled code instanciation. So perhaps  
 non-final non-static member template functions could be constrained to  
 generic-like operations and thus could become virtual.

 I remembrer myself proposing this a few months ago, but it didn't caught  
 on.

I remember you blogging about a way of compiling base classes with new methods and not needing to recompile the sub classes, I will read up on those genrics in C# and java later. If that doesn't work out, what if the compiler could check for all sub class functions within the same module and allowing a sort of limited virtual template functions, so no work through external libraries. I would prefer limited virtual over no virtual.
Mar 08 2009
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sun, 08 Mar 2009 06:44:48 +0300, Jason House <jason.james.house gmail.com>
wrote:

 The ugly const thread got me thinking about the old problem of returning  
 an
 input while preserving const safety.  I have an idea that seems
 reasonable...

 In a nutshell, I'm thinking that const(T) should be a base type for T,
 immutable(T) and return(T).  return(T) is treated in a read-only fashion,
 just like const(T) and immutable(T). Here are some of the key things I  
 think
 this achieves:
   * Input arguments are never mutated
   * Use of input parameters in calls to functions as const(T) is 100%  
 legal.
   * Temporary variables can legally be defined and used
   * Calling other functions that return return(T) is allowed
   * No code duplication
   * No code bloat (compiler only needs to generate one version of the  
 code)
   * Functions can be virtual

 Let's take a relatively simple example: max

 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }

 When max is called, the compiler would examine the inputs for a and b to
 determine what the true type for return(T) is from the callee's
 perspective...  So a call with T and immutable(T) would use const(T) as  
 the
 perceived return type while an argument of T and T would use T as the  
 return
 type.

 At the call site, the compiler would ensure type safety of how the return
 type is used.  Within max, the compiler would ensure that the arguments  
 are
 either treated as const(T) in function calls but not mixed with with  
 types
 T, const(T), or immutable(T)

 PS: The return(T) notation is an arbitrary one for the purposes of this
 post.  We need a technical solution before worrying about the color of  
 the
 bicycle shed

How about this one? class Foo { T value() { return _value; } T value() const { return _value; } } You suggest turning it into: return(T) value() { return _value; } Which is fine but it doesn't say anything about constness of 'this'. In fact, you propose *exactly* the same thing I've been proposing multiply times in past under a different name, though: sameconst(T) max(sameconst(T) a, sameconst(T) b) { return (a > b) ? a : b; } class Foo { sameconst(T) value() sameconst(this) { // return value has the same constness as 'this' (mutable, const or immutable) return _value; } } The constness is automatically propogated based on parameters. For example, - sameconst(T) == T if all the parameters are mutable - sameconst(T) == immutable(T) if all the parameters are immutable - sameconst(T) == const(T) otherwise
Mar 08 2009
parent reply Jason House <jason.james.house gmail.com> writes:
Denis Koroskin Wrote:

 On Sun, 08 Mar 2009 06:44:48 +0300, Jason House <jason.james.house gmail.com>
wrote:
 
 The ugly const thread got me thinking about the old problem of returning  
 an
 input while preserving const safety.  I have an idea that seems
 reasonable...

 In a nutshell, I'm thinking that const(T) should be a base type for T,
 immutable(T) and return(T).  return(T) is treated in a read-only fashion,
 just like const(T) and immutable(T). Here are some of the key things I  
 think
 this achieves:
   * Input arguments are never mutated
   * Use of input parameters in calls to functions as const(T) is 100%  
 legal.
   * Temporary variables can legally be defined and used
   * Calling other functions that return return(T) is allowed
   * No code duplication
   * No code bloat (compiler only needs to generate one version of the  
 code)
   * Functions can be virtual

 Let's take a relatively simple example: max

 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }

 When max is called, the compiler would examine the inputs for a and b to
 determine what the true type for return(T) is from the callee's
 perspective...  So a call with T and immutable(T) would use const(T) as  
 the
 perceived return type while an argument of T and T would use T as the  
 return
 type.

 At the call site, the compiler would ensure type safety of how the return
 type is used.  Within max, the compiler would ensure that the arguments  
 are
 either treated as const(T) in function calls but not mixed with with  
 types
 T, const(T), or immutable(T)

 PS: The return(T) notation is an arbitrary one for the purposes of this
 post.  We need a technical solution before worrying about the color of  
 the
 bicycle shed

How about this one? class Foo { T value() { return _value; } T value() const { return _value; } }

My proposal did not handle that. I wasn't thinking of that case.
 You suggest turning it into:
 
 return(T) value() {
     return _value;
 }
 
 Which is fine but it doesn't say anything about constness of 'this'.
 
 In fact, you propose *exactly* the same thing I've been proposing multiply
times in past under a different name, though:
 
 sameconst(T) max(sameconst(T) a, sameconst(T) b) {
     return (a > b) ? a : b;
 }
 
 class Foo {
     sameconst(T) value() sameconst(this) { // return value has the same
constness as 'this' (mutable, const or immutable)
         return _value;
     }
 }
 
 The constness is automatically propogated based on parameters. For example,
 - sameconst(T) == T if all the parameters are mutable
 - sameconst(T) == immutable(T) if all the parameters are immutable
 - sameconst(T) == const(T) otherwise

I'll take another look at your proposal. I thought of sameconst as a type alias rather than a typedef. I was mostly thinking of how this problem could be handled by the type system.
Mar 08 2009
parent reply Jason House <jason.james.house gmail.com> writes:
Jason House Wrote:

 Denis Koroskin Wrote:
 
 On Sun, 08 Mar 2009 06:44:48 +0300, Jason House <jason.james.house gmail.com>
wrote:
 
 The ugly const thread got me thinking about the old problem of returning  
 an
 input while preserving const safety.  I have an idea that seems
 reasonable...

 In a nutshell, I'm thinking that const(T) should be a base type for T,
 immutable(T) and return(T).  return(T) is treated in a read-only fashion,
 just like const(T) and immutable(T). Here are some of the key things I  
 think
 this achieves:
   * Input arguments are never mutated
   * Use of input parameters in calls to functions as const(T) is 100%  
 legal.
   * Temporary variables can legally be defined and used
   * Calling other functions that return return(T) is allowed
   * No code duplication
   * No code bloat (compiler only needs to generate one version of the  
 code)
   * Functions can be virtual

 Let's take a relatively simple example: max

 return(T) max(return(T) a, return(T) b){ return (a>b)?a:b; }

 When max is called, the compiler would examine the inputs for a and b to
 determine what the true type for return(T) is from the callee's
 perspective...  So a call with T and immutable(T) would use const(T) as  
 the
 perceived return type while an argument of T and T would use T as the  
 return
 type.

 At the call site, the compiler would ensure type safety of how the return
 type is used.  Within max, the compiler would ensure that the arguments  
 are
 either treated as const(T) in function calls but not mixed with with  
 types
 T, const(T), or immutable(T)

 PS: The return(T) notation is an arbitrary one for the purposes of this
 post.  We need a technical solution before worrying about the color of  
 the
 bicycle shed

How about this one? class Foo { T value() { return _value; } T value() const { return _value; } }

My proposal did not handle that. I wasn't thinking of that case.
 You suggest turning it into:
 
 return(T) value() {
     return _value;
 }
 
 Which is fine but it doesn't say anything about constness of 'this'.
 
 In fact, you propose *exactly* the same thing I've been proposing multiply
times in past under a different name, though:
 
 sameconst(T) max(sameconst(T) a, sameconst(T) b) {
     return (a > b) ? a : b;
 }
 
 class Foo {
     sameconst(T) value() sameconst(this) { // return value has the same
constness as 'this' (mutable, const or immutable)
         return _value;
     }
 }
 
 The constness is automatically propogated based on parameters. For example,
 - sameconst(T) == T if all the parameters are mutable
 - sameconst(T) == immutable(T) if all the parameters are immutable
 - sameconst(T) == const(T) otherwise

I'll take another look at your proposal. I thought of sameconst as a type alias rather than a typedef. I was mostly thinking of how this problem could be handled by the type system.

Maybe my web searches are failing me, but all I found was your posts in the "equivalent functions" thread. The details there were sparse. In that thread, your first post did not support sameconst(this), but was added a bit later. As far as I can tell, it died with Andrei posting he felf his solution was more general. I hated Andrei's reuse of typeof.
Mar 08 2009
parent reply Kagamin <spam here.lot> writes:
Jason House Wrote:

 Maybe my web searches are failing me, but all I found was your posts in the
"equivalent functions" thread. The details there were sparse. In that thread,
your first post did not support sameconst(this), but was added a bit later. As
far as I can tell, it died with Andrei posting he felf his solution was more
general. I hated Andrei's reuse of typeof.

http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=76920
Mar 09 2009
parent Jason House <jason.james.house gmail.com> writes:
Kagamin Wrote:

 Jason House Wrote:
 
 Maybe my web searches are failing me, but all I found was your posts in the
"equivalent functions" thread. The details there were sparse. In that thread,
your first post did not support sameconst(this), but was added a bit later. As
far as I can tell, it died with Andrei posting he felf his solution was more
general. I hated Andrei's reuse of typeof.

http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=76920

Hmmm... I guess I consistently like the same syntax. The value of my recent proposal was more about how the type system would work under the hood. Still, it appears others beat me to the same conclusion.
Mar 09 2009
prev sibling parent Steve Schveighoffer <schveiguy yahoo.com> writes:
On Sat, 07 Mar 2009 22:44:48 -0500, Jason House wrote:

 The ugly const thread got me thinking about the old problem of returning
 an input while preserving const safety.  I have an idea that seems
 reasonable...

I agree, it's very reasonable. That's why I created an enhancement bugzilla for it last March (almost a year ago) :) http://d.puremagic.com/issues/show_bug.cgi?id=1961 I agree the bikeshed color should not be a sticking point, but I do not think your proposal of re-using return will work. Specifically, return(n) in a function body has a meaning already, it would be ambiguous to the parser probably. I don't really love the inout(T) notation that Janice came up with, but it has an advantage that no new keywords need to be created. I'm ok with another new keyword or some punctuation/existing keyword combo (e.g. const?(T)) But I'm glad we both came up with the same solution, it further gives more validation to the idea. Please vote up my bugzilla report and add comments if you want. -Steve
Mar 08 2009