www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Round VI. Immutable arrays and pointers. Horse was wearing space-suit in fact.

reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
Seems like idea of having only arrays and pointers immutable is still not 
clear ...

Dave wrote:

 Basically, what I (and I think Regan) have in mind is that the compiler 
 would
 enforce 'in' params. the same way a C++ compiler would enforce "const 
 [type]" or
 "const [type] &" params. for analogous types in C++. That way you would 
 get 'C++
 const' functionality for params by default (implicit in params would act 
 like
 Walter's 'explicit in' proposal too).

 The difference would be that in D, although it wouldn't be enforced 
 "deeply" by
 the compiler (because as you've pointed out many times that isn't 
 practical),
 implicit and explicit 'in' would carry a 'gentlemen's agreement' that 
 these are
 immutable so that 'in' could have semantic value.

****************************************************************** ** ** Anyone already can do deep protection ** enforcement in D at any level he/she wants. Now. ** Verified by compiler at compile time in 100% of cases. ** Without "cluttering". ** ****************************************************************** See: class Field { package char[] _name; public char[] name() { return _name; } } 'name' attribute of the Field *will* be protected. You cannot change its value outside of its package. Right? Wrong! char[] s = myfield.name; s[0] = 0; // do you like it? Again, protection is already there. It is not exactly 'const' in C/C++ but it will cover 95% protection cases. You always can implement readonly or other restricting policy on struct/object level. You can build as robust as needed type system *now*! ****************************************************************** ** ** The main problem is that ONLY arrays and pointers ** cannot be protected in D. ** ****************************************************************** (Sorry for such style of emphasis.) Consider this: // somwhere in class RecordsetView .... Field[] fields = myrecordset.fields; // and in galaxy far, far away.... fields.sort; // pretty harmless, isn't it? // and in far future, in other galaxy // on the other side of the Universe // somebody will get big bang in carefully // crafted library. D, you want that? Andrew. http://terrainformatica.com
Jul 14 2005
next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 14 Jul 2005 21:31:30 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 Seems like idea of having only arrays and pointers immutable is still not
 clear ...

 Dave wrote:

 Basically, what I (and I think Regan) have in mind is that the compiler
 would
 enforce 'in' params. the same way a C++ compiler would enforce "const
 [type]" or
 "const [type] &" params. for analogous types in C++. That way you would
 get 'C++
 const' functionality for params by default (implicit in params would act
 like
 Walter's 'explicit in' proposal too).

 The difference would be that in D, although it wouldn't be enforced
 "deeply" by
 the compiler (because as you've pointed out many times that isn't
 practical),
 implicit and explicit 'in' would carry a 'gentlemen's agreement' that
 these are
 immutable so that 'in' could have semantic value.

****************************************************************** ** ** Anyone already can do deep protection ** enforcement in D at any level he/she wants. Now. ** Verified by compiler at compile time in 100% of cases. ** Without "cluttering". ** ****************************************************************** See: class Field { package char[] _name; public char[] name() { return _name; } } 'name' attribute of the Field *will* be protected. You cannot change its value outside of its package. Right? Wrong! char[] s = myfield.name; s[0] = 0; // do you like it? Again, protection is already there. It is not exactly 'const' in C/C++ but it will cover 95% protection cases. You always can implement readonly or other restricting policy on struct/object level. You can build as robust as needed type system *now*! ****************************************************************** ** ** The main problem is that ONLY arrays and pointers ** cannot be protected in D. ** ******************************************************************

IMO making 'in' immutable is part of the solution. A pointer passed as 'in' would be immutable (as would the data it referred to, or possible combinations of this). An array passed as 'in' would be immutable (as would the data to which it referred, or possible combinations of this). The other half of the solution is to have an immutable (or mutable) keyword and allow it to be applied to return values and declared variables. The real problem remains that it's impossible to verify the immutability in 100% of cases without hardware support or runtime overhead. (at least that is the conclusion I have come to based on these threads). That leaves x% verifiable immutability with compile time verification. Another point of view (raised by Walter) is aliasing, an 'in' parameter, supposedly immutable may in fact have a mutable reference in another thread and may in fact change during the function execution. I don't know about you, but I'm not asking for protection from this. I see immutable 'in' not as a contract _to_ the function but rather one _by_ the function. In other words the guarantee is only that function itself _will not_ change the parameter, not that it will not change during function execution. Regan
Jul 14 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opstxuikhn23k2f5 nrage.netwin.co.nz...
 On Thu, 14 Jul 2005 21:31:30 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:
 Seems like idea of having only arrays and pointers immutable is still not
 clear ...

 Dave wrote:

 Basically, what I (and I think Regan) have in mind is that the compiler
 would
 enforce 'in' params. the same way a C++ compiler would enforce "const
 [type]" or
 "const [type] &" params. for analogous types in C++. That way you would
 get 'C++
 const' functionality for params by default (implicit in params would act
 like
 Walter's 'explicit in' proposal too).

 The difference would be that in D, although it wouldn't be enforced
 "deeply" by
 the compiler (because as you've pointed out many times that isn't
 practical),
 implicit and explicit 'in' would carry a 'gentlemen's agreement' that
 these are
 immutable so that 'in' could have semantic value.

****************************************************************** ** ** Anyone already can do deep protection ** enforcement in D at any level he/she wants. Now. ** Verified by compiler at compile time in 100% of cases. ** Without "cluttering". ** ****************************************************************** See: class Field { package char[] _name; public char[] name() { return _name; } } 'name' attribute of the Field *will* be protected. You cannot change its value outside of its package. Right? Wrong! char[] s = myfield.name; s[0] = 0; // do you like it? Again, protection is already there. It is not exactly 'const' in C/C++ but it will cover 95% protection cases. You always can implement readonly or other restricting policy on struct/object level. You can build as robust as needed type system *now*! ****************************************************************** ** ** The main problem is that ONLY arrays and pointers ** cannot be protected in D. ** ******************************************************************

IMO making 'in' immutable is part of the solution. A pointer passed as 'in' would be immutable (as would the data it referred to, or possible combinations of this). An array passed as 'in' would be immutable (as would the data to which it referred, or possible combinations of this). The other half of the solution is to have an immutable (or mutable) keyword and allow it to be applied to return values and declared variables. The real problem remains that it's impossible to verify the immutability in 100% of cases without hardware support or runtime overhead. (at least that is the conclusion I have come to based on these threads). That leaves x% verifiable immutability with compile time verification. Another point of view (raised by Walter) is aliasing, an 'in' parameter, supposedly immutable may in fact have a mutable reference in another thread and may in fact change during the function execution. I don't know about you, but I'm not asking for protection from this. I see immutable 'in' not as a contract _to_ the function but rather one _by_ the function. In other words the guarantee is only that function itself _will not_ change the parameter, not that it will not change during function execution. Regan

Regan, let's try first to do not break what is already working. 'in' is 'in' and let it live alone - it is a parameter passing designator. Not more not less. Second: Here I am getting serious language barrier: "In other words the guarantee is only that function itself _will not_ change the parameter, not that it will not change during function execution." I cannot understand who will guarantee and what? One unknown programmer to other unknown programmer? If yes then let's just rename 'in' into 'i_swear_you' and this is it. Wooohoooo: i_swear_you_not int foo( i_swear_you int bar ) { } D then will be renamed into :D
Jul 14 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 14 Jul 2005 22:23:30 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opstxuikhn23k2f5 nrage.netwin.co.nz...
 On Thu, 14 Jul 2005 21:31:30 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:
 Seems like idea of having only arrays and pointers immutable is still  
 not
 clear ...

 Dave wrote:

 Basically, what I (and I think Regan) have in mind is that the  
 compiler
 would
 enforce 'in' params. the same way a C++ compiler would enforce "const
 [type]" or
 "const [type] &" params. for analogous types in C++. That way you  
 would
 get 'C++
 const' functionality for params by default (implicit in params would  
 act
 like
 Walter's 'explicit in' proposal too).

 The difference would be that in D, although it wouldn't be enforced
 "deeply" by
 the compiler (because as you've pointed out many times that isn't
 practical),
 implicit and explicit 'in' would carry a 'gentlemen's agreement' that
 these are
 immutable so that 'in' could have semantic value.

****************************************************************** ** ** Anyone already can do deep protection ** enforcement in D at any level he/she wants. Now. ** Verified by compiler at compile time in 100% of cases. ** Without "cluttering". ** ****************************************************************** See: class Field { package char[] _name; public char[] name() { return _name; } } 'name' attribute of the Field *will* be protected. You cannot change its value outside of its package. Right? Wrong! char[] s = myfield.name; s[0] = 0; // do you like it? Again, protection is already there. It is not exactly 'const' in C/C++ but it will cover 95% protection cases. You always can implement readonly or other restricting policy on struct/object level. You can build as robust as needed type system *now*! ****************************************************************** ** ** The main problem is that ONLY arrays and pointers ** cannot be protected in D. ** ******************************************************************

IMO making 'in' immutable is part of the solution. A pointer passed as 'in' would be immutable (as would the data it referred to, or possible combinations of this). An array passed as 'in' would be immutable (as would the data to which it referred, or possible combinations of this). The other half of the solution is to have an immutable (or mutable) keyword and allow it to be applied to return values and declared variables. The real problem remains that it's impossible to verify the immutability in 100% of cases without hardware support or runtime overhead. (at least that is the conclusion I have come to based on these threads). That leaves x% verifiable immutability with compile time verification. Another point of view (raised by Walter) is aliasing, an 'in' parameter, supposedly immutable may in fact have a mutable reference in another thread and may in fact change during the function execution. I don't know about you, but I'm not asking for protection from this. I see immutable 'in' not as a contract _to_ the function but rather one _by_ the function. In other words the guarantee is only that function itself _will not_ change the parameter, not that it will not change during function execution. Regan

Regan, let's try first to do not break what is already working. 'in' is 'in' and let it live alone - it is a parameter passing designator. Not more not less.

This is where we disagree. For all the reasons previously mentioned.
 Second: Here I am getting serious language barrier:

 "In other words the guarantee is only that function itself _will
 not_ change the parameter, not that it will not change during function
  execution."

 I cannot understand who will guarantee and what?

Compiler will check function for immutable violations.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it. Regan
Jul 15 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opstx8grmf23k2f5 nrage.netwin.co.nz...
 On Thu, 14 Jul 2005 22:23:30 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opstxuikhn23k2f5 nrage.netwin.co.nz...
 On Thu, 14 Jul 2005 21:31:30 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:
 Seems like idea of having only arrays and pointers immutable is still 
 not
 clear ...

 Dave wrote:

 Basically, what I (and I think Regan) have in mind is that the 
 compiler
 would
 enforce 'in' params. the same way a C++ compiler would enforce "const
 [type]" or
 "const [type] &" params. for analogous types in C++. That way you 
 would
 get 'C++
 const' functionality for params by default (implicit in params would 
 act
 like
 Walter's 'explicit in' proposal too).

 The difference would be that in D, although it wouldn't be enforced
 "deeply" by
 the compiler (because as you've pointed out many times that isn't
 practical),
 implicit and explicit 'in' would carry a 'gentlemen's agreement' that
 these are
 immutable so that 'in' could have semantic value.

****************************************************************** ** ** Anyone already can do deep protection ** enforcement in D at any level he/she wants. Now. ** Verified by compiler at compile time in 100% of cases. ** Without "cluttering". ** ****************************************************************** See: class Field { package char[] _name; public char[] name() { return _name; } } 'name' attribute of the Field *will* be protected. You cannot change its value outside of its package. Right? Wrong! char[] s = myfield.name; s[0] = 0; // do you like it? Again, protection is already there. It is not exactly 'const' in C/C++ but it will cover 95% protection cases. You always can implement readonly or other restricting policy on struct/object level. You can build as robust as needed type system *now*! ****************************************************************** ** ** The main problem is that ONLY arrays and pointers ** cannot be protected in D. ** ******************************************************************

IMO making 'in' immutable is part of the solution. A pointer passed as 'in' would be immutable (as would the data it referred to, or possible combinations of this). An array passed as 'in' would be immutable (as would the data to which it referred, or possible combinations of this). The other half of the solution is to have an immutable (or mutable) keyword and allow it to be applied to return values and declared variables. The real problem remains that it's impossible to verify the immutability in 100% of cases without hardware support or runtime overhead. (at least that is the conclusion I have come to based on these threads). That leaves x% verifiable immutability with compile time verification. Another point of view (raised by Walter) is aliasing, an 'in' parameter, supposedly immutable may in fact have a mutable reference in another thread and may in fact change during the function execution. I don't know about you, but I'm not asking for protection from this. I see immutable 'in' not as a contract _to_ the function but rather one _by_ the function. In other words the guarantee is only that function itself _will not_ change the parameter, not that it will not change during function execution. Regan

Regan, let's try first to do not break what is already working. 'in' is 'in' and let it live alone - it is a parameter passing designator. Not more not less.

This is where we disagree. For all the reasons previously mentioned.
 Second: Here I am getting serious language barrier:

 "In other words the guarantee is only that function itself _will
 not_ change the parameter, not that it will not change during function
  execution."

 I cannot understand who will guarantee and what?

Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly Value[] values() { ... } // may be readonly, may be not } static Field f; bool Foo( in Recordset rs ) { rs.fields[0].foo(); rs.values[0].bar(); f = rs.fields[0]; } Given Foo example above try to describe what needs to be done by compiler to verify immutability of rs parameter.
Jul 15 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
 Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".

I have done so several times, with several different ideas.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly

Then you need to label it as such. Either 'readonly', 'const', etc.
     Value[] values() { ... } // may be readonly, may be not

Not possible, make up your mind.
 }

 static Field f;

 bool Foo( in Recordset rs )
 {
     rs.fields[0].foo();
     rs.values[0].bar();
     f = rs.fields[0];
 }

 Given Foo example above try to describe
 what needs to be done by compiler to
 verify immutability of rs parameter.

The same way C++ const does it. Regan
Jul 15 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opsty7jlu423k2f5 nrage.netwin.co.nz...
 Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".

I have done so several times, with several different ideas.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly

Then you need to label it as such. Either 'readonly', 'const', etc.
     Value[] values() { ... } // may be readonly, may be not

Not possible, make up your mind.
 }

 static Field f;

 bool Foo( in Recordset rs )
 {
     rs.fields[0].foo();
     rs.values[0].bar();
     f = rs.fields[0];
 }

 Given Foo example above try to describe
 what needs to be done by compiler to
 verify immutability of rs parameter.

The same way C++ const does it.

I lost you here.... Are you proposing to mark all immutable methods and references by const/readonly as in C++?
Jul 15 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 15 Jul 2005 15:47:42 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opsty7jlu423k2f5 nrage.netwin.co.nz...
 Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".

I have done so several times, with several different ideas.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly

Then you need to label it as such. Either 'readonly', 'const', etc.
     Value[] values() { ... } // may be readonly, may be not

Not possible, make up your mind.
 }

 static Field f;

 bool Foo( in Recordset rs )
 {
     rs.fields[0].foo();
     rs.values[0].bar();
     f = rs.fields[0];
 }

 Given Foo example above try to describe
 what needs to be done by compiler to
 verify immutability of rs parameter.

The same way C++ const does it.

I lost you here.... Are you proposing to mark all immutable methods and references by const/readonly as in C++?

Yes. Have you _read_ any of the proposals I've made? Regan
Jul 15 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Sat, 16 Jul 2005 18:31:08 +1200, Regan Heath <regan netwin.co.nz> wrote:
 On Fri, 15 Jul 2005 15:47:42 -0700, Andrew Fedoniouk  
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opsty7jlu423k2f5 nrage.netwin.co.nz...
 Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".

I have done so several times, with several different ideas.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly

Then you need to label it as such. Either 'readonly', 'const', etc.
     Value[] values() { ... } // may be readonly, may be not

Not possible, make up your mind.
 }

 static Field f;

 bool Foo( in Recordset rs )
 {
     rs.fields[0].foo();
     rs.values[0].bar();
     f = rs.fields[0];
 }

 Given Foo example above try to describe
 what needs to be done by compiler to
 verify immutability of rs parameter.

The same way C++ const does it.

I lost you here.... Are you proposing to mark all immutable methods and references by const/readonly as in C++?

Yes. Have you _read_ any of the proposals I've made?

Before you suggest this is the "clutter" Walter & Co have referred to, it's not the same as C++ const as more sensible defaults can be chosen (because we're in beta stage of development), eg: in parameters: immutable return values: mutable local variables: mutable So where code does not deviate from the above no keyword/clutter is required. So far none of my proposals have attempted to address class members which do/don't mutate. As you've said this is mostly covered with private, protected, etc and get/set methods (assuming immutable arrays and pointers are implemented) "One mans clutter is another mans design contract" Regan
Jul 15 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opstztp6iy23k2f5 nrage.netwin.co.nz...
 On Fri, 15 Jul 2005 15:47:42 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opsty7jlu423k2f5 nrage.netwin.co.nz...
 Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".

I have done so several times, with several different ideas.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly

Then you need to label it as such. Either 'readonly', 'const', etc.
     Value[] values() { ... } // may be readonly, may be not

Not possible, make up your mind.
 }

 static Field f;

 bool Foo( in Recordset rs )
 {
     rs.fields[0].foo();
     rs.values[0].bar();
     f = rs.fields[0];
 }

 Given Foo example above try to describe
 what needs to be done by compiler to
 verify immutability of rs parameter.

The same way C++ const does it.

I lost you here.... Are you proposing to mark all immutable methods and references by const/readonly as in C++?

Yes. Have you _read_ any of the proposals I've made?

Yes I did. I am reading everything related to const/readonly/immutable . So, am I right interpretting you this way: Proposal is to add the same system of const/mutable tags which C++ has now - const/mutable tags for variables and class methods. Main difference is that we will make explicit 'in typename varname' modifier behave as ' in const typename varname'. Am I correct?
Jul 15 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 15 Jul 2005 23:54:29 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opstztp6iy23k2f5 nrage.netwin.co.nz...
 On Fri, 15 Jul 2005 15:47:42 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opsty7jlu423k2f5 nrage.netwin.co.nz...
 Compiler will check function for immutable violations.

It is easy to say, try to describe such "check".

I have done so several times, with several different ideas.
 One unknown programmer to other unknown programmer?

The programmer of the function will guarantee it, the compiler will enfore it.

How? class Recordset { Field[] fields() { ... } // supposed to be readonly

Then you need to label it as such. Either 'readonly', 'const', etc.
     Value[] values() { ... } // may be readonly, may be not

Not possible, make up your mind.
 }

 static Field f;

 bool Foo( in Recordset rs )
 {
     rs.fields[0].foo();
     rs.values[0].bar();
     f = rs.fields[0];
 }

 Given Foo example above try to describe
 what needs to be done by compiler to
 verify immutability of rs parameter.

The same way C++ const does it.

I lost you here.... Are you proposing to mark all immutable methods and references by const/readonly as in C++?

Yes. Have you _read_ any of the proposals I've made?

Yes I did. I am reading everything related to const/readonly/immutable . So, am I right interpretting you this way: Proposal is to add the same system of const/mutable tags which C++ has now

No. I would use different defaults, specifically 'in' parameters.
 - const/mutable tags for variables

Yes, but note local variables default to being mutable, in parameters to immutable, return values to mutable. So in the common cases no tag is required.
 and class methods.

Not methods themselves, just return values. Access protection private, protected, etc handles the rest.. or so you keep saying.
 Main difference is that we will make explicit 'in typename varname'
 modifier behave as ' in const typename varname'.

No. Implicit 'in' should be readonly IMO. Regan
Jul 16 2005
prev sibling next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

 // and in far future, in other galaxy
 // on the other side of the Universe
 // somebody will get big bang in carefully
 // crafted library.

 D, you want that?

I'm less worried about this happening in D than in other languages since D emphasizes COW so strongly. Let me make an analogy with the law that in the US cars should drive on the right side of the road. Now one can drive a car on either side and there are odd cases where it does happen and sometimes an accident results. But usually it doesn't. That situation is better than a country where there is no law about what side to drive on and so every driver needs to be extremely careful about what's coming at them. Would the US have fewer car accidents if it were impossible to drive on the wrong side? Yes. Would it be a significant difference? Probably not. That analogy isn't perfect since so many users of D will come from C++ or Java they are used to driving on the wrong side of the road (so to speak) and they don't know the rules of the road in D. To me that's the strongest rationale for immutability (for catching errors not for optimizations) in D: protection from people who don't know the rules of the road in D.
Jul 15 2005
next sibling parent reply Dave <Dave_member pathlink.com> writes:
In article <db8apd$269e$1 digitaldaemon.com>, Ben Hinkle says...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

 // and in far future, in other galaxy
 // on the other side of the Universe
 // somebody will get big bang in carefully
 // crafted library.

 D, you want that?

I'm less worried about this happening in D than in other languages since D emphasizes COW so strongly. Let me make an analogy with the law that in the US cars should drive on the right side of the road. Now one can drive a car on either side and there are odd cases where it does happen and sometimes an accident results. But usually it doesn't. That situation is better than a country where there is no law about what side to drive on and so every driver needs to be extremely careful about what's coming at them. Would the US have fewer car accidents if it were impossible to drive on the wrong side? Yes. Would it be a significant difference? Probably not. That analogy isn't perfect since so many users of D will come from C++ or Java they are used to driving on the wrong side of the road (so to speak) and they don't know the rules of the road in D. To me that's the strongest rationale for immutability (for catching errors not for optimizations) in D: protection from people who don't know the rules of the road in D.

I'm with you - it's not only the optimizations, it's consistency, clarity and ease-of-use that I think will benefit here. There were adjustments that C/C++ users had to make to move to Java w.r.t. parameter passing and I don't think it will be too much to ask for new D users as well. Especially since D doesn't have 'const' and '&' to confuse things. And in the long run will make things easier once one is accustomed to the way things should be done, because, again, it's more consistent. To an earlier statement about "in" being just 'a parameter passing designator', as such it clearly (in my mind at least) should enforce that expectation (at least to a pratical degree) consistently regardless of if the type being passed lives on the stack or the heap. in/out/inout are supposed to be analogous to the IDL. In the IDL, a change made to an "in" referant should not live past the function call. For example, in DCOM at least, changes to an "in" referant are simply not propogated back to the callee (and no, I'm not suggesting that kind of overhead here). Since there is no de/serialization barrier for D function calls, that barrier should be what the compiler allows to be consistent with the IDL idea. The way C++ "const" parameters are treated by the compiler I think is a reasonable demarkation as to what should be enforced for 'in' parameters in D. - Dave
Jul 15 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 To me that's the strongest
rationale for immutability (for catching errors not for optimizations) in 
D:
protection from people who don't know the rules of the road in D.

I'm with you - it's not only the optimizations, it's consistency, clarity and ease-of-use that I think will benefit here.

To be clear when I said "(for catching errors not for optimizations)" I meant "the rationale for adding immutability for catching errors as opposed to the rationale for adding immutability optimizations". I didn't mean that the catching errors rationale is stronger than the optimizations rationale or any other rationale. I just meant that the rationale for catching errors will in practice only catch people who don't know D's rules of the road.
Jul 15 2005
parent reply Dave <Dave_member pathlink.com> writes:
In article <db8i4p$2bhm$1 digitaldaemon.com>, Ben Hinkle says...
 To me that's the strongest
rationale for immutability (for catching errors not for optimizations) in 
D:
protection from people who don't know the rules of the road in D.

I'm with you - it's not only the optimizations, it's consistency, clarity and ease-of-use that I think will benefit here.

To be clear when I said "(for catching errors not for optimizations)" I meant "the rationale for adding immutability for catching errors as opposed to the rationale for adding immutability optimizations". I didn't mean that the catching errors rationale is stronger than the optimizations rationale or any other rationale. I just meant that the rationale for catching errors will in practice only catch people who don't know D's rules of the road.

For the record, they are about equal in weight for me but I could pretty easily be persuaded that the catching errors part is more important.. Even if optimization opportunities are not presented. What I've been blathering on about regarding making 'in' like 'const' still makes enough sense to me to make it a worthwhile change IMHO. Another point on the potential optimizations that may be a hangup to this 'in' as 'const' idea (not directed at you Ben, just more thoughts on the matter). An optimization should not be done unless it is safe. Safe to me means that both the unoptimized and optimized versions will always produce a correct result given the same input if the rules of the language are followed. In this case (with the proposed new rules for 'in') the result could be wrong whether or not an optimization is performed if the rules aren't followed, but either way will be correct 100% of the time if the rules are followed. That doesn't mean the compiler has to strictly enforce those rules, and it doesn't mean that just because the D optimizer produces a different result than a C optimizer for what *looks like* the same code, that the D optimizer is wrong - it's just operating under a different set of rules. D is *not* C ;) COW for example is a 'rule' of D that allows optimizations, but the rule is not enforced nor is COW done automagically by the compiler or runtime. If the rules of COW aren't followed, then an incorrect result may happen regardless of compiler optimizations. It's up to the programmer to follow the rules and if the compiler gives them help in that by catching transgressions to rules like COW and 'in', more power.
Jul 15 2005
parent reply David Medlock <noone nowhere.com> writes:
Dave wrote:

 In article <db8i4p$2bhm$1 digitaldaemon.com>, Ben Hinkle says...
 
To me that's the strongest
rationale for immutability (for catching errors not for optimizations) in 
D:
protection from people who don't know the rules of the road in D.

I'm with you - it's not only the optimizations, it's consistency, clarity and ease-of-use that I think will benefit here.

To be clear when I said "(for catching errors not for optimizations)" I meant "the rationale for adding immutability for catching errors as opposed to the rationale for adding immutability optimizations". I didn't mean that the catching errors rationale is stronger than the optimizations rationale or any other rationale. I just meant that the rationale for catching errors will in practice only catch people who don't know D's rules of the road.

For the record, they are about equal in weight for me but I could pretty easily be persuaded that the catching errors part is more important.. Even if optimization opportunities are not presented. What I've been blathering on about regarding making 'in' like 'const' still makes enough sense to me to make it a worthwhile change IMHO. Another point on the potential optimizations that may be a hangup to this 'in' as 'const' idea (not directed at you Ben, just more thoughts on the matter). An optimization should not be done unless it is safe. Safe to me means that both the unoptimized and optimized versions will always produce a correct result given the same input if the rules of the language are followed. In this case (with the proposed new rules for 'in') the result could be wrong whether or not an optimization is performed if the rules aren't followed, but either way will be correct 100% of the time if the rules are followed. That doesn't mean the compiler has to strictly enforce those rules, and it doesn't mean that just because the D optimizer produces a different result than a C optimizer for what *looks like* the same code, that the D optimizer is wrong - it's just operating under a different set of rules. D is *not* C ;) COW for example is a 'rule' of D that allows optimizations, but the rule is not enforced nor is COW done automagically by the compiler or runtime. If the rules of COW aren't followed, then an incorrect result may happen regardless of compiler optimizations. It's up to the programmer to follow the rules and if the compiler gives them help in that by catching transgressions to rules like COW and 'in', more power.

How about just an expression like the following( weird off the top of my head, look at the semantics, not syntax): char[] a = "Some valuable data"; const( a ) { func1(a); func2(a); } Then the compiler could do an analysis of the functions which are called in the const-block and warn if functions called within attempt to change the data. Feasible? -DavidM PS. Curious as to what instances do you think const presents optimizations?
Jul 15 2005
parent reply Dave <Dave_member pathlink.com> writes:
In article <db8ocu$2gt5$1 digitaldaemon.com>, David Medlock says...
How about just an expression like the following( weird off the top of my 
head, look at the semantics, not syntax):


char[] a = "Some valuable data";
const( a )
{
   func1(a);
   func2(a);
}

Then the compiler could do an analysis of the functions which are called 
in the const-block and warn if functions called within attempt to change 
the data.

Feasible?

I think that is as feasible as doing the 'const' check on an 'in' parameter. The problem in both cases is how 'deep' do you make the check.
-DavidM

PS.  Curious as to what instances do you think const presents 
optimizations?

The compiler could pass "value" in params by reference. For example a large struct is passed alot faster by reference. Also, with the 'immutable' rule, the compiler could then safely load an 'in' param. passed by reference into a register and not worry about another reference to the same data in the same function modifying it (and not storing the change back to memory in the meantime). If the in param is used in a tight loop that alone will make a big difference, but not loading something into a register probably precludes other optimizations as well. Right now this problem keeps optimizations from being done in not only D but C, C++, Java and many others for variables that are never even referenced, but since their *could be* another reference, the optimization(s) can't be done. This is what I'm assuming and there may be more - Walter would be the one to answer that the best I think. - Dave
Jul 15 2005
parent reply David Medlock <noone nowhere.com> writes:
Dave wrote:

 In article <db8ocu$2gt5$1 digitaldaemon.com>, David Medlock says...
 
How about just an expression like the following( weird off the top of my 
head, look at the semantics, not syntax):


char[] a = "Some valuable data";
const( a )
{
  func1(a);
  func2(a);
}

Then the compiler could do an analysis of the functions which are called 
in the const-block and warn if functions called within attempt to change 
the data.

Feasible?

I think that is as feasible as doing the 'const' check on an 'in' parameter. The problem in both cases is how 'deep' do you make the check.

Yes but this (or a similar concept) wouldn't change existing pass-by-value semantics. The depth cannot reasonably be deeper than in C++, imo.
 
-DavidM

PS.  Curious as to what instances do you think const presents 
optimizations?

The compiler could pass "value" in params by reference. For example a large struct is passed alot faster by reference.

The issue there is something like this: struct A{ int n }; void func( in A a, inout A a2 ) { .. makes several changes to a2 based on a } A temp; func( temp, temp ); This currently works in D, but would not if passed by reference. This is simply one case, there are probably others.
 Also, with the 'immutable' rule, the compiler could then safely load an 'in'
 param. passed by reference into a register and not worry about another
reference
 to the same data in the same function modifying it (and not storing the change
 back to memory in the meantime). 

I would bet this type of register optimization is already done by DMD (unless the variable is synchronized). If so, I would be very surprised if you got even a 5% speedup from adding 'const'. You also have to be wary of passing pointers(references) all over the place. Intel processors cache-misses are quite expensive, whereas structs are on the stack and are therefore in cache already. -DavidM
Jul 15 2005
parent reply Dave <Dave_member pathlink.com> writes:
In article <db8slu$2mfv$1 digitaldaemon.com>, David Medlock says...
Dave wrote:

 In article <db8ocu$2gt5$1 digitaldaemon.com>, David Medlock says...
 
How about just an expression like the following( weird off the top of my 
head, look at the semantics, not syntax):


char[] a = "Some valuable data";
const( a )
{
  func1(a);
  func2(a);
}

Then the compiler could do an analysis of the functions which are called 
in the const-block and warn if functions called within attempt to change 
the data.

Feasible?

I think that is as feasible as doing the 'const' check on an 'in' parameter. The problem in both cases is how 'deep' do you make the check.

Yes but this (or a similar concept) wouldn't change existing pass-by-value semantics. The depth cannot reasonably be deeper than in C++, imo.
 
-DavidM

PS.  Curious as to what instances do you think const presents 
optimizations?

The compiler could pass "value" in params by reference. For example a large struct is passed alot faster by reference.

The issue there is something like this: struct A{ int n }; void func( in A a, inout A a2 ) { .. makes several changes to a2 based on a } A temp; func( temp, temp );

That would be part of the immutable rule because in effect temp will be modified after the function returns, which would break the 'immutability contract' whether or not it's passed byval or byref. Let's say func was a template and instead of passing a struct a user of your library passes an instance of a class; in that case everything would get hosed up and it's inconsistent (templates are supposed to be 'generic'). So, a better solution (I think) regardless of the optimizations would be to make 'in' mean the same thing for structs and classes. I can see where your example is a case not covered by the 'const' checks made for C++; not sure what the solution would be other than to add the check for simple cases like that, but there is the slippery slope and one of the reasons why the compiler can't (practically, as mentioned elsewhere) guarantee immutability.
This currently works in D, but would not if passed by reference.  This 
is simply one case, there are probably others.

 Also, with the 'immutable' rule, the compiler could then safely load an 'in'
 param. passed by reference into a register and not worry about another
reference
 to the same data in the same function modifying it (and not storing the change
 back to memory in the meantime). 

I would bet this type of register optimization is already done by DMD (unless the variable is synchronized). If so, I would be very surprised if you got even a 5% speedup from adding 'const'. You also have to be

For value types, yes. But what about for reference types?? With the 'immutable in' idea, the optimization could be done for byref types and value types could then still be passed byref into functions and also use the register optimizations.
wary of passing pointers(references) all over the place. Intel 
processors cache-misses are quite expensive, whereas structs are on the 
stack and are therefore in cache already.

In my experience, it's generally way more expensive to be repeatedly making value type copies than the potential cache misses caused by passing things like large structs byref; with immutable in, the compiler would make the decision on whether or not to pass value types byref or byval (I base this on not only my own code, but how often I see 'const [type] &' is used for even small structs in C++ code and how often pointers are used to pass structs in C code).
-DavidM

Jul 15 2005
parent reply David Medlock <noone nowhere.com> writes:
Dave wrote:
 In article <db8slu$2mfv$1 digitaldaemon.com>, David Medlock says...
 
Dave wrote:


In article <db8ocu$2gt5$1 digitaldaemon.com>, David Medlock says...


How about just an expression like the following( weird off the top of my 
head, look at the semantics, not syntax):


char[] a = "Some valuable data";
const( a )
{
 func1(a);
 func2(a);
}

Then the compiler could do an analysis of the functions which are called 
in the const-block and warn if functions called within attempt to change 
the data.

Feasible?

I think that is as feasible as doing the 'const' check on an 'in' parameter. The problem in both cases is how 'deep' do you make the check.

Yes but this (or a similar concept) wouldn't change existing pass-by-value semantics. The depth cannot reasonably be deeper than in C++, imo.
-DavidM

PS.  Curious as to what instances do you think const presents 
optimizations?

The compiler could pass "value" in params by reference. For example a large struct is passed alot faster by reference.

The issue there is something like this: struct A{ int n }; void func( in A a, inout A a2 ) { .. makes several changes to a2 based on a } A temp; func( temp, temp );

That would be part of the immutable rule because in effect temp will be modified after the function returns, which would break the 'immutability contract' whether or not it's passed byval or byref.

possible. You aren't suggesting that are you? -DavidM
Jul 15 2005
next sibling parent Dave <Dave_member pathlink.com> writes:
In article <db95ut$2u1i$1 digitaldaemon.com>, David Medlock says...
Dave wrote:
 In article <db8slu$2mfv$1 digitaldaemon.com>, David Medlock says...
 
Dave wrote:


In article <db8ocu$2gt5$1 digitaldaemon.com>, David Medlock says...


How about just an expression like the following( weird off the top of my 
head, look at the semantics, not syntax):


char[] a = "Some valuable data";
const( a )
{
 func1(a);
 func2(a);
}

Then the compiler could do an analysis of the functions which are called 
in the const-block and warn if functions called within attempt to change 
the data.

Feasible?

I think that is as feasible as doing the 'const' check on an 'in' parameter. The problem in both cases is how 'deep' do you make the check.

Yes but this (or a similar concept) wouldn't change existing pass-by-value semantics. The depth cannot reasonably be deeper than in C++, imo.
-DavidM

PS.  Curious as to what instances do you think const presents 
optimizations?

The compiler could pass "value" in params by reference. For example a large struct is passed alot faster by reference.

The issue there is something like this: struct A{ int n }; void func( in A a, inout A a2 ) { .. makes several changes to a2 based on a } A temp; func( temp, temp );

That would be part of the immutable rule because in effect temp will be modified after the function returns, which would break the 'immutability contract' whether or not it's passed byval or byref.

possible. You aren't suggesting that are you? -DavidM

No - compiler decides. The reason the optimization isn't done on byref params in C or C++ is not because it can't be done, but because it might produce unexpected results if someone happens to write code like your example (among others). It's not a question of computer science, it's a question of the rules of the language. So, if 'in' is changed to to be 'immutable' then cases like your example would not be legal, so the optimization could then be done on params. passed by reference. A lot of Fortran compilers already do these optimizations on parameters passed byref. In a case like your example, the rule in Fortran is basically "don't do that" but it is not required to be enforced by Fortran compilers. What this thread has been discussing lately is about changing the rules for D 'in' params. to make them 'immutable' inside the function and enforcing that somehow. A side-effect of having these 'readonly' params. would be the chance to legally use these Fortran-like optimizations in D on anything passed by-whatever. The goal would be to forget worrying about how the compiler chooses to pass an 'in' parameter w/o creating a bunch of hard-to-find bugs.
Jul 15 2005
prev sibling parent AJG <AJG_member pathlink.com> writes:
Hi,

 That would be part of the immutable rule because in effect temp will be
modified
 after the function returns, which would break the 'immutability contract'
 whether or not it's passed byval or byref.
 

possible. You aren't suggesting that are you?

Just out of curiosity, what if this was actually done? Does it cause some sort of great evil? Some assumptions: 1) Most parameters are "in," therefore semantically readonly. I think this is one of the main points of the whole const/immutable debate. 2) Passing by reference is more efficient for everything except maybe int. The vast majority of C code I've seen passes structs by pointer, even if just something like struct Point { int X, Y; }. So then, what if all parameters are passed by reference, with the additional requirement that they be made readonly. What are the problems with this (not saying there aren't)? Plus, there is the added benefit of that optimization you mentioned, so it would be even more efficient. --AJG.
Jul 15 2005
prev sibling parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:db8apd$269e$1 digitaldaemon.com...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

 // and in far future, in other galaxy
 // on the other side of the Universe
 // somebody will get big bang in carefully
 // crafted library.

 D, you want that?

I'm less worried about this happening in D than in other languages since D emphasizes COW so strongly.

Seems like one more language barrier here. D as a language does not emphasize anything about COW, not strongly not weakly. Dot. And this is a problem. D documentation *recommends* to use COW.
Jul 15 2005
prev sibling next sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
I would like to return to basics again.

I. Readonly variable.

What is readonly parameter or variable?

readonly (c++::const) something means that in given scope
any write-over operations are disabled for such variable.

*given scope* - means not only function body but also
the whole application domain if e.g. such variable is static
or member field of some class. Generally speaking
such variable has type having operator= and the like
disabled.

readonly variable does not mean that variable will not be changed.
Such variable is an alias of some memory location - it just means that
using this particular alias it is imposible to overwrite that memory
location.

II. Immutable variable.

What is 'immutable' or 'immutable in' then?
Abstract meaning of immutable variable - snapshot of data under
this variable. Immutability of function parameter is a gurantee
that in the scope of this function any two consequent readings
of any data location reachable from this var will lead to the same
result.  This apply to function parameters and other variables e.g.
member field of some class.

III. 'in' parameter in current version.

Currently 'in' parameter is a copy (shallow snapshot) of some
value passed to the function. 'in' parameter variable is an alias
of stack location. By now it is *mutable* variable.

IV. readonly arrays and pointers. Implementation.

My initial proposal is not about immutability. It is about
readonly versions of two primitive reference types.
Not more not less. Implementation of this types
is simple, feasible and verifiable by compiler 100% of cases
theoretically and practically. Implementation of such two
types is pretty much mechanical process similar to modification
of static type table.

V. Implementation of 'in immutability'.

Let's take a look on possible immutability implementations:
We are considering only non-intrusive implementations here,
i.e. impementations without readonly/const tags of variables
and methods. As such tags are known to produce
"clutter", (c) Walter and others.

  V.1. Snapshot.
         Snapshot as a process, also known as 'marshalling',
         used in RPC implementations (e.g. in COM/CORBA)
         to pass parameters across process/machine
         boundaries. Demands synchronized access to passing
         value and all values reachable from it.
         Practically is not feasible to be short.

  V.2. GC-like scan against passing variable.
         Don't need too much comments here.
         In fact V.1. is a form of GC too.
         Copying GC this time.

Here is the first question: So, how is
this beaitiful "immutable in" supposed to
be implemented?

And here is the second question:
Can anybody provide *realistic* example of some
practical task where such "immutable in" is the must?
I will greatly appreciate if this example will not
be reasonably reproducible with readonly arrays/pointers
and use of access attributes for other types.

---------------------------
Andrew Fedoniouk.
http://terrainformatica.com
Jul 15 2005
next sibling parent reply AJG <AJG_member pathlink.com> writes:
Hi,

How about this:

Make 'explicit' in parameters readonly.
Make naked parameters copies ("shallow snapshots"). (As they are, currently).
Offer an "immutable" block statement for those that need it.

immutable (someVar, someOtherVar, ...) {
// I desperately need someVar and 
// someOtherVar to be immutable here.
}

For the most part, parameters need only be readonly, not actually immutable.
Immutability (for all parameters) via runtime checking is insane. I think we've
established that. It is OK, however, for small blocks, if required.

So, unless someone posts some magical cheap way to fuse readonly and immutable,
I think we are stuck. Time to make some compromises. The reason C++ has its
const is probably because that's the only feasible way. I would, however, _like_
to be proven wrong.

Cheers,
--AJG.

PS: What about the idea about making all parameters by reference. Would this
allow for optimization with compile-time verification of immutability? Would
this make _anything_ easier?

In article <dba1d0$mu7$1 digitaldaemon.com>, Andrew Fedoniouk says...
I would like to return to basics again.

I. Readonly variable.

What is readonly parameter or variable?

readonly (c++::const) something means that in given scope
any write-over operations are disabled for such variable.

*given scope* - means not only function body but also
the whole application domain if e.g. such variable is static
or member field of some class. Generally speaking
such variable has type having operator= and the like
disabled.

readonly variable does not mean that variable will not be changed.
Such variable is an alias of some memory location - it just means that
using this particular alias it is imposible to overwrite that memory
location.

II. Immutable variable.

What is 'immutable' or 'immutable in' then?
Abstract meaning of immutable variable - snapshot of data under
this variable. Immutability of function parameter is a gurantee
that in the scope of this function any two consequent readings
of any data location reachable from this var will lead to the same
result.  This apply to function parameters and other variables e.g.
member field of some class.

III. 'in' parameter in current version.

Currently 'in' parameter is a copy (shallow snapshot) of some
value passed to the function. 'in' parameter variable is an alias
of stack location. By now it is *mutable* variable.

IV. readonly arrays and pointers. Implementation.

My initial proposal is not about immutability. It is about
readonly versions of two primitive reference types.
Not more not less. Implementation of this types
is simple, feasible and verifiable by compiler 100% of cases
theoretically and practically. Implementation of such two
types is pretty much mechanical process similar to modification
of static type table.

V. Implementation of 'in immutability'.

Let's take a look on possible immutability implementations:
We are considering only non-intrusive implementations here,
i.e. impementations without readonly/const tags of variables
and methods. As such tags are known to produce
"clutter", (c) Walter and others.

  V.1. Snapshot.
         Snapshot as a process, also known as 'marshalling',
         used in RPC implementations (e.g. in COM/CORBA)
         to pass parameters across process/machine
         boundaries. Demands synchronized access to passing
         value and all values reachable from it.
         Practically is not feasible to be short.

  V.2. GC-like scan against passing variable.
         Don't need too much comments here.
         In fact V.1. is a form of GC too.
         Copying GC this time.

Here is the first question: So, how is
this beaitiful "immutable in" supposed to
be implemented?

And here is the second question:
Can anybody provide *realistic* example of some
practical task where such "immutable in" is the must?
I will greatly appreciate if this example will not
be reasonably reproducible with readonly arrays/pointers
and use of access attributes for other types.

---------------------------
Andrew Fedoniouk.
http://terrainformatica.com

Jul 15 2005
next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"AJG" <AJG_member pathlink.com> wrote in message
news:dba3e0$oqq$1 digitaldaemon.com...
 PS: What about the idea about making all parameters by reference. Would

 allow for optimization with compile-time verification of immutability?

 this make _anything_ easier?

It'll make things inefficient on modern CPU's, because of cache misses when the indirection is done.
Jul 15 2005
next sibling parent reply AJG <AJG_member pathlink.com> writes:
Hi,

In article <dba7gh$117s$1 digitaldaemon.com>, Walter says...
"AJG" <AJG_member pathlink.com> wrote in message
news:dba3e0$oqq$1 digitaldaemon.com...
 PS: What about the idea about making all parameters by reference. Would

 allow for optimization with compile-time verification of immutability?

 this make _anything_ easier?

It'll make things inefficient on modern CPU's, because of cache misses when the indirection is done.

Can I assume this happens regardless of whether it's a reference or a pointer? So then what's the "sweet spot" in this tradeoff? Where does it become more efficient to pass by reference rather than by value? Can whole structs be cached? Thanks, --AJG.
Jul 15 2005
parent "Walter" <newshound digitalmars.com> writes:
"AJG" <AJG_member pathlink.com> wrote in message
news:dba86h$11nc$1 digitaldaemon.com...
 In article <dba7gh$117s$1 digitaldaemon.com>, Walter says...
"AJG" <AJG_member pathlink.com> wrote in message
news:dba3e0$oqq$1 digitaldaemon.com...
 PS: What about the idea about making all parameters by reference. Would

 allow for optimization with compile-time verification of immutability?

 this make _anything_ easier?

It'll make things inefficient on modern CPU's, because of cache misses


the indirection is done.

Can I assume this happens regardless of whether it's a reference or a

References are just syntactic sugar for pointers, so yes.
 So then what's the "sweet spot" in this tradeoff? Where does it become

 efficient to pass by reference rather than by value?

That can only be determined on a case by case basis using careful profiling. Results also vary depending on which CPU model you're using.
 Can whole structs be cached?

Yes.
Jul 16 2005
prev sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 15 Jul 2005 22:54:22 -0700, Walter <newshound digitalmars.com>  
wrote:
 "AJG" <AJG_member pathlink.com> wrote in message
 news:dba3e0$oqq$1 digitaldaemon.com...
 PS: What about the idea about making all parameters by reference. Would

 allow for optimization with compile-time verification of immutability?

 this make _anything_ easier?

It'll make things inefficient on modern CPU's, because of cache misses when the indirection is done.

The suggestion was not to make 'all' parameters pass by reference but rather to allow the compiler to choose the most efficient in any given situation. This would only be used in situations where you don't care how it's passed, so long as it's efficient. Specifically suggested were 'in' parameters. Unlike 'out' or 'inout' passing by reference is not specified/required. (in fact it's not required for them either, but would be inefficient if implemented with copies). This would allow us to pass large structs without resorting to using pointers. There is a requirement then that an 'in' parameter passed by reference is not be mutable. Some sort of readonly enforcement would need to be used to ensure this. Regan
Jul 15 2005
parent "Walter" <newshound digitalmars.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message
news:opstzt8tg823k2f5 nrage.netwin.co.nz...
 The suggestion was not to make 'all' parameters pass by reference but
 rather to allow the compiler to choose the most efficient in any given
 situation. This would only be used in situations where you don't care how
 it's passed, so long as it's efficient.

 Specifically suggested were 'in' parameters. Unlike 'out' or 'inout'
 passing by reference is not specified/required. (in fact it's not required
 for them either, but would be inefficient if implemented with copies).

 This would allow us to pass large structs without resorting to using
 pointers.

 There is a requirement then that an 'in' parameter passed by reference is
 not be mutable. Some sort of readonly enforcement would need to be used to
 ensure this.

Ok, I understand. And you're right.
Jul 16 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"AJG" <AJG_member pathlink.com> wrote in message 
news:dba3e0$oqq$1 digitaldaemon.com...
 Hi,

 How about this:

 Make 'explicit' in parameters readonly.

In given function: void emitXML( in XMLNode root ) { ... } what this 'readonly' means exactly?
 Make naked parameters copies ("shallow snapshots"). (As they are, 
 currently).

Ok. And?
 Offer an "immutable" block statement for those that need it.

 immutable (someVar, someOtherVar, ...) {
 // I desperately need someVar and
 // someOtherVar to be immutable here.
 }

XMLNode root; immutable (root) { .... } What it means exactly? Snapshot of the whole tree with memory region lock?
 For the most part, parameters need only be readonly, not actually 
 immutable.
 Immutability (for all parameters) via runtime checking is insane. I think 
 we've
 established that. It is OK, however, for small blocks, if required.

 So, unless someone posts some magical cheap way to fuse readonly and 
 immutable,
 I think we are stuck. Time to make some compromises. The reason C++ has 
 its
 const is probably because that's the only feasible way. I would, however, 
 _like_
 to be proven wrong.

I've already published here article about possible referential immutability implementation. It uses the same principles as C++ const, readonly/mutable tags versus const/mutable. Combined with Java absence of slices and pointers plus some fixes at VM level (runtime checks) it makes referential (only) immutability practically feasible at compile time. Referential immutability, I believe, is exactly "brute force deep readonlyness" - only calls of readonly methods are allowed against given variable and all objects *potentially* accessible from it. Theoretically it is possible to deduce value of readonly tag for any given method of the class. But this practically also needs 'mutable' keyword at least.
 Cheers,
 --AJG.

 PS: What about the idea about making all parameters by reference. Would 
 this
 allow for optimization with compile-time verification of immutability? 
 Would
 this make _anything_ easier?

I cannot see how "all parameters by reference" differ from current situation in D. "compile-time verification of immutability" of any given variable is not practically possible. Again, if it would be possible then GC will not be needed as compiler can build list of free blocks in compile time. Problem of the same nature to be short. Andrew.
Jul 15 2005
parent AJG <AJG_member pathlink.com> writes:
Hi,

 Make 'explicit' in parameters readonly.

In given function: void emitXML( in XMLNode root ) { ... } what this 'readonly' means exactly?

This is readonly from your own definitions (i.e. "I. Readonly").
 Make naked parameters copies ("shallow snapshots"). (As they are, 
 currently).

Ok. And?

This is for when you want to be able to modify the parameter at the function (for convinience), but you don't want the changes to propagate to the caller. void Foo(int A) { A += 10; return (A); } // OK: A is a copy. void Foo(in int B) { B += 10; return (B); } // ERROR: A is the original, but readonly.
 Offer an "immutable" block statement for those that need it.

 immutable (someVar, someOtherVar, ...) {
 // I desperately need someVar and
 // someOtherVar to be immutable here.
 }

XMLNode root; immutable (root) { .... } What it means exactly? Snapshot of the whole tree with memory region lock?

It means immutability checked at runtime. There have been various proposals to implement this, all fairly expensive, if I recall correctly. One method is to serialize all vars at the beginning, and at the end, and compare if they are the same. I guess a checksum could also work. The point is, it can be slow, because it wouldn't happen a lot.
 For the most part, parameters need only be readonly, not actually 
 immutable.
 Immutability (for all parameters) via runtime checking is insane. I think 
 we've
 established that. It is OK, however, for small blocks, if required.

 So, unless someone posts some magical cheap way to fuse readonly and 
 immutable,
 I think we are stuck. Time to make some compromises. The reason C++ has 
 its
 const is probably because that's the only feasible way. I would, however, 
 _like_
 to be proven wrong.

I've already published here article about possible referential immutability implementation. It uses the same principles as C++ const, readonly/mutable tags versus const/mutable. Combined with Java absence of slices and pointers plus some fixes at VM level (runtime checks) it makes referential (only) immutability practically feasible at compile time. Referential immutability, I believe, is exactly "brute force deep readonlyness" - only calls of readonly methods are allowed against given variable and all objects *potentially* accessible from it.

Didn't Walter say this is impossible at compile time because the data could come in through an arbitrarily-complex path? Did you show him your article?
Theoretically it is possible to deduce value of readonly tag for
any given method of the class. But this practically also needs
'mutable' keyword at least.

 Cheers,
 --AJG.

 PS: What about the idea about making all parameters by reference. Would 
 this
 allow for optimization with compile-time verification of immutability? 
 Would
 this make _anything_ easier?

I cannot see how "all parameters by reference" differ from current situation in D. "compile-time verification of immutability" of any given variable is not practically possible.

How is this different than what your article suggests? Doesn't it do exactly that?
Again, if it would be possible then GC will not be needed as compiler
can build list of free blocks in compile time.
Problem of the same nature to be short.

Andrew. 

Cheers, --AJG.
Jul 15 2005
prev sibling parent Anonymous Bosch <anon area51.gov> writes:
On Friday July 15th, 2005 21:11:11 -0700, Andrew Fedoniouk gave us this
perl ...

 I would like to return to basics again.

All I want is the compiler to warn me when it detects that some code will change something that I've said that I don't want changed. If I get that warning, then I can decide how to handle it. Maybe COW, or maybe something else...whatever! Can I have a compiler that is actively trying to help me find my mistakes? -- Anonymous Bosch -- The view will be the same no matter which path you take to the mountain top.
Jul 16 2005
prev sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:db7e7e$143d$1 digitaldaemon.com...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."
Jul 15 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Walter" <newshound digitalmars.com> wrote in message 
news:dba6ee$vtj$1 digitaldaemon.com...
 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message
 news:db7e7e$143d$1 digitaldaemon.com...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."

The problem is that WE (team of developers) DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no. Is this particular array which arrived through three or four function calls modifiable or not? Just no time to trace what and where. Practically all this will end up with "always dup before passing" approach in design guidelines. Especially dealing with third party libraries. Walter, I understand your position. You are lucky to write compilers alone. You have a luxury to know each variable origins. But for projects in 500000 lines of code and 40 developers this COWboy's deal is not working. Trust me. People in industry will not vote for D because of it is extremely hard to assemble team of 40 developers all following COW rules. Andrew.
Jul 15 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 15 Jul 2005 23:40:52 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:
 "Walter" <newshound digitalmars.com> wrote in message
 news:dba6ee$vtj$1 digitaldaemon.com...
 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message
 news:db7e7e$143d$1 digitaldaemon.com...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."

The problem is that WE (team of developers) DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no. Is this particular array which arrived through three or four function calls modifiable or not? Just no time to trace what and where. Practically all this will end up with "always dup before passing" approach in design guidelines. Especially dealing with third party libraries. Walter, I understand your position. You are lucky to write compilers alone. You have a luxury to know each variable origins. But for projects in 500000 lines of code and 40 developers this COWboy's deal is not working. Trust me. People in industry will not vote for D because of it is extremely hard to assemble team of 40 developers all following COW rules.

A reference counting or auto pointer implementation could be used to ensure you are the sole referance. Sadly we can't write one of them in D. The other solution is what we're discussin some sort of readonly mechanism enforced by the compiler or (dbc/debug) runtime. Regan
Jul 15 2005
prev sibling next sibling parent reply David Medlock <ashleymedlock no.spam.yahoo.com> writes:
Andrew Fedoniouk wrote:
 "Walter" <newshound digitalmars.com> wrote in message 
 news:dba6ee$vtj$1 digitaldaemon.com...
 
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:db7e7e$143d$1 digitaldaemon.com...

Consider this:
// somwhere in class RecordsetView ....

Field[] fields = myrecordset.fields;

// and in galaxy far, far away....

fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."

The problem is that WE (team of developers) DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no. Is this particular array which arrived through three or four function calls modifiable or not? Just no time to trace what and where. Practically all this will end up with "always dup before passing" approach in design guidelines. Especially dealing with third party libraries. Walter, I understand your position. You are lucky to write compilers alone. You have a luxury to know each variable origins. But for projects in 500000 lines of code and 40 developers this COWboy's deal is not working. Trust me. People in industry will not vote for D because of it is extremely hard to assemble team of 40 developers all following COW rules. Andrew.

Companies adopt technology because of: 1. Support and longevity. Sun and MS will fix their software and they arent closing their doors anytime soon. 2. Availability of pools of programmers if they are needed. (this is the hard one for new languages) 3. Availability of tools. 4. Interoperability with their existing systems. #1 is always number one, but the others could be swapped depending on the company. Middle management decision makers might listen to developer talk about feature B, but in the end those reasons end up deciding. There is absolutely nothing stopping you from making immutable Objects or Collections. Walter has given you all the tools to do so. He has also given you low level powerful tools to do other tasks. As a language spec it has all the capability of C#, and C# is taking off(for the reasons above). -DavidM
Jul 16 2005
parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"David Medlock" <ashleymedlock no.spam.yahoo.com> wrote in message 
news:dbav61$1joa$1 digitaldaemon.com...
 Andrew Fedoniouk wrote:
 "Walter" <newshound digitalmars.com> wrote in message 
 news:dba6ee$vtj$1 digitaldaemon.com...

"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:db7e7e$143d$1 digitaldaemon.com...

Consider this:
// somwhere in class RecordsetView ....

Field[] fields = myrecordset.fields;

// and in galaxy far, far away....

fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."

The problem is that WE (team of developers) DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no. Is this particular array which arrived through three or four function calls modifiable or not? Just no time to trace what and where. Practically all this will end up with "always dup before passing" approach in design guidelines. Especially dealing with third party libraries. Walter, I understand your position. You are lucky to write compilers alone. You have a luxury to know each variable origins. But for projects in 500000 lines of code and 40 developers this COWboy's deal is not working. Trust me. People in industry will not vote for D because of it is extremely hard to assemble team of 40 developers all following COW rules. Andrew.

Companies adopt technology because of: 1. Support and longevity. Sun and MS will fix their software and they arent closing their doors anytime soon.

I've participated in the project which was oriented on Microsoft JavaVM (the best Java VM and tools ever). As you know Microsoft JavaVM is dead now due to legal issues. And original motivation was exactly yours. Thunderclap in clear skies an all this ... So you #1 is #1 for some extent. Those guys are not buying this argument anymore as you can see. The same happened with pretty much all COM/COM+ projects recently. Someone in MS decided that COM/COM+ is evil and Java alike .NET will be next silver bullet for next 4 years...
 2. Availability of pools of programmers if they are needed. (this is the 
 hard one for new languages)
 3. Availability of tools.
 4. Interoperability with their existing systems.

 #1 is always number one, but the others could be swapped depending on the 
 company.  Middle management decision makers might listen to developer talk 
 about feature B, but in the end those reasons end up deciding.

 There is absolutely nothing stopping you from making immutable Objects or 
 Collections.  Walter has given you all the tools to do so.  He has also 
 given you low level powerful tools to do other tasks.

I am personally pretty satisfied by what D have right now. The only one thing missed: array and AA in D are Collections. But there are no immutable versions of them. And this is a problem I am talking about. So I cannot use collection-as-a-value idiom. Lack of this also follows to the fact that D has no strings in the sense of C++/Java/C#. Andrew.
Jul 16 2005
prev sibling next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
news:dbaa5l$130t$1 digitaldaemon.com...
 "Walter" <newshound digitalmars.com> wrote in message 
 news:dba6ee$vtj$1 digitaldaemon.com...
 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message
 news:db7e7e$143d$1 digitaldaemon.com...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."

The problem is that WE (team of developers) DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no. Is this particular array which arrived through three or four function calls modifiable or not? Just no time to trace what and where. Practically all this will end up with "always dup before passing" approach in design guidelines. Especially dealing with third party libraries.

I don't know what you mean by "dup before passing". It should be "dup before writing". Passing data around is trivial. I agree if one doesn't take the time to know if something can be written to it should be duped but that is exactly what COW is about. To me the biggest problem of COW in D is "premature duping": the extra copies made by people trying to be prematurely defensive that wind up wasting GC resources. Like premature optimization, duping without writing is putting effort at the wrong location.
 Walter, I understand your position. You are lucky to
 write compilers alone. You have a luxury to know each
 variable origins. But for projects in 500000 lines of code
 and 40 developers this COWboy's deal is not working.
 Trust me.

Why? Java and Fortran seem to have large programs written in them. And Java doesn't even have COW. At least Fortran has the convention that you don't write to overlapping inputs.
 People in industry will not vote for D because of
 it is extremely hard to assemble team of 40
 developers all following COW rules.

It would be easy to assemble 40 C++ programmers to follow a slightly different COW: Const Or Walk. :-P To me the biggest things holding back D is the lack of tools, docs and libraries - the language is fine.
Jul 16 2005
next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message
news:dbavgb$1jrn$1 digitaldaemon.com...
 I don't know what you mean by "dup before passing". It should be "dup

 writing". Passing data around is trivial. I agree if one doesn't take the
 time to know if something can be written to it should be duped but that is
 exactly what COW is about. To me the biggest problem of COW in D is
 "premature duping": the extra copies made by people trying to be

 defensive that wind up wasting GC resources. Like premature optimization,
 duping without writing is putting effort at the wrong location.

Right. If there is a .dup in the code that is not obviously followed by writing to the data, then odds are pretty strong the code is not COW. For example, return data.dup; and: foo(data.dup); are almost certainly wrong (unless data is stack allocated).
Jul 16 2005
next sibling parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Walter" <newshound digitalmars.com> wrote in message 
news:dbbflg$1u3s$2 digitaldaemon.com...
 "Ben Hinkle" <ben.hinkle gmail.com> wrote in message
 news:dbavgb$1jrn$1 digitaldaemon.com...
 I don't know what you mean by "dup before passing". It should be "dup

 writing". Passing data around is trivial. I agree if one doesn't take the
 time to know if something can be written to it should be duped but that 
 is
 exactly what COW is about. To me the biggest problem of COW in D is
 "premature duping": the extra copies made by people trying to be

 defensive that wind up wasting GC resources. Like premature optimization,
 duping without writing is putting effort at the wrong location.

Right. If there is a .dup in the code that is not obviously followed by writing to the data, then odds are pretty strong the code is not COW. For example, return data.dup; and: foo(data.dup); are almost certainly wrong (unless data is stack allocated).

Sure. But if you have foo declared exactly as void foo( char[] data); and you don't have source code of that library the only thing what you can do is foo(data.dup); Problem is that for some data such buggy foo will not touch your data for other it will overwrite it. If you are lucky that data.length is equal to allocation unit then any data.length = something will create a copy (accidental COW) if not then consequent data[x] = y; will damage your data. In the complex projects it is the worst bug you can imagine. - maintainance nightmare. So you in real situation you'll have foo(data.dup); - just in case. COW is a good wish - means that under the pressure of real timelines and complexity it will be just ignored by duping everything. And COW missioners will be joyefuly eaten with songs and dances.
Jul 16 2005
prev sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Sat, 16 Jul 2005 10:17:01 -0700, Walter <newshound digitalmars.com>  
wrote:
 "Ben Hinkle" <ben.hinkle gmail.com> wrote in message
 news:dbavgb$1jrn$1 digitaldaemon.com...
 I don't know what you mean by "dup before passing". It should be "dup

 writing". Passing data around is trivial. I agree if one doesn't take  
 the
 time to know if something can be written to it should be duped but that  
 is
 exactly what COW is about. To me the biggest problem of COW in D is
 "premature duping": the extra copies made by people trying to be

 defensive that wind up wasting GC resources. Like premature  
 optimization,
 duping without writing is putting effort at the wrong location.

Right. If there is a .dup in the code that is not obviously followed by writing to the data, then odds are pretty strong the code is not COW. For example, return data.dup;

Agreed.
 and:

     foo(data.dup);

Not certain, foo might be: foo(inout char[] param); right? you can't tell at this stage. Further the data referenced by the 'data' array can be modified in this function: foo(char[] param); It is not 'readonly' only the array is, right? Regan
Jul 16 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 and:

     foo(data.dup);

Not certain, foo might be: foo(inout char[] param); right? you can't tell at this stage.

Don't dup here since foo will dup if it needs to. You are probably assuming that foo will change param but if foo does not obey COW it will document it so that you can dup even with foo(char[] param). For example the std.stream.Stream.readLine(char[] buffer) takes a regular char[] input and documents that it will fill the buffer without dup'ing. That is, it explicitly tells the user it does not use COW. ps - yes, Andrew, I know that if the author of foo forgot to mention the fact that it didn't obey COW that it would step on param so you don't have to post :-)
 Further the data referenced by the 'data' array can be modified in this 
 function:
   foo(char[] param);

 It is not 'readonly' only the array is, right?

Can you give a more concrete example? If foo changes "data" then it should document that. COW doesn't apply to functions that explicitly say they don't obey COW (not surprisingly). If a function doesn't say it disobeys COW then D programmers are free to assume it obeys COW - that's "the D way".
Jul 16 2005
next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Sat, 16 Jul 2005 20:54:02 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:
 and:

     foo(data.dup);

Not certain, foo might be: foo(inout char[] param); right? you can't tell at this stage.

Don't dup here since foo will dup if it needs to. You are probably assuming that foo will change param

No, I was simply saying if you look at "foo(data.dup)" you do not know if the param is "in" or "inout". That piece of info would tell you if the function was *intending* to modify the param or not. However if it was "in" and thus not intending to modify the param it still might, accidently (bug) or illegally (bug, the programmer forgot the inout)
 but if foo does not obey COW it will document it

How?
 so that you can dup even with foo(char[] param). For example the
 std.stream.Stream.readLine(char[] buffer) takes a regular char[] input  
 and
 documents that it will fill the buffer without dup'ing. That is, it
 explicitly tells the user it does not use COW.

I dislike this and view it as wrong. The function declaration IMO should be "std.stream.Stream.readLine(inout char[] buffer)"
 ps - yes, Andrew, I know that if the author of foo forgot to mention the
 fact that it didn't obey COW that it would step on param so you don't  
 have to post :-)

Mistakes, those things we are trying to reduce, wouldn't it be nice if the compiler helped a little here? Some sort of enforcement of readonly for "in" would reduce errors at little or no cost as far as I can see.
 Further the data referenced by the 'data' array can be modified in this
 function:
   foo(char[] param);

 It is not 'readonly' only the array is, right?

Can you give a more concrete example?

void foo(char[] param) { param[0] = 'a'; }
 If foo changes "data" then it should document that.

Definately, the *compiler* should point out this error.
 COW doesn't apply to functions that explicitly say they don't obey COW  
 (not surprisingly).

Not sure what you mean here, does "void foo(char[] param)" "explicitly say" it doesn't obey COW? I don't think it says anything WRT to COW. What it says to me is that it will *read* and not *modify* the parameter I pass it, however I *know* that is not guaranteed currently. Of course, you're reffering to documentation at this point. Wouldn't it be nicer if the information was a requirement of the function declaration, which IMO it could/would be if: - in was readonly and enforced thus requring the above function to be "void foo(inout char[] param)" or for the modification to be removed, thus correcting the *bug* in the function.
 If a function doesn't say it disobeys COW then D programmers are free to  
 assume it obeys COW - that's "the D way".

Which is 'nice' but hardly comforting if you're looking for the *bug* that mutates some of your data *somewhere* in a large program. All I'm saying is that it would be *nice* to have some help ensuring we (and others) use COW correctly. An ideal like COW is good, having no support in holding to that ideal is not good. IMO. To be clear I am not questioning the ideal, I *like* this ideal. The problem is not the ideal itself it's the support we have for that ideal. Regan
Jul 16 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opst1a3xe123k2f5 nrage.netwin.co.nz...
 On Sat, 16 Jul 2005 20:54:02 -0400, Ben Hinkle <ben.hinkle gmail.com> 
 wrote:
 and:

     foo(data.dup);

Not certain, foo might be: foo(inout char[] param); right? you can't tell at this stage.

Don't dup here since foo will dup if it needs to. You are probably assuming that foo will change param

No, I was simply saying if you look at "foo(data.dup)" you do not know if the param is "in" or "inout". That piece of info would tell you if the function was *intending* to modify the param or not. However if it was "in" and thus not intending to modify the param it still might, accidently (bug) or illegally (bug, the programmer forgot the inout)

I understand.
 but if foo does not obey COW it will document it

How?

in it's documentation :-) The stuff that says what foo actually does. I think it's unusual to call a function without knowing what it does :-P
 so that you can dup even with foo(char[] param). For example the
 std.stream.Stream.readLine(char[] buffer) takes a regular char[] input 
 and
 documents that it will fill the buffer without dup'ing. That is, it
 explicitly tells the user it does not use COW.

I dislike this and view it as wrong. The function declaration IMO should be "std.stream.Stream.readLine(inout char[] buffer)"

My complaint(s) with marking buffer as inout are: 1) it means the following code is illegal char[80] buf; char[] res = stdin.readLine(buf); it errors since buf is a static array and 'inout char[]' requires an lvalue dynamic array 2) it mistakenly gives the impression that in the code char[] buf = new char[80]; char[] res = stdin.readLine(buf); if the buffer wasn't big enough it would be resizes and the variable buf would be modified (not the array contents - the actual variable 'buf'). In fact what readLine does (and I argue should do no matter what happens with const/etc) is it returns the actual array ptr and length and leaves the input buffer variable refering to the original buffer.
 ps - yes, Andrew, I know that if the author of foo forgot to mention the
 fact that it didn't obey COW that it would step on param so you don't 
 have to post :-)

Mistakes, those things we are trying to reduce, wouldn't it be nice if the compiler helped a little here? Some sort of enforcement of readonly for "in" would reduce errors at little or no cost as far as I can see.

No-one is saying it wouldn't be nice. The question is what, if anything, would balance all the pluses and minuses.
 Further the data referenced by the 'data' array can be modified in this
 function:
   foo(char[] param);

 It is not 'readonly' only the array is, right?

Can you give a more concrete example?

void foo(char[] param) { param[0] = 'a'; }
 If foo changes "data" then it should document that.

Definately, the *compiler* should point out this error.

that's what some people say and, by the way, what dlint generates a warning for (at least it does for me - I don't think I've updated the zip file on my web site yet since I haven't rebuilt in Linux lately)
 COW doesn't apply to functions that explicitly say they don't obey COW 
 (not surprisingly).

Not sure what you mean here, does "void foo(char[] param)" "explicitly say" it doesn't obey COW? I don't think it says anything WRT to COW. What it says to me is that it will *read* and not *modify* the parameter I pass it, however I *know* that is not guaranteed currently. Of course, you're reffering to documentation at this point. Wouldn't it be nicer if the information was a requirement of the function declaration, which IMO it could/would be if: - in was readonly and enforced thus requring the above function to be "void foo(inout char[] param)" or for the modification to be removed, thus correcting the *bug* in the function.
 If a function doesn't say it disobeys COW then D programmers are free to 
 assume it obeys COW - that's "the D way".

Which is 'nice' but hardly comforting if you're looking for the *bug* that mutates some of your data *somewhere* in a large program. All I'm saying is that it would be *nice* to have some help ensuring we (and others) use COW correctly. An ideal like COW is good, having no support in holding to that ideal is not good. IMO. To be clear I am not questioning the ideal, I *like* this ideal. The problem is not the ideal itself it's the support we have for that ideal.

Yeah - sure it would be nice but it isn't a no-brainer. The arguments have been made back and forth so many times I won't even bother. I really think this horse is good and dead.
Jul 16 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Sat, 16 Jul 2005 22:31:08 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:
 but if foo does not obey COW it will document it

How?

in it's documentation :-) The stuff that says what foo actually does. I think it's unusual to call a function without knowing what it does :-P

Meaningless to the 2nd, 3rd or Xth programmer reading the code. If they can read the code, look at the function and immediately *know* without reading documentation that it does/doesn't modify a parameter then this is an advantage and will always be an advantage regardless of what is said in the documentation.
 so that you can dup even with foo(char[] param). For example the
 std.stream.Stream.readLine(char[] buffer) takes a regular char[] input
 and
 documents that it will fill the buffer without dup'ing. That is, it
 explicitly tells the user it does not use COW.

I dislike this and view it as wrong. The function declaration IMO should be "std.stream.Stream.readLine(inout char[] buffer)"

My complaint(s) with marking buffer as inout are: 1) it means the following code is illegal char[80] buf; char[] res = stdin.readLine(buf); it errors since buf is a static array and 'inout char[]' requires an lvalue dynamic array

True.
 2) it mistakenly gives the impression that in the code
   char[] buf = new char[80];
   char[] res = stdin.readLine(buf);
 if the buffer wasn't big enough it would be resizes and the variable buf
 would be modified (not the array contents - the actual variable 'buf').

True. Given the 2 points above it appears we require the ability to specify the following: 1. immutable array, immutable data (default IMO) 2. immutable array, mutable data (example above) 3. mutable array, mutable data (inout) and possibly even: 4. mutable array, immutable data a static array naturally falls into category 2 as shown above.
 In fact what readLine does (and I argue should do no matter what happens  
 with const/etc) is it returns the actual array ptr and length and leaves  
 the
 input buffer variable refering to the original buffer.

Thus allowing it to work with category 2 arrays, sure.
 ps - yes, Andrew, I know that if the author of foo forgot to mention  
 the
 fact that it didn't obey COW that it would step on param so you don't
 have to post :-)

Mistakes, those things we are trying to reduce, wouldn't it be nice if the compiler helped a little here? Some sort of enforcement of readonly for "in" would reduce errors at little or no cost as far as I can see.

No-one is saying it wouldn't be nice. The question is what, if anything, would balance all the pluses and minuses.

IMO: 1. Ability to describe the categories above. 2. Enforcement of immmutability. 3. Sensible default (category 1).
 Further the data referenced by the 'data' array can be modified in  
 this
 function:
   foo(char[] param);

 It is not 'readonly' only the array is, right?

Can you give a more concrete example?

void foo(char[] param) { param[0] = 'a'; }
 If foo changes "data" then it should document that.

Definately, the *compiler* should point out this error.

that's what some people say and, by the way, what dlint generates a warning for (at least it does for me - I don't think I've updated the zip file on my web site yet since I haven't rebuilt in Linux lately)

IMO. It's not a warning, it's an error.
 An ideal like COW is good, having no support in holding to that ideal is
 not good. IMO. To be clear I am not questioning the ideal, I *like* this
 ideal. The problem is not the ideal itself it's the support we have for
 that ideal.

Yeah - sure it would be nice but it isn't a no-brainer. The arguments have been made back and forth so many times I won't even bother. I really think this horse is good and dead.

The horse isn't dead till it's been replaced by a solution. Regan
Jul 16 2005
prev sibling parent reply Carlos <Carlos_member pathlink.com> writes:
In article <dbca7d$2gas$1 digitaldaemon.com>, Ben Hinkle says...
 and:

     foo(data.dup);

Not certain, foo might be: foo(inout char[] param); right? you can't tell at this stage.

Don't dup here since foo will dup if it needs to. You are probably assuming that foo will change param but if foo does not obey COW it will document it so that you can dup even with foo(char[] param). For example the std.stream.Stream.readLine(char[] buffer) takes a regular char[] input and documents that it will fill the buffer without dup'ing. That is, it explicitly tells the user it does not use COW. ps - yes, Andrew, I know that if the author of foo forgot to mention the fact that it didn't obey COW that it would step on param so you don't have to post :-)
 Further the data referenced by the 'data' array can be modified in this 
 function:
   foo(char[] param);

 It is not 'readonly' only the array is, right?

Can you give a more concrete example? If foo changes "data" then it should document that. COW doesn't apply to functions that explicitly say they don't obey COW (not surprisingly). If a function doesn't say it disobeys COW then D programmers are free to assume it obeys COW - that's "the D way".

Forgive me that drastic formulation, but COW "the D way" (documentation only, no concept of ownership, auto_ptr ...) simply stinks. It's unsafe, why the heck not stick with C programming if that's all that D proponents have to say about this vital question? Don't get me wrong: D _is_ definitely a very exciting language! I simply don't get how you (Ben, Walter, many others ...) can be pleased with that "gentleman's agreement" esprit of COW that is currently called for "save" D programming. Experience shows that it doesn't work in the real world, and as much as I hate C++, I'd say that even the C++ const "solution" offers much more than your COWBGA (COW by gentleman's agreement). No personal offense intended, just upset ... Peace, Carlos
Jul 16 2005
next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
 Forgive me that drastic formulation, but COW "the D way" (documentation 
 only,
 no concept of ownership, auto_ptr ...) simply stinks. It's unsafe, why the
 heck not stick with C programming if that's all that D proponents have to
 say about this vital question? Don't get me wrong: D _is_ definitely a 
 very
 exciting language! I simply don't get how you (Ben, Walter, many others 
 ...)
 can be pleased with that "gentleman's agreement" esprit of COW that is 
 currently
 called for "save" D programming. Experience shows that it doesn't work in 
 the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

cowabunga - I love it! Anyway, I trust Walter will come up with a solution that satisfies everyone. :-) It is interesting how D might end up being C++ + Java + C# + ... I feel for Walter since probably when he started D it was a much smaller goal. If I were him I wouldn't hesitate to remove parts he feels are bloat now (like bit arrays and length/$ or whatever it ended up as) I've always felt like COW is much simpler than the alternative and, being simpler, is much harder to mess up. Say "const" (with the standard caveats about casting and aliasing) to a new programmer and watch the eyes glaze over.
 No personal offense intended, just upset ...

no problem. no offense taken.
 Peace,
 Carlos

 

Jul 16 2005
prev sibling next sibling parent reply David Medlock <noone nowhere.com> writes:
Carlos wrote:

  I simply don't get how you (Ben, Walter, many others ...)
 can be pleased with that "gentleman's agreement" esprit of COW that is
currently
 called for "save" D programming. Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++ 
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).
 

And, I simply don't understand why this is such an issue. Immutables are: A. Worthless as an optimization. Optimization is context dependent. B. Not a memory management tool, which is what auto_ptr is typically used for. D has memory management, its not needed. C. Defeatable in any case(without explicit hardware support) unless you remove all pointers from the language. So it basically just comes down to a lint-like check on parameters and a new set of type modifiers to keep track of when coding. If there is something else what is it? You can currently make immutable objects. You can currently make immutable collections. But wait lets make the language twice as complex to implement so you don't have to! C# doesnt have it, but business won't adopt it until D does!! Given the other features of D, it is snake oil. It looks good held up to the light, but does very little when applied. This probably sounds a bit strong, but you guys talk about const like the sun rises and sets on it. If you cannot discern your shared objects then I would argue you don't have a grasp on your program.
 No personal offense intended...

None taken, same sentiment here... -DavidM
Jul 16 2005
next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Sat, 16 Jul 2005 22:57:38 -0400, David Medlock <noone nowhere.com>  
wrote:
 Carlos wrote:

   I simply don't get how you (Ben, Walter, many others ...)
 can be pleased with that "gentleman's agreement" esprit of COW that is  
 currently
 called for "save" D programming. Experience shows that it doesn't work  
 in the
 real world, and as much as I hate C++, I'd say that even the C++ const  
 "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

And, I simply don't understand why this is such an issue. Immutables are: A. Worthless as an optimization. Optimization is context dependent.

True.
 B. Not a memory management tool, which is what auto_ptr is typically  
 used for.  D has memory management, its not needed.

Partially true. D does have memory management, so we wouldn't be using it to manage memory. However we could use it to manage ownership and thus COW.
 C. Defeatable in any case(without explicit hardware support) unless you  
 remove all pointers from the language.

True. Pointers can and will probably always defeat it, however IMO that is irrelevant. If you use pointers, you pay for the power they give, simple as that. If you don't use pointers..
 So it basically just comes down to a lint-like check on parameters and a  
 new set of type modifiers to keep track of when coding.  If there is  
 something else what is it?

A contract. From one programmer to another of intent and implementation.
 You can currently make immutable objects. You can currently make  
 immutable collections.

But you cannot make/have immutable arrays (arguably the default collection in D)
 But wait lets make the language twice as complex to implement so you  
 don't have to!  C# doesnt have it, but business won't adopt it until D  
 does!!

In C# can you return an immutable referece to internal private data, either complete or partial (a slice)? eg. class Foo { char[] private_data; char[] get_data() { return private_data; } }
 This probably sounds a bit strong, but you guys talk about const like  
 the sun rises and sets on it.

I've never even used it.
 If you cannot discern your shared objects then I would argue you don't  
 have a grasp on your program.

All I see this as, is: - a means to make discerning shared objects easier - some additional support when programming with shared objects - a means of conveying intent/design to others Regan
Jul 16 2005
parent Carlos <Carlos_member pathlink.com> writes:
Thanks Regan,

you've taken the words out of my mouth (means I just wanted to
say the same :).

Peace,
Carlos


In article <opst1fzxu423k2f5 nrage.netwin.co.nz>, Regan Heath says...
On Sat, 16 Jul 2005 22:57:38 -0400, David Medlock <noone nowhere.com>  
wrote:
 Carlos wrote:

   I simply don't get how you (Ben, Walter, many others ...)
 can be pleased with that "gentleman's agreement" esprit of COW that is  
 currently
 called for "save" D programming. Experience shows that it doesn't work  
 in the
 real world, and as much as I hate C++, I'd say that even the C++ const  
 "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

And, I simply don't understand why this is such an issue. Immutables are: A. Worthless as an optimization. Optimization is context dependent.

True.
 B. Not a memory management tool, which is what auto_ptr is typically  
 used for.  D has memory management, its not needed.

Partially true. D does have memory management, so we wouldn't be using it to manage memory. However we could use it to manage ownership and thus COW.
 C. Defeatable in any case(without explicit hardware support) unless you  
 remove all pointers from the language.

True. Pointers can and will probably always defeat it, however IMO that is irrelevant. If you use pointers, you pay for the power they give, simple as that. If you don't use pointers..
 So it basically just comes down to a lint-like check on parameters and a  
 new set of type modifiers to keep track of when coding.  If there is  
 something else what is it?

A contract. From one programmer to another of intent and implementation.
 You can currently make immutable objects. You can currently make  
 immutable collections.

But you cannot make/have immutable arrays (arguably the default collection in D)
 But wait lets make the language twice as complex to implement so you  
 don't have to!  C# doesnt have it, but business won't adopt it until D  
 does!!

In C# can you return an immutable referece to internal private data, either complete or partial (a slice)? eg. class Foo { char[] private_data; char[] get_data() { return private_data; } }
 This probably sounds a bit strong, but you guys talk about const like  
 the sun rises and sets on it.

I've never even used it.
 If you cannot discern your shared objects then I would argue you don't  
 have a grasp on your program.

All I see this as, is: - a means to make discerning shared objects easier - some additional support when programming with shared objects - a means of conveying intent/design to others Regan

Jul 17 2005
prev sibling parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"David Medlock" <noone nowhere.com> wrote in message 
news:dbcheo$2ljb$1 digitaldaemon.com...
 Carlos wrote:

  I simply don't get how you (Ben, Walter, many others ...)
 can be pleased with that "gentleman's agreement" esprit of COW that is 
 currently
 called for "save" D programming. Experience shows that it doesn't work in 
 the
 real world, and as much as I hate C++, I'd say that even the C++ const 
 "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

And, I simply don't understand why this is such an issue. Immutables are: A. Worthless as an optimization. Optimization is context dependent. B. Not a memory management tool, which is what auto_ptr is typically used for. D has memory management, its not needed. C. Defeatable in any case(without explicit hardware support) unless you remove all pointers from the language.

100% agree. Personally I am not proposing immutability of any sort as it is abstract non-reacheable dream of humanity. I am proposing two simple types: read-only array/slice type and read-only pointer.
 So it basically just comes down to a lint-like check on parameters and a 
 new set of type modifiers to keep track of when coding.  If there is 
 something else what is it?

 You can currently make immutable objects. You can currently make immutable 
 collections. But wait lets make the language twice as complex to implement 
 so you don't have to!  C# doesnt have it, but business won't adopt it 
 until D does!!

100% agree. One comment though: Necessary condition of business adoptation today is robust String solution in language. D does not provide such solution in principle now. read-only arrays can solve this problem: String (value) is in fact read-only array of chars. Read-only array is just a type as any other.
 Given the other features of D, it is snake oil.  It looks good held up to 
 the light, but does very little when applied.

 This probably sounds a bit strong, but you guys talk about const like the 
 sun rises and sets on it.  If you cannot discern your shared objects then 
 I would argue you don't have a grasp on your program.

We came here from different worlds with different experiences. For some of us it looks weird that in such in principle good language which has builtin STL comparable features you are not able to create and pass readonly arrays but is able to define readonly class or struct. Feeling of incompleteness of language design. In my personal case it was even practical problem - I've wasted three days trying to find where one particular array was changed. I didn't remember problem of such magnitude in all my C++ projects.
 No personal offense intended...

None taken, same sentiment here...

The same.
Jul 16 2005
prev sibling next sibling parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Carlos" <Carlos_member pathlink.com> wrote in message 
news:dbcd8k$2ih2$1 digitaldaemon.com...
 In article <dbca7d$2gas$1 digitaldaemon.com>, Ben Hinkle says...
 and:

     foo(data.dup);

Not certain, foo might be: foo(inout char[] param); right? you can't tell at this stage.

Don't dup here since foo will dup if it needs to. You are probably assuming that foo will change param but if foo does not obey COW it will document it so that you can dup even with foo(char[] param). For example the std.stream.Stream.readLine(char[] buffer) takes a regular char[] input and documents that it will fill the buffer without dup'ing. That is, it explicitly tells the user it does not use COW. ps - yes, Andrew, I know that if the author of foo forgot to mention the fact that it didn't obey COW that it would step on param so you don't have to post :-)
 Further the data referenced by the 'data' array can be modified in this
 function:
   foo(char[] param);

 It is not 'readonly' only the array is, right?

Can you give a more concrete example? If foo changes "data" then it should document that. COW doesn't apply to functions that explicitly say they don't obey COW (not surprisingly). If a function doesn't say it disobeys COW then D programmers are free to assume it obeys COW - that's "the D way".

Forgive me that drastic formulation, but COW "the D way" (documentation only, no concept of ownership, auto_ptr ...) simply stinks. It's unsafe, why the heck not stick with C programming if that's all that D proponents have to say about this vital question? Don't get me wrong: D _is_ definitely a very exciting language! I simply don't get how you (Ben, Walter, many others ...) can be pleased with that "gentleman's agreement" esprit of COW that is currently called for "save" D programming. Experience shows that it doesn't work in the real world, and as much as I hate C++, I'd say that even the C++ const "solution" offers much more than your COWBGA (COW by gentleman's agreement). No personal offense intended, just upset ... Peace, Carlos

Thanks, Carlos, I am also the one who don't understand programming based on "gentleman's agreement" principles. Specification should start then from definition of who is the gentleman and all needed lyrics about.
Jul 16 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
Another comment:

 Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

I wouldn't be so quick to accept claims about industry experience or existing experience and apply them to D. I'm not aware of any other language that has GC and COW. Trying to practice COW in a non-GC world like C/C++ is hard to do so the barrier is too high. One can practice COW in a GC language like Java but it isn't done widely. So naturally if one is writing a library in Java and trying to decide upon COW or defensive-duping one will chose defensive-duping since most people won't expect COW. In D since everyone should expect COW it would be strange to *not* use COW.
Jul 17 2005
next sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dbe3f9$p88$1 digitaldaemon.com...
 Another comment:

 Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

I wouldn't be so quick to accept claims about industry experience or existing experience and apply them to D. I'm not aware of any other language that has GC and COW.

Sorry, I does not understand this statement. GC is builtin in D runtime - so "D has..." is applicable here. Does D have builtin COW?
 Trying to practice COW in a non-GC world like C/C++ is hard to do so the 
 barrier is too high. One can practice COW in a GC language like Java but 
 it isn't done widely. So naturally if one is writing a library in Java and 
 trying to decide upon COW or defensive-duping one will chose 
 defensive-duping since most people won't expect COW. In D since everyone 
 should expect COW it would be strange to *not* use COW.

Java authors were realists. (.NET also apply) Probably this is the reason? As Java systems proven to work in projects of various scales then I believe this realistic assumption makes some sense. No? COW is a universal technique and used everywhere when it makes sense. Java (as D) allows only to do this manually this is why it is used in Java but not so widespread. C++ allows to automate COW algorithms - this is why COW can be used naturally in C++ (e.g. http://doc.trolltech.com/3.3/shclass.html#3) More: 1. The COW design pattern is quite orthogonal to the problem of read-only arrays and pointers. char[#] toUpper( char[#] str ) { // 1) can be implemented using COW or not, // 2) will not modify str. } 2. COW is expensive in terms of effectiveness and size of final code: char[] toUpper( char[] str ) { bool changed; for(...) { if( need to write ) // these if's are not if( changed ) // desirable in the cycle body //do new copy allocation else // proceed } } 3. In real cases and classes COW leads to non-trivial implementations - many consequences here: - demand of more skilled programmers. - maintainance, code reviews costs, etc. Andrew.
Jul 17 2005
next sibling parent Carlos <Carlos_member pathlink.com> writes:
In article <dbefq2$11oo$1 digitaldaemon.com>, Andrew Fedoniouk says...
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dbe3f9$p88$1 digitaldaemon.com...
 Another comment:

 Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

I wouldn't be so quick to accept claims about industry experience or existing experience and apply them to D. I'm not aware of any other language that has GC and COW.

Sorry, I does not understand this statement. GC is builtin in D runtime - so "D has..." is applicable here. Does D have builtin COW?

Yep, that's the point! In C++ COW is transparent to the caller/client of an API (copy made without him even being aware of it, that's why I mentioned auto_ptr in my original post). The situation in D is whole lotta different unfortunately ... Peace, Carlos
 Trying to practice COW in a non-GC world like C/C++ is hard to do so the 
 barrier is too high. One can practice COW in a GC language like Java but 
 it isn't done widely. So naturally if one is writing a library in Java and 
 trying to decide upon COW or defensive-duping one will chose 
 defensive-duping since most people won't expect COW. In D since everyone 
 should expect COW it would be strange to *not* use COW.

Java authors were realists. (.NET also apply) Probably this is the reason? As Java systems proven to work in projects of various scales then I believe this realistic assumption makes some sense. No? COW is a universal technique and used everywhere when it makes sense. Java (as D) allows only to do this manually this is why it is used in Java but not so widespread. C++ allows to automate COW algorithms - this is why COW can be used naturally in C++ (e.g. http://doc.trolltech.com/3.3/shclass.html#3) More: 1. The COW design pattern is quite orthogonal to the problem of read-only arrays and pointers. char[#] toUpper( char[#] str ) { // 1) can be implemented using COW or not, // 2) will not modify str. } 2. COW is expensive in terms of effectiveness and size of final code: char[] toUpper( char[] str ) { bool changed; for(...) { if( need to write ) // these if's are not if( changed ) // desirable in the cycle body //do new copy allocation else // proceed } } 3. In real cases and classes COW leads to non-trivial implementations - many consequences here: - demand of more skilled programmers. - maintainance, code reviews costs, etc. Andrew.

Jul 17 2005
prev sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
news:dbefq2$11oo$1 digitaldaemon.com...
 "Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
 news:dbe3f9$p88$1 digitaldaemon.com...
 Another comment:

 Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

I wouldn't be so quick to accept claims about industry experience or existing experience and apply them to D. I'm not aware of any other language that has GC and COW.

Sorry, I does not understand this statement. GC is builtin in D runtime - so "D has..." is applicable here. Does D have builtin COW?

I don't see what your confusion is about - of course D doesn't have builtin COW. I think you know that. Are you confused by my use of the tem "language" to mean both what a compiler will enforce and what the conventions "enforce"? I assume you are just being argumentative here.
 Trying to practice COW in a non-GC world like C/C++ is hard to do so the 
 barrier is too high. One can practice COW in a GC language like Java but 
 it isn't done widely. So naturally if one is writing a library in Java 
 and trying to decide upon COW or defensive-duping one will chose 
 defensive-duping since most people won't expect COW. In D since everyone 
 should expect COW it would be strange to *not* use COW.

Java authors were realists. (.NET also apply) Probably this is the reason?

or they were wimps and gave into the naysayers without testing it out first :-)
 As Java systems proven to work in projects of various scales then
 I believe this realistic assumption makes some sense. No?

Defensive dup'ing isn't a disaster - it just reduces performance.
 COW is a universal technique and used everywhere when
 it makes sense. Java (as D) allows only to do this manually
 this is why it is used in Java but not so widespread.

 C++ allows to automate COW algorithms - this is
 why COW can be used naturally in C++
 (e.g. http://doc.trolltech.com/3.3/shclass.html#3)

<shudder> :-)
 More:

 1. The COW design pattern is quite orthogonal to the
 problem of read-only arrays and pointers.

 char[#] toUpper( char[#] str )
 {
   // 1) can be implemented using COW or not,
   // 2) will not modify str.
 }

The need for read-only arrays is reduced by COW, though. The question is if the need is great enough to make it worth-while.
 2. COW is expensive in terms of effectiveness
 and size of final code:

 char[] toUpper( char[] str )
 {
   bool changed;
   for(...)
   {
       if( need to write ) // these if's are not
         if( changed )     // desirable in the cycle body
           //do new copy allocation
         else
          // proceed
   }
 }

Maybe it's just me but that doesn't look too expensive to me. If one is really concerned about the if's the for loop can be explicitly broken up but that would be a last resort optimization.
 3. In real cases and classes COW leads to non-trivial
 implementations - many consequences here:
 - demand of more skilled programmers.
 - maintainance, code reviews costs, etc.

A system using an equal mix of COW and non-COW will require more effort to maintain IMHO than one that consistently applies one style as much as possible. Non-COW code should be used when one is reasonably certain performance will be a concern.
Jul 17 2005
prev sibling next sibling parent Carlos <Carlos_member pathlink.com> writes:
In article <dbe3f9$p88$1 digitaldaemon.com>, Ben Hinkle says...
Another comment:

 Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

I wouldn't be so quick to accept claims about industry experience or existing experience and apply them to D. I'm not aware of any other language that has GC and COW. Trying to practice COW in a non-GC world like C/C++ is hard to do so the barrier is too high. One can practice COW in a GC language like Java but it isn't done widely. So naturally if one is writing a library in Java and trying to decide upon COW or defensive-duping one will chose defensive-duping since most people won't expect COW. In D since everyone should expect COW it would be strange to *not* use COW.

I agree that not using COW would be alien to D as it is now. But, suppose I'd be able to say "the returned array is readonly" by some compiler-enforced keyword (part of the method signature, _not_ documentation), neither the caller nor the API designer has to care about unnecessary duping and it would be a clear contract (or "promise", from developer to developer). The current situation will only lead to duping "just in case" and is unreliable (humans fail) too. I'd see it as a not too small improvement, if we could at least say "you need to make a copy here" and that is enforced by the compiler. I'm aware that such compiler checks are not waterproof guarantees and could be bypassed purposely, but I'm not asking for a 100% tight compiler proof here. To express myself clearly: I'm not against COW per se, what I don't like (and feel is dangerous "in the real world") is the "by gentleman's agreement" approach to COW that D is taking right now. Peace, Carlos
Jul 17 2005
prev sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sun, 17 Jul 2005 13:11:02 -0400, Ben Hinkle wrote:

 Another comment:
 
 Experience shows that it doesn't work in the
 real world, and as much as I hate C++, I'd say that even the C++
 const "solution" offers much more than your COWBGA (COW by gentleman's
 agreement).

I wouldn't be so quick to accept claims about industry experience or existing experience and apply them to D. I'm not aware of any other language that has GC and COW.

The interpreted language called 'Euphoria' has both, and neither are optional or manual. The language just does both for you and the coder is not even aware of what's happening in the background. -- Derek Parnell Melbourne, Australia 18/07/2005 7:36:17 AM
Jul 17 2005
parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
 I wouldn't be so quick to accept claims about industry experience or
 existing experience and apply them to D. I'm not aware of any other 
 language
 that has GC and COW.

The interpreted language called 'Euphoria' has both, and neither are optional or manual. The language just does both for you and the coder is not even aware of what's happening in the background.

True - I meant user-enforced COW. Another example of a language like the one you mention is MATLAB. In MATLAB all data has value semantics enforced by MATLAB using COW without the user ever knowing. For example the code x = zeros(100); y = x; makes a 100x100 matrix of zeros and makes a shared-data copy in y. The line y(10,10) = 1; will copy the data to a new 100x100 matrix and set element (10,10) to 1. MATLAB also manages all memory automatically.
Jul 17 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dbavgb$1jrn$1 digitaldaemon.com...
 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
 news:dbaa5l$130t$1 digitaldaemon.com...
 "Walter" <newshound digitalmars.com> wrote in message 
 news:dba6ee$vtj$1 digitaldaemon.com...
 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message
 news:db7e7e$143d$1 digitaldaemon.com...
 Consider this:
 // somwhere in class RecordsetView ....

 Field[] fields = myrecordset.fields;

 // and in galaxy far, far away....

 fields.sort; // pretty harmless, isn't it?

The example code violates the basic COW principle of "Do not modify the referenced data unless you KNOW you are the sole reference to it."

The problem is that WE (team of developers) DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no. Is this particular array which arrived through three or four function calls modifiable or not? Just no time to trace what and where. Practically all this will end up with "always dup before passing" approach in design guidelines. Especially dealing with third party libraries.

I don't know what you mean by "dup before passing". It should be "dup before writing". Passing data around is trivial. I agree if one doesn't take the time to know if something can be written to it should be duped but that is exactly what COW is about. To me the biggest problem of COW in D is "premature duping": the extra copies made by people trying to be prematurely defensive that wind up wasting GC resources. Like premature optimization, duping without writing is putting effort at the wrong location.
 Walter, I understand your position. You are lucky to
 write compilers alone. You have a luxury to know each
 variable origins. But for projects in 500000 lines of code
 and 40 developers this COWboy's deal is not working.
 Trust me.

Why? Java and Fortran seem to have large programs written in them. And Java doesn't even have COW. At least Fortran has the convention that you don't write to overlapping inputs.

What does it mean exactly: "And Java doesn't even have COW." ? Java has Strings and other value referential types, we already discussed this many times...
 People in industry will not vote for D because of
 it is extremely hard to assemble team of 40
 developers all following COW rules.

It would be easy to assemble 40 C++ programmers to follow a slightly different COW: Const Or Walk. :-P To me the biggest things holding back D is the lack of tools, docs and libraries - the language is fine.

Think about this hypothesis: "D has no strong libraries because its type system is not complete - it has no readonly referential types including strings". For example:
Jul 16 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 Why? Java and Fortran seem to have large programs written in them. And 
 Java doesn't even have COW. At least Fortran has the convention that you 
 don't write to overlapping inputs.

What does it mean exactly: "And Java doesn't even have COW." ? Java has Strings and other value referential types, we already discussed this many times...

COW is a general principle that applies to more than just strings. Java strings are immutable but that doesn't mean Java programmers use COW- they use immutable strings. Everywhere else they dup like crazy just in case someone tries to overwrite their fields or arrays. You know this well since you've used the example of Swing and small classes like Point and Rectangle which get defensively dup'ed just because Java doesn't use COW.
 People in industry will not vote for D because of
 it is extremely hard to assemble team of 40
 developers all following COW rules.

It would be easy to assemble 40 C++ programmers to follow a slightly different COW: Const Or Walk. :-P To me the biggest things holding back D is the lack of tools, docs and libraries - the language is fine.

Think about this hypothesis: "D has no strong libraries because its type system is not complete - it has no readonly referential types including strings".

or maybe "D has no strong libraries because we spend all our time arguing about const" :-)
 For example:

Jul 16 2005
parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message 
news:dbblrv$22kg$1 digitaldaemon.com...
 Why? Java and Fortran seem to have large programs written in them. And 
 Java doesn't even have COW. At least Fortran has the convention that you 
 don't write to overlapping inputs.

What does it mean exactly: "And Java doesn't even have COW." ? Java has Strings and other value referential types, we already discussed this many times...

COW is a general principle that applies to more than just strings. Java strings are immutable but that doesn't mean Java programmers use COW- they use immutable strings. Everywhere else they dup like crazy just in case someone tries to overwrite their fields or arrays. You know this well since you've used the example of Swing and small classes like Point and Rectangle which get defensively dup'ed just because Java doesn't use COW.

So could you tell me why they are not using COW? What is preventing them to use it?
 People in industry will not vote for D because of
 it is extremely hard to assemble team of 40
 developers all following COW rules.

It would be easy to assemble 40 C++ programmers to follow a slightly different COW: Const Or Walk. :-P To me the biggest things holding back D is the lack of tools, docs and libraries - the language is fine.

Think about this hypothesis: "D has no strong libraries because its type system is not complete - it has no readonly referential types including strings".

or maybe "D has no strong libraries because we spend all our time arguing about const" :-)

:) Remeber that first I did attempt to write such library and after that I've started screaming. Seriosly: I am getting almost once per two weeks letter like: "We need a fast GUI for our application. Preferrably platform independent,etc. UI forms are stored in DB as HTML - total 2000 different forms. Development will be outsourced in .... What would you recommend?" I simply cannot recommend Harmonia now for that. This "outsourced in..." is too far from Canada and I physically cannot sing that song about the cow. And Canada provide no other COW enforcment tools like ballistic missiles or something like this. Three battalions of humanitarian aid forces. This is it and extremely good - correctly designed type system.
Jul 16 2005
prev sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:dbaa5l$130t$1 digitaldaemon.com...
 "Walter" <newshound digitalmars.com> wrote in message
 news:dba6ee$vtj$1 digitaldaemon.com...
 The example code violates the basic COW principle of "Do not modify the
 referenced data unless you KNOW you are the sole reference to it."

DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no.

The corollary to the stated principle is "If you don't KNOW that you are the sole owner of the referenced data, then dup it before modifying it."
 Is this particular array which arrived through three or four function

 modifiable or not? Just no time to trace what and where.
 Practically all this will end up with "always dup before passing"
 approach in design guidelines. Especially dealing with third party
 libraries.

No, "always dup before passing" is approaching the problem from the wrong direction, and is not COW. COW is "dup when you modify it". Never dup because someone ELSE might modify it. Dup when you DO modify it and you DON'T KNOW if you are the sole owner.
 Walter, I understand your position. You are lucky to
 write compilers alone. You have a luxury to know each
 variable origins. But for projects in 500000 lines of code
 and 40 developers this COWboy's deal is not working.
 Trust me.

 People in industry will not vote for D because of
 it is extremely hard to assemble team of 40
 developers all following COW rules.

Jul 16 2005
parent "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Walter" <newshound digitalmars.com> wrote in message 
news:dbbflf$1u3s$1 digitaldaemon.com...
 "Andrew Fedoniouk" <news terrainformatica.com> wrote in message
 news:dbaa5l$130t$1 digitaldaemon.com...
 "Walter" <newshound digitalmars.com> wrote in message
 news:dba6ee$vtj$1 digitaldaemon.com...
 The example code violates the basic COW principle of "Do not modify the
 referenced data unless you KNOW you are the sole reference to it."

DON'T KNOW in ALMOST ALL CASES is this variable is a sole owner of array or no.

The corollary to the stated principle is "If you don't KNOW that you are the sole owner of the referenced data, then dup it before modifying it."
 Is this particular array which arrived through three or four function

 modifiable or not? Just no time to trace what and where.
 Practically all this will end up with "always dup before passing"
 approach in design guidelines. Especially dealing with third party
 libraries.

No, "always dup before passing" is approaching the problem from the wrong direction, and is not COW. COW is "dup when you modify it". Never dup because someone ELSE might modify it. Dup when you DO modify it and you DON'T KNOW if you are the sole owner.

Stop, Walter, please don't waste your time on me. I personally know what is COW, how to use it and where. Basic principle in industry - either COW must be automatic for most commonly used cases either enforced somehow. Otherwise it will end up with dup everywhere.
 Walter, I understand your position. You are lucky to
 write compilers alone. You have a luxury to know each
 variable origins. But for projects in 500000 lines of code
 and 40 developers this COWboy's deal is not working.
 Trust me.

 People in industry will not vote for D because of
 it is extremely hard to assemble team of 40
 developers all following COW rules.


Jul 16 2005