www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - const?? When and why? This is ugly!

reply hasen <hasan.aljudy gmail.com> writes:
I haven't been following D for over a year .. now I notice there's a const!!

In C++, in my experience, the only time I need const is when I pass 
temporary objects by reference and the compiler refuses, so I make the 
parameter const to make the compiler happy.

void my_func( const SomeType& obj ) { ... }

This is annoying because now const propagates like a virus! any function 
that I call on `obj` must also be changed to accept a `const` reference. 
any  method on obj which doesn't change it must also be marked as 
`const` in order for it to be callable from inside this function.

This whole stupid process wouldn't even be needed in the first place if 
C++ had a garbage collector, because then I would always "new" them (as 


SomeType *obj = new SomeType();

but because there's no garbage collector, I have to create the object 
not as a reference.

SomeType obj();

Again, this is all stupid C++ stuff that shouldn't even be needed in the 
first place.

However, with all that being said, that's just my opinion, maybe over 
the time some people found some actually useful uses for const, great, 
so they can use it if they want.

What really annoys me is the viral nature of const.

Yesterday I was reading this: http://www.digitalmars.com/d/2.0/const3.html

(btw, this page is empty: http://www.digitalmars.com/d/2.0/const.html )

and, to my surprise, I see:

char[] p = "world"; // error, cannot implicitly convert invariant
		    // to mutable


and all I can think of is: WHAT - THE - HELL??!!!!!!

invariant(char)[] is ugly! might as well be saying std::vector<char> 
(ok, not the best example of stupidity, but I hope you get my point).
(and, is invariant(char)[] the correct type? or must it be 
invariant(char)[5]??)

This is not the D that I loved. What's going on?

P.S. look here 
http://stackoverflow.com/questions/557011/d-programming-language-char-arrays/

I know you can use "auto" and all that, but still .. I don't feel good 
about this.
Mar 01 2009
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from hasen (hasan.aljudy gmail.com)'s article

 I haven't been following D for over a year .. now I notice there's a const!!
 In C++, in my experience, the only time I need const is when I pass
 temporary objects by reference and the compiler refuses, so I make the
 parameter const to make the compiler happy.
 void my_func( const SomeType& obj ) { ... }
 This is annoying because now const propagates like a virus! any function
 that I call on `obj` must also be changed to accept a `const` reference.
 any  method on obj which doesn't change it must also be marked as
 `const` in order for it to be callable from inside this function.
 This whole stupid process wouldn't even be needed in the first place if
 C++ had a garbage collector, because then I would always "new" them (as

 SomeType *obj = new SomeType();
 but because there's no garbage collector, I have to create the object
 not as a reference.
 SomeType obj();
 Again, this is all stupid C++ stuff that shouldn't even be needed in the
 first place.
 However, with all that being said, that's just my opinion, maybe over
 the time some people found some actually useful uses for const, great,
 so they can use it if they want.
 What really annoys me is the viral nature of const.
 Yesterday I was reading this: http://www.digitalmars.com/d/2.0/const3.html
 (btw, this page is empty: http://www.digitalmars.com/d/2.0/const.html )
 and, to my surprise, I see:
 char[] p = "world"; // error, cannot implicitly convert invariant
 		    // to mutable
 and all I can think of is: WHAT - THE - HELL??!!!!!!
 invariant(char)[] is ugly! might as well be saying std::vector<char>
 (ok, not the best example of stupidity, but I hope you get my point).
 (and, is invariant(char)[] the correct type? or must it be
 invariant(char)[5]??)
 This is not the D that I loved. What's going on?
 P.S. look here
 http://stackoverflow.com/questions/557011/d-programming-language-char-arrays/
 I know you can use "auto" and all that, but still .. I don't feel good
 about this.


Uhh, that's what aliases are for.  The string alias is defined automatically in
object, and string is an alias for immutable(char)[].

Also, modulo a few library functions that use strings/immutable(char)[]s, when
they should be using const(char)[]s, one can avoid const by simply not using it
if
one doesn't like it or doesn't need it for the program they're writing.
Mar 01 2009
parent reply grauzone <none example.net> writes:
dsimcha wrote:

 == Quote from hasen (hasan.aljudy gmail.com)'s article

 I haven't been following D for over a year .. now I notice there's a const!!
 In C++, in my experience, the only time I need const is when I pass
 temporary objects by reference and the compiler refuses, so I make the
 parameter const to make the compiler happy.
 void my_func( const SomeType& obj ) { ... }
 This is annoying because now const propagates like a virus! any function
 that I call on `obj` must also be changed to accept a `const` reference.
 any  method on obj which doesn't change it must also be marked as
 `const` in order for it to be callable from inside this function.
 This whole stupid process wouldn't even be needed in the first place if
 C++ had a garbage collector, because then I would always "new" them (as

 SomeType *obj = new SomeType();
 but because there's no garbage collector, I have to create the object
 not as a reference.
 SomeType obj();
 Again, this is all stupid C++ stuff that shouldn't even be needed in the
 first place.
 However, with all that being said, that's just my opinion, maybe over
 the time some people found some actually useful uses for const, great,
 so they can use it if they want.
 What really annoys me is the viral nature of const.
 Yesterday I was reading this: http://www.digitalmars.com/d/2.0/const3.html
 (btw, this page is empty: http://www.digitalmars.com/d/2.0/const.html )
 and, to my surprise, I see:
 char[] p = "world"; // error, cannot implicitly convert invariant
 		    // to mutable
 and all I can think of is: WHAT - THE - HELL??!!!!!!
 invariant(char)[] is ugly! might as well be saying std::vector<char>
 (ok, not the best example of stupidity, but I hope you get my point).
 (and, is invariant(char)[] the correct type? or must it be
 invariant(char)[5]??)
 This is not the D that I loved. What's going on?
 P.S. look here
 http://stackoverflow.com/questions/557011/d-programming-language-char-arrays/
 I know you can use "auto" and all that, but still .. I don't feel good
 about this.


 
 Uhh, that's what aliases are for.  The string alias is defined automatically in
 object, and string is an alias for immutable(char)[].


That Phobos introduces this alias is not really an argument that speaks 
in favor of the syntax.

Also, I don't quite understand yet what constness is good for. While it 
seems to be a good idea, it seems to be more an annoyance in practice. 
It doesn't seem to pay off. What is this additional complexity for? And 
really, what was wrong with not having constness? All I hear is 
something about threading, but I didn't see anything convincing yet. No 
real examples, not even a full concept. Only some loose arguments. Come 
on, the newest compiler releases can do const, now convince me already 
that it's a good feature!

Regarding pure functions: isn't CTFE already enough?


 Also, modulo a few library functions that use strings/immutable(char)[]s, when
 they should be using const(char)[]s, one can avoid const by simply not using
it if
 one doesn't like it or doesn't need it for the program they're writing.
Mar 02 2009
parent reply Jason House <jason.james.house gmail.com> writes:
grauzone Wrote:


 dsimcha wrote:

 == Quote from hasen (hasan.aljudy gmail.com)'s article
 Uhh, that's what aliases are for.  The string alias is defined automatically in
 object, and string is an alias for immutable(char)[].


 
 That Phobos introduces this alias is not really an argument that speaks 
 in favor of the syntax.
 
 Also, I don't quite understand yet what constness is good for. While it 
 seems to be a good idea, it seems to be more an annoyance in practice. 
 It doesn't seem to pay off. What is this additional complexity for? And 
 really, what was wrong with not having constness? All I hear is 
 something about threading, but I didn't see anything convincing yet. No 
 real examples, not even a full concept. Only some loose arguments. Come 
 on, the newest compiler releases can do const, now convince me already 
 that it's a good feature!


Given the tone of your messages, I assume convincing you is impossible. The
biggest help I can offer is to tell you that "the D you loved" still exists as
D 1.0... Walter still maintains the compiler for it and Tango, Descent, GDC,
LDC, and many other things all support D1.
 

 Regarding pure functions: isn't CTFE already enough?


The big driver of D2 is provably safe multithreading without locks. Pure
functions are the embodiment of that, and immutable data is required for that.
Const is used to handle either mutable or immutable data. 

Does that help?
Mar 02 2009
next sibling parent reply grauzone <none example.net> writes:
Jason House wrote:

 grauzone Wrote:
 

 dsimcha wrote:

 == Quote from hasen (hasan.aljudy gmail.com)'s article
 Uhh, that's what aliases are for.  The string alias is defined automatically in
 object, and string is an alias for immutable(char)[].


 That Phobos introduces this alias is not really an argument that speaks 
 in favor of the syntax.

 Also, I don't quite understand yet what constness is good for. While it 
 seems to be a good idea, it seems to be more an annoyance in practice. 
 It doesn't seem to pay off. What is this additional complexity for? And 
 really, what was wrong with not having constness? All I hear is 
 something about threading, but I didn't see anything convincing yet. No 
 real examples, not even a full concept. Only some loose arguments. Come 
 on, the newest compiler releases can do const, now convince me already 
 that it's a good feature!


 
 Given the tone of your messages, I assume convincing you is impossible. The
biggest help I can offer is to tell you that "the D you loved" still exists as
D 1.0... Walter still maintains the compiler for it and Tango, Descent, GDC,
LDC, and many other things all support D1.


Forgive me, I always get the tone wrong. As for the "D you loved", 
please don't assume things. I think D2.0 is a huge improvement over 
D1.0. But of course, there are always some issues. Nothing is perfect, 
after all.

And actually, I'm not opposed to these new (actually old) ideas. I just 
want more information. And yes, I don't like the current const 
mechanism, because I think it introduces complexity for no gain. That's 
why I'd like to know more in the first place.

And I think I'm not the only one who would prefer to see ast macros to 
be implemented rather than const.


 Regarding pure functions: isn't CTFE already enough?


 
 The big driver of D2 is provably safe multithreading without locks. Pure
functions are the embodiment of that, and immutable data is required for that.
Const is used to handle either mutable or immutable data. 


You probably mean that data shared between threads has to be immutable. 
Then all threads can access that data without synchronization. 
Immutability will sure be useful here: the type system guarantees, that 
data can never be modified. But how does this work for complex types 
like objects? Is there an idup for objects? Does it do a deep copy of an 
object graph? How is communication between threads supposed to work? 
Shouldn't all global variables be immutable then? Please, where can I 
find complete, thought-through concepts?

Does this approach fit D at all? D is a C based system programming 
language. Does it work to demand from everyone to obey the const rules? 
Couldn't all this be done manually, instead of bothering the programmer 
with a complex type system?

Sometimes, you have to leave the more spiffy features to languages like 
Erlang or Haskell which are suited better for this. For example, Erlang 
doesn't share any state by default, and in Haskell, everything is 
immutable. But they use very special mechanisms to make up for the 
problems caused by this, and that's why you can't just pick the 
cherries: it won't fit, it will feel unnatural, and everything will be a 
mess. It's a good idea to copy good features from other languages, but 
never forget where you come from.

Regarding pureness: I don't think it has anything do to with locking. A 
function could just take const arguments, and everything is safe. It 
can't write to shared data either, because shared data is supposed to be 
immutable, right? In this context, it doesn't matter if the function 
writes to anything else, because that data will be thread local.

This rather seems to have to do with optimization by caching results, 
and with automatic parallelization. Well then, I want to see how this 
can be applied in D! Again, forgive me my tone. I mean, it's not like 
I'm threatening anyone or that I want to start flamewars, I just want to 
know more about this.


 Does that help?


Not really. I just wasted time by writing this post, and probably the 
time of people who read this post. It's a shame.
Mar 02 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:

 You probably mean that data shared between threads has to be immutable. 
 Then all threads can access that data without synchronization. 
 Immutability will sure be useful here: the type system guarantees, that 
 data can never be modified. But how does this work for complex types 
 like objects? Is there an idup for objects? Does it do a deep copy of an 
 object graph? How is communication between threads supposed to work? 
 Shouldn't all global variables be immutable then? Please, where can I 
 find complete, thought-through concepts?


You cannot find them yet because the thinking-through process is still 
ongoing.


 Does this approach fit D at all? D is a C based system programming 
 language. Does it work to demand from everyone to obey the const rules? 
 Couldn't all this be done manually, instead of bothering the programmer 
 with a complex type system?


C is still having trouble removing those pesky buffer overruns. Writing 
a large manycore application in C will be anything but casual programming.


 Sometimes, you have to leave the more spiffy features to languages like 
 Erlang or Haskell which are suited better for this. For example, Erlang 
 doesn't share any state by default, and in Haskell, everything is 
 immutable. But they use very special mechanisms to make up for the 
 problems caused by this, and that's why you can't just pick the 
 cherries: it won't fit, it will feel unnatural, and everything will be a 
 mess. It's a good idea to copy good features from other languages, but 
 never forget where you come from.


We plan to add the necessary mechanisms.


 Regarding pureness: I don't think it has anything do to with locking. A 
 function could just take const arguments, and everything is safe. It 
 can't write to shared data either, because shared data is supposed to be 
 immutable, right? In this context, it doesn't matter if the function 
 writes to anything else, because that data will be thread local.


Not all shared data is immutable. Data marked as "shared" is shared. The 
cool thing is that D switches the default to the "right" default.

The style of programming in which you create a new thread that joyously 
sees and can modify *all* of the memory space available to every other 
thread will go the way of the dinosaur. It's just not tenable anymore, 
and I predict today's mainstream programming languages abiding to that 
model are facing major troubles in the near future.


Andrei
Mar 02 2009
next sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Mon, Mar 2, 2009 at 12:25 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 The style of programming in which you create a new thread that joyously sees
 and can modify *all* of the memory space available to every other thread
 will go the way of the dinosaur. It's just not tenable anymore, and I
 predict today's mainstream programming languages abiding to that model are
 facing major troubles in the near future.


Eheh, sounds like the dawn of protected memory 20 years ago ;)
Mar 02 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:

 On Mon, Mar 2, 2009 at 12:25 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 The style of programming in which you create a new thread that joyously sees
 and can modify *all* of the memory space available to every other thread
 will go the way of the dinosaur. It's just not tenable anymore, and I
 predict today's mainstream programming languages abiding to that model are
 facing major troubles in the near future.


 
 Eheh, sounds like the dawn of protected memory 20 years ago ;)


Absolutely. The industry has cycles and a lot of its history rhymes. 
Things like multitasking, client-server computing, flat memory 
addressing, virtual memory, machine virtualization... they've all come 
and went a couple of times over decades.

As for my bet, I'm pretty confident in it. The default-isolated model is 
gaining traction in newer languages, in older languages that are seeing 
a new youth with the advent of manycores, and even in applications. 
Google Chrome figured that it must use OS-provided memory isolation for 
its tabs.

D is positioned very well, probably uniquely, to take advantage of a 
default-isolated model within a largely mutative language, and that is 
in no small part due to const and immutable. Other languages are 
struggling, see http://tinyurl.com/4apat7. I'm very glad that the tide 
on this group has effectively reversed with regard to the general 
opinion about const and immutable.


Andrei
Mar 02 2009
next sibling parent downs <default_357-line yahoo.de> writes:
Andrei Alexandrescu wrote:

 D is positioned very well, probably uniquely, to take advantage of a
 default-isolated model within a largely mutative language, and that is
 in no small part due to const and immutable. Other languages are
 struggling, see http://tinyurl.com/4apat7. I'm very glad that the tide
 on this group has effectively reversed with regard to the general
 opinion about const and immutable.
 
 
 Andrei



I'm not convinced, I'm just withholding judgment until there's a stable 2.0
implemented with an open-source compiler.

I suspect I'm not the only one here who thinks that way :)
Mar 02 2009
prev sibling parent reply grauzone <none example.net> writes:
 struggling, see http://tinyurl.com/4apat7. I'm very glad that the tide 
 on this group has effectively reversed with regard to the general 
 opinion about const and immutable.


I think they got annoyed and tired by the weekly (daily?) new const 
proposal threads and went silent.
Mar 02 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:

 struggling, see http://tinyurl.com/4apat7. I'm very glad that the 
 tide on this group has effectively reversed with regard to the 
 general opinion about const and immutable.


 
 I think they got annoyed and tired by the weekly (daily?) new const 
 proposal threads and went silent.


and in another post:


 I mean, it's not like I'm threatening anyone or that I want to start
 flamewars, I just want to know more about this.


I was wondering how one goes with the other.


Andrei
Mar 02 2009
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 Not all shared data is immutable. Data marked as "shared" is shared. The 
 cool thing is that D switches the default to the "right" default.
 
 The style of programming in which you create a new thread that joyously 
 sees and can modify *all* of the memory space available to every other 
 thread will go the way of the dinosaur. It's just not tenable anymore, 
 and I predict today's mainstream programming languages abiding to that 
 model are facing major troubles in the near future.


I agree.

Also, take a look at Clojure, one of the very few languages designed to manage
multi-core CPUs. In Clojure you can use mutable data and data structures, but
the default is immutable data. They argue that's the "right" default if you
want to program for multi-core CPUs. If this and the Clojure experiment turn
out as true/right, D2/D3 may have to change more :-)

In the close future D std lib may enjoy to grow several immutable data
structures (finger trees, etc), even if the D2 language & community on the
whole has not embraced the immutability much yet. Such data structures and
other features in the std lib can't hurt. We'll surely talk more about this
topic.

Bye,
bearophile
Mar 02 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
bearophile wrote:

 Andrei Alexandrescu:

 Not all shared data is immutable. Data marked as "shared" is
 shared. The cool thing is that D switches the default to the
 "right" default.
 
 The style of programming in which you create a new thread that
 joyously sees and can modify *all* of the memory space available to
 every other thread will go the way of the dinosaur. It's just not
 tenable anymore, and I predict today's mainstream programming
 languages abiding to that model are facing major troubles in the
 near future.


 
 I agree.
 
 Also, take a look at Clojure, one of the very few languages designed
 to manage multi-core CPUs. In Clojure you can use mutable data and
 data structures, but the default is immutable data. They argue that's
 the "right" default if you want to program for multi-core CPUs. If
 this and the Clojure experiment turn out as true/right, D2/D3 may
 have to change more :-)


Or Clojure. (I'm not even kidding. D has a very compelling computation 
model that beats the pants off traditional functional approaches.)

The thing is, in a few short years "if you want to program for 
multi-core CPUs" will sound as funny as "if you want to program for CPUs 
with more than 16-bit addressing" sounds today. Yet it was a valid 
qualification to add in the past, when 32- and 64-bit systems were a rarity.


 In the close future D std lib may enjoy to grow several immutable
 data structures (finger trees, etc), even if the D2 language &
 community on the whole has not embraced the immutability much yet.
 Such data structures and other features in the std lib can't hurt.
 We'll surely talk more about this topic.


Looking forward to your contributions to Phobos. Just drop 
pass-by-delegate like a bad habit. :o)


Andrei
Mar 02 2009
prev sibling parent reply grauzone <none example.net> writes:
 Sometimes, you have to leave the more spiffy features to languages 
 like Erlang or Haskell which are suited better for this. For example, 
 Erlang doesn't share any state by default, and in Haskell, everything 
 is immutable. But they use very special mechanisms to make up for the 
 problems caused by this, and that's why you can't just pick the 
 cherries: it won't fit, it will feel unnatural, and everything will be 
 a mess. It's a good idea to copy good features from other languages, 
 but never forget where you come from.


 
 We plan to add the necessary mechanisms.


What mechanisms will these be? I'm sure others are curious too.
Mar 02 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:

 Sometimes, you have to leave the more spiffy features to languages 
 like Erlang or Haskell which are suited better for this. For example, 
 Erlang doesn't share any state by default, and in Haskell, everything 
 is immutable. But they use very special mechanisms to make up for the 
 problems caused by this, and that's why you can't just pick the 
 cherries: it won't fit, it will feel unnatural, and everything will 
 be a mess. It's a good idea to copy good features from other 
 languages, but never forget where you come from.


 We plan to add the necessary mechanisms.


 
 What mechanisms will these be? I'm sure others are curious too.


Still under discussion. We're considering for example message-passing 
queues a la Erlang.

Andrei
Mar 02 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 We're considering for example message-passing queues a la Erlang.<


Actors, like in Erlang and Scala? (Please ignore this comment if it misses the
point of this discussion).

Bye,
bearophile
Mar 02 2009
prev sibling next sibling parent dennis luehring <dl.soluz gmx.net> writes:
 Still under discussion. We're considering for example message-passing 
 queues a la Erlang.


yes yes yes ...
Mar 02 2009
prev sibling parent Christopher Wright <dhasenan gmail.com> writes:
Andrei Alexandrescu wrote:

 grauzone wrote:

 Sometimes, you have to leave the more spiffy features to languages 
 like Erlang or Haskell which are suited better for this. For 
 example, Erlang doesn't share any state by default, and in Haskell, 
 everything is immutable. But they use very special mechanisms to 
 make up for the problems caused by this, and that's why you can't 
 just pick the cherries: it won't fit, it will feel unnatural, and 
 everything will be a mess. It's a good idea to copy good features 
 from other languages, but never forget where you come from.


 We plan to add the necessary mechanisms.


 What mechanisms will these be? I'm sure others are curious too.


 
 Still under discussion. We're considering for example message-passing 
 queues a la Erlang.
 
 Andrei


A library-level feature, I hope, but I'd be quite interested in seeing 
how this would be implemented as a pure function.
Mar 03 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
grauzone wrote:

 Does this approach fit D at all? D is a C based system programming 
 language. Does it work to demand from everyone to obey the const rules? 
 Couldn't all this be done manually, instead of bothering the programmer 
 with a complex type system?


Doing it manually is how it's done in C. And the results are it is a 
massive failure. It's just impractical to write reliable, portable, 
inspectable, and maintainable multithreaded code in C. Even the experts 
can't get the trivial cases right, as there was a famous recent example 
in Dr. Dobb's (forgot the name of the article at the moment).
Mar 02 2009
parent reply grauzone <none example.net> writes:
Walter Bright wrote:

 grauzone wrote:

 Does this approach fit D at all? D is a C based system programming 
 language. Does it work to demand from everyone to obey the const 
 rules? Couldn't all this be done manually, instead of bothering the 
 programmer with a complex type system?


 
 Doing it manually is how it's done in C. And the results are it is a 
 massive failure. It's just impractical to write reliable, portable, 
 inspectable, and maintainable multithreaded code in C. Even the experts 
 can't get the trivial cases right, as there was a famous recent example 
 in Dr. Dobb's (forgot the name of the article at the moment).


I didn't mean going back to programming with locks. Instead you could 
use the new ideas without extending the type system. As far as I 
understand, the language extensions are only needed for verification (so 
far).
Mar 02 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
grauzone wrote:

 I didn't mean going back to programming with locks. Instead you could 
 use the new ideas without extending the type system. As far as I 
 understand, the language extensions are only needed for verification (so 
 far).


Without verification, it's programming by hopeful convention. If you 
want a reliable system, you need more than hope <g>.
Mar 02 2009
parent reply hasen <hasan.aljudy gmail.com> writes:
Walter Bright wrote:

 grauzone wrote:

 I didn't mean going back to programming with locks. Instead you could 
 use the new ideas without extending the type system. As far as I 
 understand, the language extensions are only needed for verification 
 (so far).


 
 Without verification, it's programming by hopeful convention. If you 
 want a reliable system, you need more than hope <g>.


Well .. if you think about OOP and private/public ..

Dynamic languages like python and smalltalk don't enforce 
private/public, and that never was a problem. And, smalltalk is *the* OO 
language (AFAIK)

(this is not really an argument against const per se, it's just an 
argument against an argument for const)
Mar 03 2009
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
hasen wrote:


 Walter Bright wrote:

 grauzone wrote:

 I didn't mean going back to programming with locks. Instead you could 
 use the new ideas without extending the type system. As far as I 
 understand, the language extensions are only needed for verification 
 (so far).


 
 Without verification, it's programming by hopeful convention. If you 
 want a reliable system, you need more than hope <g>.


 
 Well .. if you think about OOP and private/public ..
 
 Dynamic languages like python and smalltalk don't enforce 
 private/public, and that never was a problem. And, smalltalk is *the* OO 
 language (AFAIK)
 
 (this is not really an argument against const per se, it's just an 
 argument against an argument for const)


If you have a dynamic language you have a different way of programming. In D 
when I make a typo, the compiler catches it. When I do the same in Ruby, I 
have a unit test that spits out a method missing exception with a trace. 
Suppose D doesn't catch my typo and then my application crashes at runtime 
without such a trace, that will be a nightmare.
Mar 04 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Lutger:

 If you have a dynamic language you have a different way of programming. In D 
 when I make a typo, the compiler catches it. When I do the same in Ruby, I 
 have a unit test that spits out a method missing exception with a trace. 
 Suppose D doesn't catch my typo and then my application crashes at runtime 
 without such a trace, that will be a nightmare. 


This is a complex and long debated topic.
In my D programs I put almost as many unittests as I put in Python programs,
because experience shows me the type system of D doesn't catch that many bugs.
The result seems strong enough D programs.
While I debug I use the 'Phobos hack' to add something like those traces to D
(I don't understand why such feature isn't built-in yet, it's essential when I
debug D programs).

Bye,
bearophile
Mar 04 2009
next sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
bearophile wrote:

 Lutger:

 If you have a dynamic language you have a different way of programming. In D 
 when I make a typo, the compiler catches it. When I do the same in Ruby, I 
 have a unit test that spits out a method missing exception with a trace. 
 Suppose D doesn't catch my typo and then my application crashes at runtime 
 without such a trace, that will be a nightmare. 


 
 This is a complex and long debated topic.
 In my D programs I put almost as many unittests as I put in Python programs,
because experience shows me the type system of D doesn't catch that many bugs.
The result seems strong enough D programs.
 While I debug I use the 'Phobos hack' to add something like those traces to D
(I don't understand why such feature isn't built-in yet, it's essential when I
debug D programs).


Agreed on testing, and agreed on stack traces.

However, how do you test the unit test system?
Mar 04 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Christopher Wright:

However, how do you test the unit test system?<


I don't understand the question.

Bye,
bearophile
Mar 04 2009
parent Christopher Wright <dhasenan gmail.com> writes:
bearophile wrote:

 Christopher Wright:

 However, how do you test the unit test system?<


 
 I don't understand the question.
 
 Bye,
 bearophile


If you have a system that runs unittests, how do you make sure that it 
is not buggy? You need to devise another test system.

It's not a question that most people should care about.
Mar 04 2009
prev sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
bearophile wrote:


 Lutger:

 If you have a dynamic language you have a different way of programming. 




In D 

 when I make a typo, the compiler catches it. When I do the same in Ruby, 




I 

 have a unit test that spits out a method missing exception with a trace. 
 Suppose D doesn't catch my typo and then my application crashes at 




runtime 

 without such a trace, that will be a nightmare. 


 
 This is a complex and long debated topic.
 In my D programs I put almost as many unittests as I put in Python 


programs, because experience shows me the type system of D doesn't catch 
that many bugs. The result seems strong enough D programs.

 While I debug I use the 'Phobos hack' to add something like those traces 


to D (I don't understand why such feature isn't built-in yet, it's essential 
when I debug D programs).

 
 Bye,
 bearophile


I didn't mean to imply that with D the type system unit tests and stack 
traces are obsolete or that D is better / worse.

When you say 'not that many bugs', how often does you code compile with 
errors? Because every compile time error you encounter is a bug in a dynamic 
language, and within that perspective I think it's quite a lot. Now, that 
doesn't mean dynamic languages are more bug prone, just that the way you 
program with them is different. Different enough that you can't compare type 
systems of static and dynamic languages that easily.
Mar 04 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Lutger:


When you say 'not that many bugs', how often does you code compile with errors?<


I am not able to give you a good answer, I am sorry, so I can just chat a
little.
What I can tell you is that I now know both Python and D (I know Python more,
but my experience of D is getting sufficient), and I am usually able to write
correct programs in quite less time in Python. "correct programs" includes unit
tests, debugging, global testing, etc too. It's not easy to tell when a program
is "debugged enough", it's an art.
To write a "correct" program in C I need even more time and sometimes at the
end I am not sure yet the program is debugged enough (despite I know C for far
more time than D). This is what I care of.



Because every compile time error you encounter is a bug in a dynamic language,
and within that perspective I think it's quite a lot.<


The situation isn't so linear. In Python I often don't put many bugs in the
first place, because syntax and semantics is less bug prone or more flexible,
etc.
In Python I have the shell, where I can try out single lines of code or whole
functions, that helps both in writing correct code and in debugging it (and
later you can even copy&paste such interactive sessions, with small changes, to
use them as doctests).



Now, that doesn't mean dynamic languages are more bug prone, just that the way
you program with them is different. Different enough that you can't compare
type systems of static and dynamic languages that easily.<


I agree. Yet, I have often written the same small programs in both D and Python
so I can compare the whole programming time, how much long they are, how much
reliable they are, etc. I have adopted in my D coding some of the good
practices I have developed in Python (regarding unit testing, functional-style
programming, using higher-level functions, using templates to make more
flexible functions), and the results seems good enough :-)

Bye,
bearophile
Mar 04 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
bearophile wrote:

 What I can tell you is that I now know both Python and
 D (I know Python more, but my experience of D is getting sufficient),
 and I am usually able to write correct programs in quite less time in
 Python. "correct programs" includes unit tests, debugging, global
 testing, etc too. It's not easy to tell when a program is "debugged
 enough", it's an art. To write a "correct" program in C I need even
 more time and sometimes at the end I am not sure yet the program is
 debugged enough (despite I know C for far more time than D).


This is definitely strongly related to experience. I find that over time 
coding in any language, I gradually write fewer and fewer bugs. I 
essentially learn how to avoid the ones peculiar to that particular 
language.

A lot of influence on D comes from trying to design out of existence 
particular bugs in C/C++ that cost me a lot of time and effort to track 
down. I tend to not make those errors anymore in my C/C++ code, but the 
taste is still bitter and the learning curve is long. Others shouldn't 
have to follow the same path.
Mar 04 2009
prev sibling parent hasen <hasan.aljudy gmail.com> writes:
Jason House wrote:

 Given the tone of your messages, I assume convincing you is impossible. The
biggest help I can offer is to tell you that "the D you loved" still exists as
D 1.0... Walter still maintains the compiler for it and Tango, Descent, GDC,
LDC, and many other things all support D1.
  


Just a quick note, grauzone is not the OP, it's me :)
Mar 02 2009
prev sibling next sibling parent BCS <none anon.com> writes:
Hello hasen,


 I haven't been following D for over a year .. now I notice there's a
 const!!
 


[...]

 
 This is annoying because now const propagates like a virus!


Yes const is viral (that can't be avoided)


 This whole stupid process wouldn't even be needed in the first place
 if C++ had a garbage collector, because then I would always "new" them

 
 SomeType *obj = new SomeType();
 
 but because there's no garbage collector, I have to create the object
 not as a reference.
 
 SomeType obj();
 
 Again, this is all stupid C++ stuff that shouldn't even be needed in
 the first place.
 


IIRC const has exactly zero to do with new/reference/value stuff. It is totaly 
about knowing if somthing will get changed out from under you. This is needed 
to make unsable large scale libraries. And has a HUGE impact on parallel 
code.


 However, with all that being said, that's just my opinion, maybe over
 the time some people found some actually useful uses for const, great,
 so they can use it if they want.
 


[...]

 
 char[] p = "world"; // error, cannot implicitly convert invariant
 // to mutable
 and all I can think of is: WHAT - THE - HELL??!!!!!!


That's totally correct. On linux under D1.0 (that would work BTW) if you 
then try to alter p you get an access violation. String literals can't be 
modified and that what const/invariant says.
Mar 01 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 02 Mar 2009 06:28:24 +0300, hasen <hasan.aljudy gmail.com> wrote:


 I haven't been following D for over a year .. now I notice there's a  
 const!!

 In C++, in my experience, the only time I need const is when I pass  
 temporary objects by reference and the compiler refuses, so I make the  
 parameter const to make the compiler happy.

 void my_func( const SomeType& obj ) { ... }

 This is annoying because now const propagates like a virus! any function  
 that I call on `obj` must also be changed to accept a `const` reference.  
 any  method on obj which doesn't change it must also be marked as  
 `const` in order for it to be callable from inside this function.

 This whole stupid process wouldn't even be needed in the first place if  
 C++ had a garbage collector, because then I would always "new" them (as  


 SomeType *obj = new SomeType();

 but because there's no garbage collector, I have to create the object  
 not as a reference.

 SomeType obj();

 Again, this is all stupid C++ stuff that shouldn't even be needed in the  
 first place.

 However, with all that being said, that's just my opinion, maybe over  
 the time some people found some actually useful uses for const, great,  
 so they can use it if they want.

 What really annoys me is the viral nature of const.

 Yesterday I was reading this:  
 http://www.digitalmars.com/d/2.0/const3.html

 (btw, this page is empty: http://www.digitalmars.com/d/2.0/const.html )

 and, to my surprise, I see:

 char[] p = "world"; // error, cannot implicitly convert invariant
 		    // to mutable


 and all I can think of is: WHAT - THE - HELL??!!!!!!

 invariant(char)[] is ugly! might as well be saying std::vector<char>  
 (ok, not the best example of stupidity, but I hope you get my point).
 (and, is invariant(char)[] the correct type? or must it be  
 invariant(char)[5]??)

 This is not the D that I loved. What's going on?

 P.S. look here  
 http://stackoverflow.com/questions/557011/d-programming-language-char-arrays/

 I know you can use "auto" and all that, but still .. I don't feel good  
 about this.


I you don't understand something, it doesn't mean something is wrong with the
language. Perhaps, something wrong with you.

Regarding this exaple:

char[] p = "world";

It is rejected for a reason. Modifying the "world" through p will cause a
segfault anyway. So it *must* be const. In case it won't, it will cause bugs
that are even worse to track:

// file 1
char[] a = "hello";
a[4] = '!';

// file 2
writefln("hello"); // prints 'hell!'
Mar 02 2009
prev sibling next sibling parent reply hasen <hasan.aljudy gmail.com> writes:
hasen wrote:

 I haven't been following D for over a year .. now I notice there's a 
 const!!
 
 [...]


It seems the argument is centered around making "pure functions" possible.

But, surely there's a better way of doing this than borrowing const from 
C++.

Just like import is a better approach than #include and `namespace`

Yesterday I noticed this page 
http://www.digitalmars.com/d/2.0/const-faq.html
which I haven't seen before posting the topic yesterday.


which becomes increasingly important the more people that are involved 
with the code. In other words, it scales very well."





Languages that make it easy to program them will supplant languages that 
don't."

So, this is driven by the need to popularize D as a very good tool for 
multicore applications!

"Transitive const is key to bringing D into this paradigm."

Really? have you considered other possibilities?

How about, adding a new attribute to functions: `pure`

pure real sin( real x ) { ... }

and design the rest around this concept.

The compiler must make sure that this function is really pure:
- native types must be passed by value, not by reference
- doesn't accept pointers
- <insert some condition for objects and arrays>

If these requirements aren't met, the compiler will spit some error 
message "in function <F>: doing <X> voilates the `pure` attribute"

objects and arrays will need some special treatment or requirements in 
order to be passed to pure functions.
What are those requirements? I have given it a very deep thought yet(*), 
but I'm sure this approach is better than `const`.

Also, make `string` a builtin type that's immutable (like python), and 
discourage the use of char[] as a type for strings, (unless used to 
implement a special string class).

The need for a special string type also stems from the fact that 
char[a..b] slices bytes, not characters!


(*) A quick idea might be:
- For arrays, any statement of the type `a[b] = c` will be illegal 
inside a pure function. and pure functions can only call pure functions.
- For objects, any statement of the form `a.b = c` is illegal, and there 
must be a way to know if a method call will change the object or not. 
(It would be best if the compiler could detect this automatically if 
it's possible).
- Another approach, is to prohibit passing objects all together, and 
introduce some new construct that's immutable (such as a tuple, again, 
like python).

`pure` will probably have the same viral effects as `const`, in that any 
function called from a pure function must also be pure, and this viral 
nature will propagate to classes/objects as well.

However, the advantage is not complicating the type system more than is 
needed.

Also, what's the deal with const pointers?? Why should `pure` function 
be able to use pointers at all? Is there any real-life use case where a 
pure function needs to access memory instead of some abstract concept 
like a variable/array/tuple?


If you think about it, CTFE (compile time function execution) are `pure` 
already, and they're detected automatically by the compiler, with no 
need for explicitly marking them as such.
Mar 02 2009
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
hasen:

 Really? have you considered other possibilities?
 How about, adding a new attribute to functions: `pure`


D2 already implements of plans to implement both transitive immutability, pure
functions and to make by default data not accessible between threads :-)
See pure here:
http://www.digitalmars.com/d/2.0/function.html
(And in the future it will probably adopt even more immutability-related idioms
and data structures).

Bye,
bearophile
Mar 02 2009
prev sibling next sibling parent Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Mon, Mar 2, 2009 at 1:32 PM, hasen <hasan.aljudy gmail.com> wrote:


 How about, adding a new attribute to functions: `pure`

 pure real sin( real x ) { ... }

 and design the rest around this concept.

 The compiler must make sure that this function is really pure:
 - native types must be passed by value, not by reference
 - doesn't accept pointers
 - <insert some condition for objects and arrays>

 If these requirements aren't met, the compiler will spit some error message
 "in function <F>: doing <X> voilates the `pure` attribute"


Umm.. maybe you didn't read far enough, but D2 already has pure functions.
http://www.digitalmars.com/d/2.0/function.html


 Also, make `string` a builtin type that's immutable (like python), and
 discourage the use of char[] as a type for strings, (unless used to
 implement a special string class).

 The need for a special string type also stems from the fact that char[a..b]
 slices bytes, not characters!


Again, 'string' is defined as an alias to invariant(char)[].  It's an
immutable builtin type.  It still has the problem of slicing/indexing
on bytes rather than characters, but the alias is already there,
Phobos already uses it extensively, and if it were changed to a struct
in the future it would probably cause few to no problems.
Mar 02 2009
prev sibling next sibling parent BCS <ao pathlink.com> writes:
Reply to hasen,


 "Transitive const is key to bringing D into this paradigm."
 
 Really? have you considered other possibilities?
 
 How about, adding a new attribute to functions: `pure`
 
 pure real sin( real x ) { ... }
 
 and design the rest around this concept.


A lot of the current const system IS designed around plans to add pure
functions. 
You can almost read immutable as "valid as an arg for a pure function call."


 
 The compiler must make sure that this function is really pure:
 - native types must be passed by value, not by reference
 - doesn't accept pointers
 - <insert some condition for objects and arrays>
 If these requirements aren't met, the compiler will spit some error
 message "in function <F>: doing <X> voilates the `pure` attribute"
 


Um... that would be even more restrictive than const. Outside pointers const 
is more or less irrelevant with regards to function arguments so you might 
as well have said "pure functions can't take anything that is const or CAN 
be const"


 objects and arrays will need some special treatment or requirements in
 order to be passed to pure functions.


Const is a lot of very smart peoples best effort to do that special treatment.



 (*) A quick idea might be:
 - For arrays, any statement of the type `a[b] = c` will be illegal
 inside a pure function. and pure functions can only call pure
 functions.


pure int fn()
{
    int[5] foo;
    foo[0] = 5;  /// why ban this
    return foo[0];
}


 However, the advantage is not complicating the type system more than
 is needed.
 
 Also, what's the deal with const pointers?? Why should `pure` function
 be able to use pointers at all? Is there any real-life use case where
 a pure function needs to access memory instead of some abstract
 concept like a variable/array/tuple?


Say I have several pre computed multi-MB tables that need to be used by a 
function. The tables never change. The function is still pure but I really 
can't pass the table on the stack. The same thing to a lesser degree happens 
with any reference type that is much larger than a pointer.
Mar 02 2009
prev sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
hasen wrote:


 hasen wrote:

 I haven't been following D for over a year .. now I notice there's a 
 const!!
 
 [...]


 
 It seems the argument is centered around making "pure functions" possible.
 
 But, surely there's a better way of doing this than borrowing const from 
 C++.


Reading the docs you must have noticed the difference between C++ const and 
D. In D, const and immutable are transitive, in C++ they are not. D has 
immutable, C++ does not. These two are the most important concepts and make 
the D design that much different, you can't seriously claim it is borrowed 
from C++. 

Const in C++ is (merely) a compiler aided documentation tool, whereas 
const+immutable in D are the basis of a functional programming niche inside 
D.
Mar 02 2009
prev sibling next sibling parent Jason House <jason.james.house gmail.com> writes:
hasen Wrote:


 hasen wrote:

 I haven't been following D for over a year .. now I notice there's a 
 const!!
 
 [...]


 
 It seems the argument is centered around making "pure functions" possible.
 
 But, surely there's a better way of doing this than borrowing const from 
 C++.
 
 Just like import is a better approach than #include and `namespace`
 
 Yesterday I noticed this page 
 http://www.digitalmars.com/d/2.0/const-faq.html
 which I haven't seen before posting the topic yesterday.
 

 which becomes increasingly important the more people that are involved 
 with the code. In other words, it scales very well."
 


 

 Languages that make it easy to program them will supplant languages that 
 don't."
 
 So, this is driven by the need to popularize D as a very good tool for 
 multicore applications!
 
 "Transitive const is key to bringing D into this paradigm."
 
 Really? have you considered other possibilities?
 
 How about, adding a new attribute to functions: `pure`
 
 pure real sin( real x ) { ... }
 
 and design the rest around this concept.
 
 The compiler must make sure that this function is really pure:
 - native types must be passed by value, not by reference
 - doesn't accept pointers
 - <insert some condition for objects and arrays>
 
 If these requirements aren't met, the compiler will spit some error 
 message "in function <F>: doing <X> voilates the `pure` attribute"
 
 objects and arrays will need some special treatment or requirements in 
 order to be passed to pure functions.
 What are those requirements? I have given it a very deep thought yet(*), 
 but I'm sure this approach is better than `const`.
 
 Also, make `string` a builtin type that's immutable (like python), and 
 discourage the use of char[] as a type for strings, (unless used to 
 implement a special string class).
 
 The need for a special string type also stems from the fact that 
 char[a..b] slices bytes, not characters!
 
 
 (*) A quick idea might be:
 - For arrays, any statement of the type `a[b] = c` will be illegal 
 inside a pure function. and pure functions can only call pure functions.
 - For objects, any statement of the form `a.b = c` is illegal, and there 
 must be a way to know if a method call will change the object or not. 
 (It would be best if the compiler could detect this automatically if 
 it's possible).
 - Another approach, is to prohibit passing objects all together, and 
 introduce some new construct that's immutable (such as a tuple, again, 
 like python).
 
 `pure` will probably have the same viral effects as `const`, in that any 
 function called from a pure function must also be pure, and this viral 
 nature will propagate to classes/objects as well.
 
 However, the advantage is not complicating the type system more than is 
 needed.
 
 Also, what's the deal with const pointers?? Why should `pure` function 
 be able to use pointers at all? Is there any real-life use case where a 
 pure function needs to access memory instead of some abstract concept 
 like a variable/array/tuple?
 
 
 If you think about it, CTFE (compile time function execution) are `pure` 
 already, and they're detected automatically by the compiler, with no 
 need for explicitly marking them as such.
 



You're getting closer...

Your scheme uses a lot of deep copying which can kill performance. What if
there was a way to skip such deep copies? Or at least force the caller to copy
the data before use? How do you implement thread safe deep copies?
Mar 02 2009
prev sibling next sibling parent reply Jason House <jason.james.house gmail.com> writes:
Andrei Alexandrescu Wrote:


 grauzone wrote:

 Sometimes, you have to leave the more spiffy features to languages 
 like Erlang or Haskell which are suited better for this. For example, 
 Erlang doesn't share any state by default, and in Haskell, everything 
 is immutable. But they use very special mechanisms to make up for the 
 problems caused by this, and that's why you can't just pick the 
 cherries: it won't fit, it will feel unnatural, and everything will 
 be a mess. It's a good idea to copy good features from other 
 languages, but never forget where you come from.


 We plan to add the necessary mechanisms.


 
 What mechanisms will these be? I'm sure others are curious too.


 
 Still under discussion. We're considering for example message-passing 
 queues a la Erlang.
 
 Andrei


Is there any chance of you/Walter posting your current conclusions and open
issues? There are many on this list, including myself, who would enjoy
participating in the design.
Mar 03 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jason House wrote:

 Andrei Alexandrescu Wrote:
 

 grauzone wrote:

 Sometimes, you have to leave the more spiffy features to
 languages like Erlang or Haskell which are suited better for
 this. For example, Erlang doesn't share any state by default,
 and in Haskell, everything is immutable. But they use very
 special mechanisms to make up for the problems caused by
 this, and that's why you can't just pick the cherries: it
 won't fit, it will feel unnatural, and everything will be a
 mess. It's a good idea to copy good features from other 
 languages, but never forget where you come from.


 We plan to add the necessary mechanisms.


 What mechanisms will these be? I'm sure others are curious too.


 Still under discussion. We're considering for example
 message-passing queues a la Erlang.
 
 Andrei


 
 Is there any chance of you/Walter posting your current conclusions
 and open issues? There are many on this list, including myself, who
 would enjoy participating in the design.


Most of the burden is on Bartosz.

Andrei
Mar 03 2009
prev sibling next sibling parent reply novice2 <sorry noem.ail> writes:
Denis Koroskin Wrote:


 // file 1
 char[] a = "hello";
 a[4] = '!';
 
 // file 2
 writefln("hello"); // prints 'hell!'



bad example :)
why so behaviour? imho, it may be occur bacause of bad compiler design only.
like C compiler for space optimization collect all mentioning of similar
strings and allocate only one for all of occurence. and even in this case it
must allocate it in non-writable memory section.
Mar 03 2009
parent "Denis Koroskin" <2korden gmail.com> writes:
On Wed, 04 Mar 2009 08:55:18 +0300, novice2 <sorry noem.ail> wrote:


 Denis Koroskin Wrote:


 // file 1
 char[] a = "hello";
 a[4] = '!';

 // file 2
 writefln("hello"); // prints 'hell!'



 bad example :)
 why so behaviour? imho, it may be occur bacause of bad compiler design  
 only. like C compiler for space optimization collect all mentioning of  
 similar strings and allocate only one for all of occurence. and even in  
 this case it must allocate it in non-writable memory section.


Yeah, that's why it is forbidden. It won't compile in D2.
Mar 03 2009
prev sibling next sibling parent reply Burton Radons <burton.radons gmail.com> writes:
I don't get it either. Any possible application for const in the form of code
correctness went out the window once the invariant virus forced all strings to
be invariant whether they were or not; so I still need to use dup to guarantee
that data won't change underneath me, but then I need to cast it! It doesn't
affect pure at all, because pure can be passed invariants which are just casted
- the compiler needs to use rules which are a hell of a lot more binding than
anything we can provide it to make these determinations. Now that const is not
a storage class, it's actually not possible to declare function variables which
should be stored in read-only memory (unless if it's encoded somewhere in the
thirty or so combinations you can use), which also damages pure. It's a lot
more confusing to deal with const data altogether than it used to be.

When I switched from D 1.0 to 2.0 I tried several times to port some large
pieces of code over and ultimately gave up, just as everyone has given up
trying to do it in C++. It's a hard task moving code through that kind of
change.

I've learned to handle it but I would really like to not be fighting the
compiler all the time. Is that what I'm supposed to be doing here, really?
Mar 05 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Burton Radons wrote:

 I don't get it either. Any possible application for const in the form
 of code correctness went out the window once the invariant virus
 forced all strings to be invariant whether they were or not; so I
 still need to use dup to guarantee that data won't change underneath
 me, but then I need to cast it!


My experience is that when dup is used to guarantee that the data won't 
change underneath the problem is being addressed at the wrong end. The 
mutator of the data dups it, not the user.


 It doesn't affect pure at all,
 because pure can be passed invariants which are just casted - the
 compiler needs to use rules which are a hell of a lot more binding
 than anything we can provide it to make these determinations. Now
 that const is not a storage class, it's actually not possible to
 declare function variables which should be stored in read-only memory
 (unless if it's encoded somewhere in the thirty or so combinations
 you can use), which also damages pure. It's a lot more confusing to
 deal with const data altogether than it used to be.
 
 When I switched from D 1.0 to 2.0 I tried several times to port some
 large pieces of code over and ultimately gave up, just as everyone
 has given up trying to do it in C++. It's a hard task moving code
 through that kind of change.
 
 I've learned to handle it but I would really like to not be fighting
 the compiler all the time. Is that what I'm supposed to be doing
 here, really?


Can you be more specific about what was stimying you, as perhaps we can 
think of a solution.
Mar 05 2009
parent reply Burton Radons <burton.radons gmail.com> writes:
Walter Bright Wrote:


 Burton Radons wrote:

 I don't get it either. Any possible application for const in the form
 of code correctness went out the window once the invariant virus
 forced all strings to be invariant whether they were or not; so I
 still need to use dup to guarantee that data won't change underneath
 me, but then I need to cast it!


 
 My experience is that when dup is used to guarantee that the data won't 
 change underneath the problem is being addressed at the wrong end. The 
 mutator of the data dups it, not the user.


I don't know the origin of the string or where it's going. I only know it's
"invariant (char) []" now, because you're required to have it "invariant (char)
[]" because of the viral nature of const and that's how the string is defined,
so that's what my API takes. It certainly doesn't mean that anyone is
intentionally using the API incorrectly; everything takes "invariant (char)
[]", so my function doesn't deserve special treatment without stating so.

A more accurate way would be for the string type to be "const (char) []", and
for functions which retain strings that can't change to take "invariant (char)
[]". That makes pretty good claims about the nature of the string, but it would
clearly result in lots of cast management.

I think all these problems boil down to the fact that invariant tells you about
the container rather than the object itself; but whether the object actually is
invariant is totally independent of the container. The genius of D 1.0's const
is that it makes an actual, actionable true statement about the object. That
was a HUGE benefit. This tries to push it back into C territory and I don't
think it works.

I don't buy that this is going to lead to any MP bonuses either. Const has
turned into a form of the sufficiently complex compiler fallacy, the cure for
any ail. But if you can cast const on or off, then any declaration you make
about the mutability of the data is going to be wrong at some point, and
compilers can't sometimes build the right code.


 It doesn't affect pure at all,
 because pure can be passed invariants which are just casted - the
 compiler needs to use rules which are a hell of a lot more binding
 than anything we can provide it to make these determinations. Now
 that const is not a storage class, it's actually not possible to
 declare function variables which should be stored in read-only memory
 (unless if it's encoded somewhere in the thirty or so combinations
 you can use), which also damages pure. It's a lot more confusing to
 deal with const data altogether than it used to be.
 
 When I switched from D 1.0 to 2.0 I tried several times to port some
 large pieces of code over and ultimately gave up, just as everyone
 has given up trying to do it in C++. It's a hard task moving code
 through that kind of change.
 
 I've learned to handle it but I would really like to not be fighting
 the compiler all the time. Is that what I'm supposed to be doing
 here, really?


 
 Can you be more specific about what was stimying you, as perhaps we can 
 think of a solution.


Exactly the same thing as trying to do it in C++. You're stuck iteratively
applying const to pretty much everything, and applying tons of casts to get the
compiler to shut up. D does provide the mechanisms to have proper safe
mutableCast/constCast/invariantCast functions (well, safe except that they make
it so that neither compiler nor programmer can make any good statements about
the nature of the data), but when trying to shift code around you usually just
want to make it work.

I remember that one thing that really got to me was that you had a foreach term
as invariant at that point if it weren't defined as inout. The thing is, that
was correct and could've led you to a sweet easy compiler optimisation, but
it's like const is in this balance where properly implemented it would be
brutal and ugly, so we need to back everything off until it's only at an
acceptable level of annoyance.

Oh, and templates. Templates that have nothing to do with any of these matters
keep on having to be changed to appease the const god. Dealing with templates
when you're doing really crazy stuff's bad enough without having that around.
Mar 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Burton Radons wrote:

 A more accurate way would be for the string type to be "const (char)
 []", and for functions which retain strings that can't change to take
 "invariant (char) []". That makes pretty good claims about the nature
 of the string, but it would clearly result in lots of cast
 management.


I use that all the time, it's a great idiom. What cast management needs 
to be done? What I need to do is occasionally insert an .idup on the 
client side because the callee wants a copy. So that's that.


 I think all these problems boil down to the fact that invariant tells
 you about the container rather than the object itself; but whether
 the object actually is invariant is totally independent of the
 container. The genius of D 1.0's const is that it makes an actual,
 actionable true statement about the object. That was a HUGE benefit.
 This tries to push it back into C territory and I don't think it
 works.


I don't think I understand most of this, possibly because some of it is 
wrong. D2's immutable does offer a solid guarantee about what's going on 
and offers a programming model that makes it easy to write correct code 
without undue aliasing. So C doesn't quite enter into the picture there.

Objects in a container being invariant tell a lot about the container. 
That property makes the container shareable without a risk.


 I don't buy that this is going to lead to any MP bonuses either.


Wait and see.


 Const has turned into a form of the sufficiently complex compiler
 fallacy, the cure for any ail. But if you can cast const on or off,
 then any declaration you make about the mutability of the data is
 going to be wrong at some point, and compilers can't sometimes build
 the right code.


In D you will be able to break any design with a cast, unless you use 
the not-yet-defined D2 which disallows all risky casts. So the fact that 
you can cast const away is hardly changing anything.


 It doesn't affect pure at all, because pure can be passed
 invariants which are just casted - the compiler needs to use
 rules which are a hell of a lot more binding than anything we can
 provide it to make these determinations. Now that const is not a
 storage class, it's actually not possible to declare function
 variables which should be stored in read-only memory (unless if
 it's encoded somewhere in the thirty or so combinations you can
 use), which also damages pure. It's a lot more confusing to deal
 with const data altogether than it used to be.
 
 When I switched from D 1.0 to 2.0 I tried several times to port
 some large pieces of code over and ultimately gave up, just as
 everyone has given up trying to do it in C++. It's a hard task
 moving code through that kind of change.
 
 I've learned to handle it but I would really like to not be
 fighting the compiler all the time. Is that what I'm supposed to
 be doing here, really?


 Can you be more specific about what was stimying you, as perhaps we
 can think of a solution.


 
 Exactly the same thing as trying to do it in C++. You're stuck
 iteratively applying const to pretty much everything, and applying
 tons of casts to get the compiler to shut up.


But this is misusing const. It means it didn't belong there in the first 
place. Const (and immutable) in D appear less frequently than in C++ 
because they provide superior guarantees.


 Oh, and templates. Templates that have nothing to do with any of
 these matters keep on having to be changed to appease the const god.
 Dealing with templates when you're doing really crazy stuff's bad
 enough without having that around.


Agreed, qualifiers do make templates a tad harder to define. Fortunately 
std.traits.Unqual (to be released) can be helpful there.


Andrei
Mar 06 2009
next sibling parent reply Burton Radons <burton.radons shaw.ca> writes:
Andrei Alexandrescu Wrote:


 Burton Radons wrote:

 A more accurate way would be for the string type to be "const (char)
 []", and for functions which retain strings that can't change to take
 "invariant (char) []". That makes pretty good claims about the nature
 of the string, but it would clearly result in lots of cast
 management.


 
 I use that all the time, it's a great idiom. What cast management needs 
 to be done? What I need to do is occasionally insert an .idup on the 
 client side because the callee wants a copy. So that's that.


So long as the object definition of string is "invariant (char) []", I can't
guarantee anything about the nature of the object because you need to cast to
"invariant (char) []" to be able to interface with any API.

The good side is that when I changed it to be defined as "const (char) []" only
one line of code made a squeak. That gives me solid actionable information. If
an API is declared as istring, then whatever you give it must not ever change.
If an API is declared as string, then whatever happens in there, it won't
change the data. Pretty good!


 I think all these problems boil down to the fact that invariant tells
 you about the container rather than the object itself; but whether
 the object actually is invariant is totally independent of the
 container. The genius of D 1.0's const is that it makes an actual,
 actionable true statement about the object. That was a HUGE benefit.
 This tries to push it back into C territory and I don't think it
 works.


 
 I don't think I understand most of this, possibly because some of it is 
 wrong. D2's immutable does offer a solid guarantee about what's going on 
 and offers a programming model that makes it easy to write correct code 
 without undue aliasing. So C doesn't quite enter into the picture there.
 
 Objects in a container being invariant tell a lot about the container. 
 That property makes the container shareable without a risk.
 

 I don't buy that this is going to lead to any MP bonuses either.


 
 Wait and see.


I don't need to wait and see when twenty years of "hinted optimisation" have
had predictable results. If the programmer can set an incorrect state doing
something which he's forced to do, then any compiler which uses this state as
an actual description about the situation will cause problems because the
compiler's opportunities to apply these optimisations will shift over the
course of the development of the program. Code works, add one line, code
doesn't work. Code works, try it on another machine, code doesn't work.

The only way I can see this working is if the compiler really did have a good
idea about the nature of the state, at which point the programmer's statements
are completely superfluous. That's not coincidentally exactly how we do these
optimisations now: determine whether the state is in such a way that we can do
this safely, and only then actually apply the optimisation.

That can involve automatic MP of a sort - I understand that's the way the
C-based MP systems work, where you tell it that the state of the program is
such and such before letting it go ahead.

Fully automatic unambiguous effective parallelisation requires that everything
the programmer says is true. It's not something that can be stuffed into a
language with pointers and external API calls. If you think you can do that,
fine. But you haven't, and nobody else has, to my knowledge.
Mar 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Burton Radons wrote:

 Andrei Alexandrescu Wrote:
 

 Burton Radons wrote:

 A more accurate way would be for the string type to be "const
 (char) []", and for functions which retain strings that can't
 change to take "invariant (char) []". That makes pretty good
 claims about the nature of the string, but it would clearly
 result in lots of cast management.


 I use that all the time, it's a great idiom. What cast management
 needs to be done? What I need to do is occasionally insert an .idup
 on the client side because the callee wants a copy. So that's that.
 


 
 So long as the object definition of string is "invariant (char) []",
 I can't guarantee anything about the nature of the object because you
 need to cast to "invariant (char) []" to be able to interface with
 any API.
 
 The good side is that when I changed it to be defined as "const
 (char) []" only one line of code made a squeak. That gives me solid
 actionable information. If an API is declared as istring, then
 whatever you give it must not ever change. If an API is declared as
 string, then whatever happens in there, it won't change the data.
 Pretty good!


I have trouble following what you're saying. If what you're saying is 
essentially that in char[] is a better parameter definition than string 
for functions that don't need to escape their string argument, then yes, 
you are entirely right.

So what I recommend is:

void foo(in char[] s); // foo looks at s, doesn't escape it
void bar(string s); // bar needs to save s
void baz(char[] s); // baz needs to change s' contents


 I think all these problems boil down to the fact that invariant
 tells you about the container rather than the object itself; but
 whether the object actually is invariant is totally independent
 of the container. The genius of D 1.0's const is that it makes an
 actual, actionable true statement about the object. That was a
 HUGE benefit. This tries to push it back into C territory and I
 don't think it works.


 I don't think I understand most of this, possibly because some of
 it is wrong. D2's immutable does offer a solid guarantee about
 what's going on and offers a programming model that makes it easy
 to write correct code without undue aliasing. So C doesn't quite
 enter into the picture there.
 
 Objects in a container being invariant tell a lot about the
 container. That property makes the container shareable without a
 risk.
 

 I don't buy that this is going to lead to any MP bonuses either.


 Wait and see.


 
 I don't need to wait and see when twenty years of "hinted
 optimisation" have had predictable results. If the programmer can set
 an incorrect state doing something which he's forced to do, then any
 compiler which uses this state as an actual description about the
 situation will cause problems because the compiler's opportunities to
 apply these optimisations will shift over the course of the
 development of the program. Code works, add one line, code doesn't
 work. Code works, try it on another machine, code doesn't work.


You are completely, thoroughly losing me. I can only assume you are 
misunderstanding the role of const and immutable in manycore programming 
and build from there.


 The only way I can see this working is if the compiler really did
 have a good idea about the nature of the state, at which point the
 programmer's statements are completely superfluous. That's not
 coincidentally exactly how we do these optimisations now: determine
 whether the state is in such a way that we can do this safely, and
 only then actually apply the optimisation.
 
 That can involve automatic MP of a sort - I understand that's the way
 the C-based MP systems work, where you tell it that the state of the
 program is such and such before letting it go ahead.
 
 Fully automatic unambiguous effective parallelisation requires that
 everything the programmer says is true. It's not something that can
 be stuffed into a language with pointers and external API calls. If
 you think you can do that, fine. But you haven't, and nobody else
 has, to my knowledge.


I continue being lost. Words assemble in phrases, phrases assemble in 
sentences, sentences parse properly, but I can't understand one thing. I 
need to defer a response to others.


Andrei
Mar 06 2009
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On Fri, 06 Mar 2009 13:30:22 -0800, Andrei Alexandrescu wrote:


 Burton Radons wrote:

 Andrei Alexandrescu Wrote:
 

 Burton Radons wrote:

 A more accurate way would be for the string type to be "const (char)
 []", and for functions which retain strings that can't change to take
 "invariant (char) []". That makes pretty good claims about the nature
 of the string, but it would clearly result in lots of cast
 management.


 I use that all the time, it's a great idiom. What cast management
 needs to be done? What I need to do is occasionally insert an .idup on
 the client side because the callee wants a copy. So that's that.
 
 


 So long as the object definition of string is "invariant (char) []", I
 can't guarantee anything about the nature of the object because you
 need to cast to "invariant (char) []" to be able to interface with any
 API.
 
 The good side is that when I changed it to be defined as "const (char)
 []" only one line of code made a squeak. That gives me solid actionable
 information. If an API is declared as istring, then whatever you give
 it must not ever change. If an API is declared as string, then whatever
 happens in there, it won't change the data. Pretty good!


 
 I have trouble following what you're saying. If what you're saying is
 essentially that in char[] is a better parameter definition than string
 for functions that don't need to escape their string argument, then yes,
 you are entirely right.
 
 So what I recommend is:
 
 void foo(in char[] s); // foo looks at s, doesn't escape it
 void bar(string s); // bar needs to save s
 void baz(char[] s); // baz needs
 to change s' contents


I think what Burton is saying is by annointing immutable(char)[] as the 
type "string," you are essentially sending a message to developers that 
all strings should be immutable, and all *string parameters* should be 
declared immutable.  What this does is force developers who want to deal 
in const or mutable chars have to do lots of duplication or casting, 
which either makes your code dog slow, or makes your code break const.

Evidence is how (at least in previous releases) anything in Phobos that 
took an argument that was a utf8 string of characters used the parameter 
type "string", making it very difficult to use when you don't have string 
types.  If you want to find a substring in a string, it makes no sense 
that you first have to make the argument invariant.  a substring function 
isn't saving a pointer to that data.

I think the complaint is simply that string is defined as immutable(char)
[] and therefore is promoted as *the only* string type to use.  Using 
other forms (such as in char[] or const(char)[] or char[]) doesn't look 
like the argument is a string, when the word "string" is already taken to 
mean something else.

-Steve
Mar 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:

 On Fri, 06 Mar 2009 13:30:22 -0800, Andrei Alexandrescu wrote:
 

 Burton Radons wrote:

 Andrei Alexandrescu Wrote:


 Burton Radons wrote:

 A more accurate way would be for the string type to be "const (char)
 []", and for functions which retain strings that can't change to take
 "invariant (char) []". That makes pretty good claims about the nature
 of the string, but it would clearly result in lots of cast
 management.


 I use that all the time, it's a great idiom. What cast management
 needs to be done? What I need to do is occasionally insert an .idup on
 the client side because the callee wants a copy. So that's that.


 So long as the object definition of string is "invariant (char) []", I
 can't guarantee anything about the nature of the object because you
 need to cast to "invariant (char) []" to be able to interface with any
 API.

 The good side is that when I changed it to be defined as "const (char)
 []" only one line of code made a squeak. That gives me solid actionable
 information. If an API is declared as istring, then whatever you give
 it must not ever change. If an API is declared as string, then whatever
 happens in there, it won't change the data. Pretty good!


 I have trouble following what you're saying. If what you're saying is
 essentially that in char[] is a better parameter definition than string
 for functions that don't need to escape their string argument, then yes,
 you are entirely right.

 So what I recommend is:

 void foo(in char[] s); // foo looks at s, doesn't escape it
 void bar(string s); // bar needs to save s
 void baz(char[] s); // baz needs
 to change s' contents


 
 I think what Burton is saying is by annointing immutable(char)[] as the 
 type "string," you are essentially sending a message to developers that 
 all strings should be immutable, and all *string parameters* should be 
 declared immutable.  What this does is force developers who want to deal 
 in const or mutable chars have to do lots of duplication or casting, 
 which either makes your code dog slow, or makes your code break const.
 
 Evidence is how (at least in previous releases) anything in Phobos that 
 took an argument that was a utf8 string of characters used the parameter 
 type "string", making it very difficult to use when you don't have string 
 types.  If you want to find a substring in a string, it makes no sense 
 that you first have to make the argument invariant.  a substring function 
 isn't saving a pointer to that data.
 
 I think the complaint is simply that string is defined as immutable(char)
 [] and therefore is promoted as *the only* string type to use.  Using 
 other forms (such as in char[] or const(char)[] or char[]) doesn't look 
 like the argument is a string, when the word "string" is already taken to 
 mean something else.
 
 -Steve


I see. Phobos is being changed to accept in char[] instead of string
wherever applicable. As far as what the default "string" ought to be,
immutable(char)[] is the safest of the three so I think it should be that.

Andrei
Mar 06 2009
next sibling parent reply jerry quinn <jlquinn optonline.net> writes:
Andrei Alexandrescu Wrote:


 Steven Schveighoffer wrote:

 I think what Burton is saying is by annointing immutable(char)[] as the 
 type "string," you are essentially sending a message to developers that 
 all strings should be immutable, and all *string parameters* should be 
 declared immutable.  What this does is force developers who want to deal 
 in const or mutable chars have to do lots of duplication or casting, 
 which either makes your code dog slow, or makes your code break const.
 
 Evidence is how (at least in previous releases) anything in Phobos that 
 took an argument that was a utf8 string of characters used the parameter 
 type "string", making it very difficult to use when you don't have string 
 types.  If you want to find a substring in a string, it makes no sense 
 that you first have to make the argument invariant.  a substring function 
 isn't saving a pointer to that data.
 
 I think the complaint is simply that string is defined as immutable(char)
 [] and therefore is promoted as *the only* string type to use.  Using 
 other forms (such as in char[] or const(char)[] or char[]) doesn't look 
 like the argument is a string, when the word "string" is already taken to 
 mean something else.
 
 -Steve


 
 I see. Phobos is being changed to accept in char[] instead of string
 wherever applicable. As far as what the default "string" ought to be,
 immutable(char)[] is the safest of the three so I think it should be that.


So far this discussion has no examples.  Let me toss one out as a dart board to
see what folks think:

char[] s = get_some_data();  // accesses a large buffer that i don't want to
copy
size_t pos = my_find(s, "blah");
s[pos] = "c";

string s2 = "another string";
size_t pos2 = my_find(s2, "blah");
another_func(s2[pos2 .. pos2+4]);

size_t my_find(string haystack, string needle)
{
  // Perform some kind of search that is read-only
}

another_func(string s) {}

---

It seems like string is the obvious parameter type to use for my_find, but it
doesn't work because we're passing mutable data in.  Instead we have to use
const(char)[], which is less intuitive and uglier.

Or did I miss the thrust of the argument?
Mar 06 2009
parent reply Derek Parnell <derek psych.ward> writes:
On Fri, 06 Mar 2009 18:25:13 -0500, jerry quinn wrote:


 So far this discussion has no examples.  Let me toss one out
 as a dart board to see what folks think:
 
 char[] s = get_some_data();  // accesses a large buffer that i don't want to
copy
 size_t pos = my_find(s, "blah");
 s[pos] = "c";
 
 string s2 = "another string";
 size_t pos2 = my_find(s2, "blah");
 another_func(s2[pos2 .. pos2+4]);
 
 size_t my_find(string haystack, string needle)
 {
   // Perform some kind of search that is read-only
 }
 
 another_func(string s) {}
 
 ---
 
 It seems like string is the obvious parameter type to use for
 my_find, but it doesn't work because we're passing mutable data
 in.  Instead we have to use const(char)[], which is less
 intuitive and uglier.
 
 Or did I miss the thrust of the argument?


Most of this discussion seems to assume that there is only two types of
data - immutable and mutable, but there is a third type - "potentially
mutable".

invariant(char)[] --> immutable 
                  --> Once set, it cannot be changed by anything.
const(char)[]     --> potentially mutable
                  --> The compiler ensure that the routine that declares 
                      this will not change it, but it can be changed by
                      other routines.
char[]            --> mutable
                  --> Anything can change this.


The alias "string" only refers to immutable stuff. Can we come up with
aliases for "potentially mutable" and "mutable" too?

I would argue that the parameter signature for my_find() needs to use the
"const(char)[]" type because that means that the coder and compiler says
that this function won't change the input but we don't particularly care if
the input is immutable, or mutable by something else.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 06 2009
next sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Derek Parnell wrote:

 I would argue that the parameter signature for my_find() needs to use the
 "const(char)[]" type because that means that the coder and compiler says
 that this function won't change the input but we don't particularly care if
 the input is immutable, or mutable by something else.


If you want it to be callable by all three variations on character 
arrays, then yes.
Mar 06 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
When we first got into what to do with strings and 
const/immutable/mutable, I was definitely in the camp that strings 
should be mutable char[], or at worst const(char)[]. The thing is, 
Andrei pointed out to me, languages that are considered very good at 
dealing with strings (like Perl) use immutable strings. The fascinating 
thing about strings in such languages is:

"Nobody notices they are immutable, they just work."

So what is it about immutability that makes strings "just work" in a 
natural and intuitive manner? The insight is that it enables strings, 
which are reference types, to behave exactly as if they were value types.

After all, it never occurs to anyone to think that the integer 123 could 
be a "mutable" integer and perhaps be 133 the next time you look at it. 
If you put 123 into a variable, it stays 123. It's immutable. People 
intuitively expect strings to behave the same way. Only C programmers 
expect that once they assign a string to a variable, that string may 
change in place.

C has it backwards by making strings mutable, and it's one of the main 
reasons why dealing with strings in C is such a gigantic pain. But as a 
longtime C programmer, I was so used to that I didn't notice what a pain 
it was until I started using other languages where string manipulation 
was a breeze.

The way to do strings in D is to have them be immutable. If you are 
building a string by manipulating its parts, start with mutable, when 
finished then convert it to immutable and 'publish' it to the rest of 
the program. Mutable char[] arrays should only exist as temporaries. 
This is exactly the opposite of the way one does it in C, but if you do 
it this way, you'll find you never need to defensively dup the string 
"just in case" and things just seem to naturally work out.

I tend to agree that if you try to do strings the C way in D2, you'll 
probably find it to be frustrating experience.
Mar 06 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:

 When we first got into what to do with strings and 
 const/immutable/mutable, I was definitely in the camp that strings 
 should be mutable char[], or at worst const(char)[]. The thing is, 
 Andrei pointed out to me, languages that are considered very good at 
 dealing with strings (like Perl) use immutable strings. The fascinating 
 thing about strings in such languages is:
 
 "Nobody notices they are immutable, they just work."
 
 So what is it about immutability that makes strings "just work" in a 
 natural and intuitive manner? The insight is that it enables strings, 
 which are reference types, to behave exactly as if they were value types.
 
 After all, it never occurs to anyone to think that the integer 123 could 
 be a "mutable" integer and perhaps be 133 the next time you look at it. 
 If you put 123 into a variable, it stays 123. It's immutable. People 
 intuitively expect strings to behave the same way. Only C programmers 
 expect that once they assign a string to a variable, that string may 
 change in place.
 
 C has it backwards by making strings mutable, and it's one of the main 
 reasons why dealing with strings in C is such a gigantic pain. But as a 
 longtime C programmer, I was so used to that I didn't notice what a pain 
 it was until I started using other languages where string manipulation 
 was a breeze.
 
 The way to do strings in D is to have them be immutable. If you are 
 building a string by manipulating its parts, start with mutable, when 
 finished then convert it to immutable and 'publish' it to the rest of 
 the program. Mutable char[] arrays should only exist as temporaries. 
 This is exactly the opposite of the way one does it in C, but if you do 
 it this way, you'll find you never need to defensively dup the string 
 "just in case" and things just seem to naturally work out.
 
 I tend to agree that if you try to do strings the C way in D2, you'll 
 probably find it to be frustrating experience.


That hit the spot.

Andrei
Mar 07 2009
prev sibling next sibling parent Piotrek <starpit tlen.pl> writes:
Walter Bright wrote:

 "Nobody notices they are immutable, they just work."
 
 So what is it about immutability that makes strings "just work" in a 
 natural and intuitive manner? The insight is that it enables strings, 
 which are reference types, to behave exactly as if they were value types.
 
 After all, it never occurs to anyone to think that the integer 123 could 
 be a "mutable" integer and perhaps be 133 the next time you look at it. 
 If you put 123 into a variable, it stays 123. It's immutable. People 
 intuitively expect strings to behave the same way. Only C programmers 
 expect that once they assign a string to a variable, that string may 
 change in place.
 
 C has it backwards by making strings mutable, and it's one of the main 
 reasons why dealing with strings in C is such a gigantic pain. But as a 
 longtime C programmer, I was so used to that I didn't notice what a pain 
 it was until I started using other languages where string manipulation 
 was a breeze.


I could fall into infinite loop while agreeing with you.

Cheers
Mar 07 2009
prev sibling next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Walter Bright wrote:
...

 The way to do strings in D is to have them be immutable. If you are 
 building a string by manipulating its parts, start with mutable, when 
 finished then convert it to immutable and 'publish' it to the rest of 
 the program. Mutable char[] arrays should only exist as temporaries. 
 This is exactly the opposite of the way one does it in C, but if you do 
 it this way, you'll find you never need to defensively dup the string 
 "just in case" and things just seem to naturally work out.
 
 I tend to agree that if you try to do strings the C way in D2, you'll 
 probably find it to be frustrating experience.


That is a really helpful insight. It also means string programming is a bit 
different in D2 than in D1.

At some point in time, it might be helpful to add a little introduction 
'howto program with strings' to the D documentation. After all, it is a 
major feature of D and departure from C and C++.
Mar 07 2009
parent Derek Parnell <derek psych.ward> writes:
On Sat, 07 Mar 2009 10:48:13 +0100, Lutger wrote:


 Walter Bright wrote:



 I tend to agree that if you try to do strings the C way in D2, you'll 
 probably find it to be frustrating experience.




 

 That is a really helpful insight. It also means string programming is a bit 
 different in D2 than in D1.


Tell me about it! When I converted Bud to D2, it was a nightmare. It took
many, many hours of edit-compile cycles to get a clean compile. Then
debugging it took ages due to still trying to think in D1 string terms,
which gave me lots of weird and wrong strings during run time. 

After a lot of trial and error, I finally groked the D2 string concept and
got on top of the issue. But I wanted to have the same source support D1
and D2, which leads to a whole new set of horrors.

The lessons I've learned from this exercise include ...
(a) Wait until D2 is stablised.
(b) Use a text macro processor if you want one source to support D1 and D2.
(c) Any project that is not a simple application, might be better
re-written with D2 than converted from D1.
(d) D2 strings are a useful idea. However, one still needs const(char)[]
and char[] types, so useful mnemonics for these is a good idea.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 07 2009
prev sibling next sibling parent reply Burton Radons <burton.radons gmail.com> writes:
Walter Bright Wrote:


 When we first got into what to do with strings and 
 const/immutable/mutable, I was definitely in the camp that strings 
 should be mutable char[], or at worst const(char)[]. The thing is, 
 Andrei pointed out to me, languages that are considered very good at 
 dealing with strings (like Perl) use immutable strings. The fascinating 
 thing about strings in such languages is:
 
 "Nobody notices they are immutable, they just work."


That's what we said about strings in 1.0. You modify it, you copy it, or you
tell the user. The gentleman's agreement worked perfectly and that came without
a mess of keywords, without implicit or explicit restrictions on behaviour,
without having to condition templates.

Perl would be more powerful if its strings were mutable, not less, although not
by much due to the interpreter.
Mar 07 2009
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sat, 07 Mar 2009 07:11:45 -0500, Burton Radons wrote:


 Perl would be more powerful if its strings
 were mutable, not less, although not by much
 due to the interpreter.


I think we have a terminology issue.

We have character arrays (some fixed length, others variable length -
doesn't matter). In D's world view, data can be invariant (nothing gets to
change it), const (other routines can modify it but this routine will not),
or mutable (anything can change it). So in D we have some character arrays
that are invariant (eg. Literals), some are const, and some are mutable. It
is a pity that D's term "string" is being used in discussions as if it is
synonymous with character array - but it is not. It only refers to certain
types of character arrays - the invariant ones. We really need some simple
terms for const and mutable character arrays.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Derek Parnell wrote:

 On Sat, 07 Mar 2009 07:11:45 -0500, Burton Radons wrote:
 

 Perl would be more powerful if its strings
 were mutable, not less, although not by much
 due to the interpreter.


 
 I think we have a terminology issue.
 
 We have character arrays (some fixed length, others variable length -
 doesn't matter). In D's world view, data can be invariant (nothing gets to
 change it), const (other routines can modify it but this routine will not),
 or mutable (anything can change it). So in D we have some character arrays
 that are invariant (eg. Literals), some are const, and some are mutable. It
 is a pity that D's term "string" is being used in discussions as if it is
 synonymous with character array - but it is not. It only refers to certain
 types of character arrays - the invariant ones. We really need some simple
 terms for const and mutable character arrays.
 


I don't think char[] is half bad. const(char)[] is a mouthful, but most 
of the time those are function parameters, where the handy in char[] 
applies.

Andrei
Mar 07 2009
parent Derek Parnell <derek psych.ward> writes:
On Sat, 07 Mar 2009 07:09:34 -0800, Andrei Alexandrescu wrote:



 I don't think char[] is half bad. const(char)[] is a mouthful, but most 
 of the time those are function parameters, where the handy in char[] 
 applies.


I was think more as unambiguous words we can use in discussions and not so
much as aliases in our code.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 07 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Burton Radons wrote:

 That's what we said about strings in 1.0. You modify it, you copy it,
 or you tell the user. The gentleman's agreement worked perfectly and
 that came without a mess of keywords, without implicit or explicit
 restrictions on behaviour, without having to condition templates.


The one flaw in it was the behavior I consistently saw of "I'm copying 
the string just to be sure I own it and nobody else changes it." D was 
meant for copy-on-write, which means copy the string *only* if you 
change it. No defensive copying. No "just in case" copying. The 
gentleman's agreement failed as far as I could tell.

With immutable strings, the gentleman's agreement is enforced.
Mar 07 2009
next sibling parent reply grauzone <none example.net> writes:
Walter Bright wrote:

 Burton Radons wrote:

 That's what we said about strings in 1.0. You modify it, you copy it,
 or you tell the user. The gentleman's agreement worked perfectly and
 that came without a mess of keywords, without implicit or explicit
 restrictions on behaviour, without having to condition templates.


 
 The one flaw in it was the behavior I consistently saw of "I'm copying 
 the string just to be sure I own it and nobody else changes it." D was 
 meant for copy-on-write, which means copy the string *only* if you 
 change it. No defensive copying. No "just in case" copying. The 
 gentleman's agreement failed as far as I could tell.
 
 With immutable strings, the gentleman's agreement is enforced.


What about automatic, built-in copy on write?
Mar 07 2009
next sibling parent Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
grauzone wrote:

 Walter Bright wrote:

 Burton Radons wrote:

 That's what we said about strings in 1.0. You modify it, you copy it,
 or you tell the user. The gentleman's agreement worked perfectly and
 that came without a mess of keywords, without implicit or explicit
 restrictions on behaviour, without having to condition templates.


 The one flaw in it was the behavior I consistently saw of "I'm copying 
 the string just to be sure I own it and nobody else changes it." D was 
 meant for copy-on-write, which means copy the string *only* if you 
 change it. No defensive copying. No "just in case" copying. The 
 gentleman's agreement failed as far as I could tell.

 With immutable strings, the gentleman's agreement is enforced.


 
 What about automatic, built-in copy on write?


Then it would happen even when you *know* you're the only one with a 
reference. Worse, it'd happen multiple times if you modify multiple 
characters in a row...
Mar 07 2009
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
grauzone wrote:

 Walter Bright wrote:

 Burton Radons wrote:

 That's what we said about strings in 1.0. You modify it, you copy it,
 or you tell the user. The gentleman's agreement worked perfectly and
 that came without a mess of keywords, without implicit or explicit
 restrictions on behaviour, without having to condition templates.


 The one flaw in it was the behavior I consistently saw of "I'm copying 
 the string just to be sure I own it and nobody else changes it." D was 
 meant for copy-on-write, which means copy the string *only* if you 
 change it. No defensive copying. No "just in case" copying. The 
 gentleman's agreement failed as far as I could tell.

 With immutable strings, the gentleman's agreement is enforced.


 
 What about automatic, built-in copy on write?


No go with threads. COW sounded like a great idea for std::string in 
ancient times when threads were a rarity. Today, virtually all C++ 
implementations actively dropped COW and replaced it with eager copy + 
small string optimization for short strings. D really has the best of 
all worlds solution.

Andrei
Mar 07 2009
prev sibling parent Walter Bright <newshound1 digitalmars.com> writes:
grauzone wrote:

 What about automatic, built-in copy on write?


Can't do it efficiently without hardware support.
Mar 07 2009
prev sibling parent reply Burton Radons <burton.radons gmail.com> writes:
Walter Bright Wrote:


 Burton Radons wrote:

 That's what we said about strings in 1.0. You modify it, you copy it,
 or you tell the user. The gentleman's agreement worked perfectly and
 that came without a mess of keywords, without implicit or explicit
 restrictions on behaviour, without having to condition templates.


 
 The one flaw in it was the behavior I consistently saw of "I'm copying 
 the string just to be sure I own it and nobody else changes it." D was 
 meant for copy-on-write, which means copy the string *only* if you 
 change it. No defensive copying. No "just in case" copying. The 
 gentleman's agreement failed as far as I could tell.
 
 With immutable strings, the gentleman's agreement is enforced.


Am I going to become a broken record on this? Because "invariant (char) []" is
the string type, data that is going to be mutable will always find its way into
that type in order to deal with an API which WILL use string as its arguments,
not writing out "const (char) []". It gives me no information about the future
of the object while removing the apparent need for the gentleman's agreement.
Therefore I have no way of knowing what the actual pedigree of this string I've
been given has. It may be invariant, it may be mutable.

I want this to be addressed directly. Exactly how am I wrong on this point? Is
it not conceivable that mutable data gets casted to invariant in this case?
Mar 07 2009
next sibling parent Derek Parnell <derek psych.ward> writes:
On Sat, 07 Mar 2009 17:08:58 -0500, Burton Radons wrote:


 Am I going to become a broken record on this? Because
 "invariant (char) []" is the string type, data that
 is going to be mutable will always find its way into
 that type in order to deal with an API which WILL use
 string as its arguments, not writing out
 "const (char) []". 


I'm starting to think that 'string' for function parameters should be a
rare thing. For a function to insist that it only recieves immutable data
sounds like the function is worried that it might accidently change data.
And that sounds like a bug to me. It is shifting the responsibility to the
caller for the data's integrity.


 It gives me no information about
 the future of the object while removing the apparent
 need for the gentleman's agreement. Therefore I have
 no way of knowing what the actual pedigree of this
 string I've been given has. It may be invariant, it
 may be mutable.


But why would your function care about that? Let's assume your function's
signature is 'const' for its parameters because it does not intend to
modify any of them. If the caller passes invariant data then your function
cannot modify the arguments. If the caller passes mutable data, the
compiler won't allow your function to modify the parameters either, due to
the const signature. So why is it important that the function should know
the mutability of the passed data?

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 07 2009
prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
If I may restate your case, it is that given function that does 
something with character arrays:

int foo(string s);

and you wish to pass a mutable character array to it. If foo was 
declared as:

int foo(const(char)[] s);

then it would just work. So why is it declared immutable(char)[] when 
that isn't actually necessary?

The answer is to encourage the use of immutable strings. I believe the 
future of programming will tend towards ever more use of immutable data, 
as immutable data:

1. is implicitly sharable between threads
2. is more conducive to static analysis of programs
3. makes it easier for programmers to understand code
4. enables better code generation
5. allows taking a private reference to without needing to make a copy

const(char)[], on the other hand, still leaves us with the temptation to 
make a copy "just in case". If I, as a user, sees:

int foo(const(char)[] s)

what if foo() keeps a private reference to s (which it might if it does 
lazy evaluation)? Now I, as a caller, mutate s[] and muck up foo. So, to 
fix it, I do:

foo(s.dup);    // defensive copy in case foo keeps a reference to s

But the implementor of foo() doesn't know it's getting its own private 
copy, so the first line of foo() is:

int foo(const(char)[] s)
{
     s = s.dup;   // make sure we own a copy
}

so the defensive, robust code has TWO unnecessary copies.
Mar 07 2009
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sat, 07 Mar 2009 14:43:50 -0800, Walter Bright wrote:


 int foo(const(char)[] s)
 
 what if foo() keeps a private reference to s (which it might if it does 
 lazy evaluation)? Now I, as a caller, mutate s[] and muck up foo. So, to 
 fix it, I do:
 
 foo(s.dup);    // defensive copy in case foo keeps a reference to s


In foo's defence, if it takes a private reference, then it should also take
a copy. In fact, should it be allowed to take a private reference of data
which might be modified after it returns?

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 07 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Derek Parnell wrote:

 On Sat, 07 Mar 2009 14:43:50 -0800, Walter Bright wrote:
 

 int foo(const(char)[] s)

 what if foo() keeps a private reference to s (which it might if it does 
 lazy evaluation)? Now I, as a caller, mutate s[] and muck up foo. So, to 
 fix it, I do:

 foo(s.dup);    // defensive copy in case foo keeps a reference to s


 
 In foo's defence, if it takes a private reference, then it should also take
 a copy.


Yup, and as I said, an extra copy "just in case".


 In fact, should it be allowed to take a private reference of data
 which might be modified after it returns?
 


Instead of adding more complexity to const so it acts more like 
immutable, why not just use immutable <g> ?
Mar 07 2009
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:

 If I may restate your case, it is that given function that does 
 something with character arrays:
 
 int foo(string s);
 
 and you wish to pass a mutable character array to it. If foo was 
 declared as:
 
 int foo(const(char)[] s);
 
 then it would just work. So why is it declared immutable(char)[] when 
 that isn't actually necessary?
 
 The answer is to encourage the use of immutable strings. I believe the 
 future of programming will tend towards ever more use of immutable data, 
 as immutable data:
 
 1. is implicitly sharable between threads


In fact const data is also implicitly sharable between threads. This is 
because shared is not implicitly convertible to const. No?

Andrei
Mar 07 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:

 Walter Bright wrote:

 If I may restate your case, it is that given function that does 
 something with character arrays:

 int foo(string s);

 and you wish to pass a mutable character array to it. If foo was 
 declared as:

 int foo(const(char)[] s);

 then it would just work. So why is it declared immutable(char)[] when 
 that isn't actually necessary?

 The answer is to encourage the use of immutable strings. I believe the 
 future of programming will tend towards ever more use of immutable 
 data, as immutable data:

 1. is implicitly sharable between threads


 
 In fact const data is also implicitly sharable between threads.


No. You have to declare it "shared const" to make it sharable between 
threads.
Mar 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:

 Andrei Alexandrescu wrote:

 Walter Bright wrote:

 If I may restate your case, it is that given function that does 
 something with character arrays:

 int foo(string s);

 and you wish to pass a mutable character array to it. If foo was 
 declared as:

 int foo(const(char)[] s);

 then it would just work. So why is it declared immutable(char)[] when 
 that isn't actually necessary?

 The answer is to encourage the use of immutable strings. I believe 
 the future of programming will tend towards ever more use of 
 immutable data, as immutable data:

 1. is implicitly sharable between threads


 In fact const data is also implicitly sharable between threads.


 
 No. You have to declare it "shared const" to make it sharable between 
 threads.


Sorry, I got confused. What I meant was that a function accepting a 
const T can count on other threads leaving T alone, which is the 
converse of what you say. Cool!

Andrei
Mar 07 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:

 Sorry, I got confused. What I meant was that a function accepting a 
 const T can count on other threads leaving T alone, which is the 
 converse of what you say.


Yes.

  Cool!

 
 Andrei
Mar 07 2009
prev sibling parent reply Burton Radons <burton.radons gmail.com> writes:
Walter Bright Wrote:


 If I may restate your case, it is that given function that does 
 something with character arrays:
 
 int foo(string s);
 
 and you wish to pass a mutable character array to it. If foo was 
 declared as:
 
 int foo(const(char)[] s);
 
 then it would just work. So why is it declared immutable(char)[] when 
 that isn't actually necessary?


No, that's not the problem at all. The problem is this line in object.d:

    alias invariant (char) [] string;

There are two interesting features here. It's what D calls a string, and it's
invariant, making the declaration that whoever has a reference to that string
can hold onto it forever without ever expecting its contents to be modified or
destroyed.

So, while building my string I use a function which replaces matching
substrings in a string with another string. If that function were to declare my
parameters as strings they'd do two things: they'd tell the reader that its
parameters can never change over the course of the program because it may
retain copies of the parameters. That is a strong, highly prescriptive
statement. So I would expect the function to be implemented like this:

   const (char) [] replace (const (char) [] s, const (char) [] from, const
(char) [] to)

But it's not. std.string.replace is implemented like this:

   string replace (string s, string from, string to)

This is for a number of reasons. It's easiest to assume that the default is
going to be the correct one. The const syntax is hard to read, so it's avoided,
and "string" is more readily descriptive than "const (char) []". So, I pass my
mutable string to std.string.replace, which only accepts invariant data.

This wouldn't be too bad because const is worthless when optimising, but if
invariant is going to be given any weight then we must never cause data to be
casted to invariant unless if it's actually invariant data. So, the sensible
default is "const (char) []" for strings, a selection of aliases in object.d
for the others, and safe casting templates in object.d.
Mar 07 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Burton Radons wrote:

 This wouldn't be too bad because const is worthless when optimising,
 but if invariant is going to be given any weight then we must never
 cause data to be casted to invariant unless if it's actually
 invariant data. So, the sensible default is "const (char) []" for
 strings, a selection of aliases in object.d for the others, and safe
 casting templates in object.d.


What I interpret from this is that you see strings as fundamentally 
mutable character arrays, and sometimes in special cases you can make 
them immutable. I propose turning that view on its head - regard strings 
as fundamentally immutable, and having a mutable char array is a rare 
thing that only appears in isolated places in the program.

In the find() example, the implementation of it actually uses a mutable 
char[] to build the result. When the result is done, it is converted to 
immutable and "published" by returning it. The mutable array never 
escapes the function; it is completely sandboxed in.

What sold me on immutable strings was going through my code and looking 
to see where I *actually* was mutating the strings in place rather than 
just passing them around or storing them or copying them into another 
buffer. It turns out it was a vanishingly small number. I was startled. 
Not only that, those places could be, with a minor bit of refactoring, 
further reduced in number without sacrifice. I stacked this against the 
gain by eliminating all those places that were doing copies, and it was 
clear that immutable strings as default was a winner.
Mar 07 2009
parent reply Burton Radons <burton.radons gmail.com> writes:
Walter Bright Wrote:


 Burton Radons wrote:

 This wouldn't be too bad because const is worthless when optimising,
 but if invariant is going to be given any weight then we must never
 cause data to be casted to invariant unless if it's actually
 invariant data. So, the sensible default is "const (char) []" for
 strings, a selection of aliases in object.d for the others, and safe
 casting templates in object.d.


 
 What I interpret from this is that you see strings as fundamentally 
 mutable character arrays, and sometimes in special cases you can make 
 them immutable. I propose turning that view on its head - regard strings 
 as fundamentally immutable, and having a mutable char array is a rare 
 thing that only appears in isolated places in the program.


No, I don't. You are misunderstanding me, and I'm not sure why or how. Here's a
(contrived) example of where my concern may come into play:

   int [] a = new int [1];

   a [0] = 1;

   auto b = cast (invariant (int) []) a;

   a [0] += b [0];
   a [0] += b [0];
   writef ("%s\n", a [0]);
   // Normal result: 4.
   // Optimiser which assumes invariant data can't change: 3

Yes, the code is an abuse of the const system. THAT'S EXACTLY MY POINT. Casting
mutable data to invariant leads to situations like these. Only data which will
never change can be made invariant. Putting "alias invariant (char) [] string"
in object.d induces these situations and makes it seem like it's a good idea.
Mar 07 2009
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Burton Radons wrote:

 Walter Bright Wrote:
 

 Burton Radons wrote:

 This wouldn't be too bad because const is worthless when
 optimising, but if invariant is going to be given any weight then
 we must never cause data to be casted to invariant unless if it's
 actually invariant data. So, the sensible default is "const
 (char) []" for strings, a selection of aliases in object.d for
 the others, and safe casting templates in object.d.


 What I interpret from this is that you see strings as fundamentally
  mutable character arrays, and sometimes in special cases you can
 make them immutable. I propose turning that view on its head -
 regard strings as fundamentally immutable, and having a mutable
 char array is a rare thing that only appears in isolated places in
 the program.


 
 No, I don't. You are misunderstanding me, and I'm not sure why or
 how.


I guess I just cannot figure out where you're coming from.


 Here's a (contrived) example of where my concern may come into
 play:
 
 int [] a = new int [1];
 
 a [0] = 1;
 
 auto b = cast (invariant (int) []) a;
 
 a [0] += b [0]; a [0] += b [0]; writef ("%s\n", a [0]); // Normal
 result: 4. // Optimiser which assumes invariant data can't change: 3
 
 Yes, the code is an abuse of the const system. THAT'S EXACTLY MY
 POINT. Casting mutable data to invariant leads to situations like
 these. Only data which will never change can be made invariant.
 Putting "alias invariant (char) [] string" in object.d induces these
 situations and makes it seem like it's a good idea.


I'm still not understanding you, because this is a contrived example 
that I cannot see the point of nor can I see where it would be 
legitimately used.
Mar 07 2009
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sat, 07 Mar 2009 23:40:52 -0800, Walter Bright wrote:


 I'm still not understanding you, because this is a contrived example 
 that I cannot see the point of nor can I see where it would be 
 legitimately used.


I can see Burton's concern, and I'm very surprised that the compiler allows
this to happen. Here is a slightly more explicit version of Burton's code.

import std.stdio;
void main()
{
  int [] a = new int [1];

   a [0] = 1;

   invariant (int) [] b = cast (invariant (int) []) a;

   writef ("a=%s b=%s\n", a [0], b[0]);
   a [0] += b [0];
   writef ("a=%s b=%s\n", a [0], b[0]);
   a [0] += b [0];
   writef ("a=%s b=%s\n", a [0], b[0]);
}
   
The problem is that we have declared 'b' as invariant, but the program is
allowed to change it. That is the issue.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 08 2009
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Derek Parnell wrote:

 import std.stdio;
 void main()
 {
   int [] a = new int [1];
 
    a [0] = 1;
 
    invariant (int) [] b = cast (invariant (int) []) a;
 
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
 }
    
 The problem is that we have declared 'b' as invariant, but the program is
 allowed to change it. That is the issue.
 


The following also compiles:

char c;
int* p = cast(int*)&c;
*p = 5;

and is clearly buggy code. Whenever you use a cast, the onus is on the 
programmer to know what they are doing. The cast is an escape from the 
typing system.
Mar 08 2009
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sun, 08 Mar 2009 00:36:06 -0800, Walter Bright wrote:


 Derek Parnell wrote:

 import std.stdio;
 void main()
 {
   int [] a = new int [1];
 
    a [0] = 1;
 
    invariant (int) [] b = cast (invariant (int) []) a;
 
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
 }
    
 The problem is that we have declared 'b' as invariant, but the program is
 allowed to change it. That is the issue.
 


 
 The following also compiles:
 
 char c;
 int* p = cast(int*)&c;
 *p = 5;
 
 and is clearly buggy code. Whenever you use a cast, the onus is on the 
 programmer to know what they are doing. The cast is an escape from the 
 typing system.


Walter, you have side-stepped the problem in question by talking about a
totally different problem.

Burtons code says "b is invariant", but the program allows it to be
changed. Your code does NOT say that any of those variables is invariant.
The problem is NOT with the cast (although that is a totally different
issue). The problem is that the code says "invariant" but the data gets
changed anyhow. The method of changing the data is not the issue. The issue
is that is gets changed at all.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 08 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Derek Parnell wrote:

 Walter, you have side-stepped the problem in question by talking about a
 totally different problem.


It's the same issue. When you use a cast, you are subverting the type 
system. That means you have to be sure you are doing it right. The 
compiler cannot help you.


 Burtons code says "b is invariant", but the program allows it to be
 changed. Your code does NOT say that any of those variables is invariant.
 The problem is NOT with the cast (although that is a totally different
 issue). The problem is that the code says "invariant" but the data gets
 changed anyhow. The method of changing the data is not the issue. The issue
 is that is gets changed at all.


When you cast something to immutable, you can no longer change it. It's 
a one-way ticket.
Mar 08 2009
parent reply Derek Parnell <derek psych.ward> writes:
On Sun, 08 Mar 2009 01:19:15 -0800, Walter Bright wrote:


 Derek Parnell wrote:

 Walter, you have side-stepped the problem in question by talking about a
 totally different problem.


 
 It's the same issue. When you use a cast, you are subverting the type 
 system. That means you have to be sure you are doing it right. The 
 compiler cannot help you.


I disagree. There are two issues being discussed. One by Burton and another
by yourself. Burton is showing how that data declared as immutable can be
modified. You are talking about the dangers of using casts. NOT THE SAME
ISSUE. 
 

 Burtons code says "b is invariant", but the program allows it to be
 changed. Your code does NOT say that any of those variables is invariant.
 The problem is NOT with the cast (although that is a totally different
 issue). The problem is that the code says "invariant" but the data gets
 changed anyhow. The method of changing the data is not the issue. The issue
 is adxthat is gets changed at all.


 
 When you cast something to immutable, you can no longer change it. It's 
 a one-way ticket.


Walter, did you actually see and run that code? I did cast something to
immutable but it got changed anyway. Is this a compiler bug or what? You
say that if one casts something to immutable that therefore one can no
longer change it - but it DID get changed.

I know that a better way to code this example would have been to use the
.idup functionality, but that is not the point. I relied on the compiler
ensuring that everything declared as immutable would not be modified. The
compiler failed.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 08 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Derek Parnell wrote:

 On Sun, 08 Mar 2009 01:19:15 -0800, Walter Bright wrote:
 

 Derek Parnell wrote:

 Walter, you have side-stepped the problem in question by talking about a
 totally different problem.


 It's the same issue. When you use a cast, you are subverting the type 
 system. That means you have to be sure you are doing it right. The 
 compiler cannot help you.


 
 I disagree. There are two issues being discussed. One by Burton and another
 by yourself. Burton is showing how that data declared as immutable can be
 modified. You are talking about the dangers of using casts. NOT THE SAME
 ISSUE. 


It is THE VERY SAME ISSUE.

Andrei
Mar 08 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Derek Parnell wrote:

 I know that a better way to code this example would have been to use the
 .idup functionality, but that is not the point. I relied on the compiler
 ensuring that everything declared as immutable would not be modified. The
 compiler failed.


It is the same issue. When you use a cast, you are *explicitly* 
defeating the language's type checking ability. It means that the onus 
is on the one doing the cast to get it right.
Mar 08 2009
parent reply Steve Schveighoffer <schveiguy yahoo.com> writes:
On Sun, 08 Mar 2009 17:24:09 -0700, Walter Bright wrote:


 Derek Parnell wrote:

 I know that a better way to code this example would have been to use
 the .idup functionality, but that is not the point. I relied on the
 compiler ensuring that everything declared as immutable would not be
 modified. The compiler failed.


 
 It is the same issue. When you use a cast, you are *explicitly*
 defeating the language's type checking ability. It means that the onus
 is on the one doing the cast to get it right.


Except when you want invariant data, then cast is *required*.  At that 
point, it is a language feature, not a defeat of the typesystem.  I think 
there is some merit to the arguments presented in this thread, but I 
don't think the answer is to get rid of invariant.  Perhaps make the 
compiler more strict when creating invariant data?  I liked the ideas 
that people presented about having unique mutable references (before and 
in this thread).  This might even be solvable in a library given all the 
advances in structs.

So it's not exactly the same issue, because in one you are doing 
something totally useless and stupid.  And in the other, it is a language 
*requirement* to use casting to get invariant data.  However, in both 
cases, the onus is on the developer, which sucks in the latter case...

Walter: Use invariant when you can, it's the best!
User: ok, how do I use it?
Walter: You need to cast mutable data to invariant, but it's on you to 
make sure nobody changes the original mutable data.  Casting circumvents 
the typesystem, so the compiler can't help you.
User: :(

-Steve
Mar 08 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steve Schveighoffer wrote:

 On Sun, 08 Mar 2009 17:24:09 -0700, Walter Bright wrote:
 

 Derek Parnell wrote:

 I know that a better way to code this example would have been to use
 the .idup functionality, but that is not the point. I relied on the
 compiler ensuring that everything declared as immutable would not be
 modified. The compiler failed.


 It is the same issue. When you use a cast, you are *explicitly*
 defeating the language's type checking ability. It means that the onus
 is on the one doing the cast to get it right.


 
 Except when you want invariant data, then cast is *required*.


Not at all. Calling idup (or, more elegantly, to!TargetType) or 
assumeUnique under the circumstances documented by assumeUnique are all 
safe. Sometimes the price for added safety is one extra copy. But the 
added safety is worth it more often than not.

As far as signatures of functions in std.string, I agree that those not 
needing a string of immutable characters should just accept in Char[] 
(where Char is one of the three character types). That should make 
people using mutable and immutable strings equally joyous.


 At that 
 point, it is a language feature, not a defeat of the typesystem.  I think 
 there is some merit to the arguments presented in this thread, but I 
 don't think the answer is to get rid of invariant.  Perhaps make the 
 compiler more strict when creating invariant data?  I liked the ideas 
 that people presented about having unique mutable references (before and 
 in this thread).  This might even be solvable in a library given all the 
 advances in structs.


Unique has been discussed extensively a couple of years ago when we were 
defining const and immutable. We decided to forgo it and go with 
assumeUnique.


 So it's not exactly the same issue, because in one you are doing 
 something totally useless and stupid.  And in the other, it is a language 
 *requirement* to use casting to get invariant data.  However, in both 
 cases, the onus is on the developer, which sucks in the latter case...


NO. It is not a language requirement. If what you have is mutable data 
and someone else wants immutable data, make a copy.


 Walter: Use invariant when you can, it's the best!
 User: ok, how do I use it?
 Walter: You need to cast mutable data to invariant, but it's on you to 
 make sure nobody changes the original mutable data.  Casting circumvents 
 the typesystem, so the compiler can't help you.
 User: :(


Casts to immutable should not be part of most programs.


Andrei
Mar 08 2009
next sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Sun, 08 Mar 2009 19:24:32 -0700, Andrei Alexandrescu wrote:


 As far as signatures of functions in std.string, I agree that those not 
 needing a string of immutable characters should just accept in Char[] 
 (where Char is one of the three character types). That should make 
 people using mutable and immutable strings equally joyous.


So you agree that *the standard library* should avoid using immutable
when it's not strictly necessary after all.  This is quite in contrary
to what Walter says.  Now the question is, how any *other* libraries
differ?  I usually don't care about how you use immutable in your
personal code.  But if there is an XML library which takes immutables
everywhere,  it seriously limits my freedom in chosing my coding style.

Immutable string manipulation produces tons of garbage objects.  Every
time you want to replace '\\' with '/' you get garbage, if not as many
times as you encounter '\\' in the string.  This is something we avoided
at any cost in mobile Java games.  This is the attitude which makes
"large" Java applications so memory-hungry.  I have a hard time to
believe this magically became OK in D.
Mar 09 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Sergey Gromov wrote:

 Sun, 08 Mar 2009 19:24:32 -0700, Andrei Alexandrescu wrote:
 

 As far as signatures of functions in std.string, I agree that those not 
 needing a string of immutable characters should just accept in Char[] 
 (where Char is one of the three character types). That should make 
 people using mutable and immutable strings equally joyous.


 
 So you agree that *the standard library* should avoid using immutable
 when it's not strictly necessary after all.


I do.


 This is quite in contrary
 to what Walter says.


It is. This may be because of a slight difference in philosophy; I think 
any entity (function, type) of the standard library should accept the 
most general types it could conceivably work with. In fact I'd be happy 
put all or most significant algorithms in std.string inside std.algorithm.

   Now the question is, how any *other* libraries

 differ?  I usually don't care about how you use immutable in your
 personal code.  But if there is an XML library which takes immutables
 everywhere,  it seriously limits my freedom in chosing my coding style.


I agree but disagree with "seriously".


 Immutable string manipulation produces tons of garbage objects.


This I disagree with. There may be extra copies, but you also save other 
copies. Also there's never risky aliasing - coding with immutable 
strings is safe. D simply can't afford to seriously suggest in this day 
and age a programming style with aliased mutable strings.


 Every
 time you want to replace '\\' with '/' you get garbage, if not as many
 times as you encounter '\\' in the string.  This is something we avoided
 at any cost in mobile Java games.  This is the attitude which makes
 "large" Java applications so memory-hungry.  I have a hard time to
 believe this magically became OK in D.


It does become OK (albeit not magically) in D because D does offer you 
the option to work with char[] if you need to. I understand that the 
problem of you calling into libraries only taking string still remains, 
but really you can't have everything. I think we're in better shape than 
most languages.


Andrei
Mar 09 2009
prev sibling next sibling parent reply Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:

 As far as signatures of functions in std.string, I agree that those not 
 needing a string of immutable characters should just accept in Char[] 
 (where Char is one of the three character types). That should make 
 people using mutable and immutable strings equally joyous.


Also std.conv (especially the scheduled-to-be-deprecated toInt(), etc),
and probably also the functions in std.file, such as exists().
Mar 09 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Don wrote:

 Andrei Alexandrescu wrote:

 As far as signatures of functions in std.string, I agree that those 
 not needing a string of immutable characters should just accept in 
 Char[] (where Char is one of the three character types). That should 
 make people using mutable and immutable strings equally joyous.


 
 Also std.conv (especially the scheduled-to-be-deprecated toInt(), etc),
 and probably also the functions in std.file, such as exists().


Yah. That's fixed already, but I concluded I can't check in my changes 
to phobos until a few bugs are fixed, notably 2674 
(http://d.puremagic.com/issues/show_bug.cgi?id=2674). Please vote up :o).


Andrei
Mar 09 2009
parent Don <nospam nospam.com> writes:
Andrei Alexandrescu wrote:

 Don wrote:

 Andrei Alexandrescu wrote:

 As far as signatures of functions in std.string, I agree that those 
 not needing a string of immutable characters should just accept in 
 Char[] (where Char is one of the three character types). That should 
 make people using mutable and immutable strings equally joyous.


 Also std.conv (especially the scheduled-to-be-deprecated toInt(), etc),
 and probably also the functions in std.file, such as exists().


 
 Yah. That's fixed already, but I concluded I can't check in my changes 
 to phobos until a few bugs are fixed, notably 2674 
 (http://d.puremagic.com/issues/show_bug.cgi?id=2674). Please vote up :o).
 
 
 Andrei


Yes.
BTW, in my own code, a major reason why string -> const(char)[] is such 
a necessary change is that std.stream.readLine() returns 'char[]', not 
'string'. So you can hardly do anything without a cast or an idup.

Don.
Mar 09 2009
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On Sun, 08 Mar 2009 19:24:32 -0700, Andrei Alexandrescu wrote:


 Steve Schveighoffer wrote:

 On Sun, 08 Mar 2009 17:24:09 -0700, Walter Bright wrote:
 

 Derek Parnell wrote:

 I know that a better way to code this example would have been to use
 the .idup functionality, but that is not the point. I relied on the
 compiler ensuring that everything declared as immutable would not be
 modified. The compiler failed.


 It is the same issue. When you use a cast, you are *explicitly*
 defeating the language's type checking ability. It means that the onus
 is on the one doing the cast to get it right.


 
 Except when you want invariant data, then cast is *required*.


 
 Not at all. Calling idup (or, more elegantly, to!TargetType) or
 assumeUnique under the circumstances documented by assumeUnique are all
 safe. Sometimes the price for added safety is one extra copy. But the
 added safety is worth it more often than not.


idup is not a viable option in certain cases, if memory usage or 
performance are important.  It might not even be available if the object 
in question doesn't implement a function that does it.  assumeUnique 
suffers from the same problems as casting.  It just avoids any potential 
casting that changes types instead of constancy.

I agree that in 90% of cases, using invariant strings is beneficial, and 
not a burden.  I think the arguments against are for those small cases 
where performance is significantly hindered, and having a library which 
uses string where it should use in char[] makes life difficult for those 
cases.  I understand you are fixing this, so that is good.


 
 As far as signatures of functions in std.string, I agree that those not
 needing a string of immutable characters should just accept in Char[]
 (where Char is one of the three character types). That should make
 people using mutable and immutable strings equally joyous.
 


What might be useful is coming up with an alias for char[], like mstring 
or something that shakes off the notion that a mutable char[] is not a 
string.


 At that
 point, it is a language feature, not a defeat of the typesystem.  I
 think there is some merit to the arguments presented in this thread,
 but I don't think the answer is to get rid of invariant.  Perhaps make
 the compiler more strict when creating invariant data?  I liked the
 ideas that people presented about having unique mutable references
 (before and in this thread).  This might even be solvable in a library
 given all the advances in structs.


 
 Unique has been discussed extensively a couple of years ago when we were
 defining const and immutable. We decided to forgo it and go with
 assumeUnique.


I see in your reply to walter that it could be done with some bug fixes, 
I think this should be an important step to make.


 So it's not exactly the same issue, because in one you are doing
 something totally useless and stupid.  And in the other, it is a
 language *requirement* to use casting to get invariant data.  However,
 in both cases, the onus is on the developer, which sucks in the latter
 case...


 
 NO. It is not a language requirement. If what you have is mutable data
 and someone else wants immutable data, make a copy.


Unless copying hinders performance so much that you would rather take the 
safety hit and start casting.  Imagine having to copy every message that 
was received on a network stream just so you could use string functions 
on it.  Or having to duplicate all the data read into a small buffer from 
a file just to search for strings in it.


 Walter: Use invariant when you can, it's the best! User: ok, how do I
 use it?
 Walter: You need to cast mutable data to invariant, but it's on you to
 make sure nobody changes the original mutable data.  Casting
 circumvents the typesystem, so the compiler can't help you. User: :(


 
 Casts to immutable should not be part of most programs.


assumeUnique is a cast.  You have not avoided that.  The extra steps it 
goes through is not enough to quiet this discussion.  We are not talking 
about using assumeUnique or casting everywhere in a program.  We are 
talking about how difficult it is to call functions that take strings 
when they should take const(char)[] (a practice that is going to continue 
since the string label has been attached to immutable(char)[] only) in 
the rare cases when you need the functions.

-Steve
Mar 09 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Steve Schveighoffer wrote:

 Walter: Use invariant when you can, it's the best!
 User: ok, how do I use it?
 Walter: You need to cast mutable data to invariant, but it's on you to 
 make sure nobody changes the original mutable data.  Casting circumvents 
 the typesystem, so the compiler can't help you.
 User: :(


Unfortunately, we could not come up with a typesafe scheme for going 
from mutable to immutable that was reasonable. The transition is up to 
the user to do correctly, but it isn't a terrible burden, and is simple 
to get right.
Mar 08 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Walter Bright wrote:

 Steve Schveighoffer wrote:

 Walter: Use invariant when you can, it's the best!
 User: ok, how do I use it?
 Walter: You need to cast mutable data to invariant, but it's on you to 
 make sure nobody changes the original mutable data.  Casting 
 circumvents the typesystem, so the compiler can't help you.
 User: :(


 
 Unfortunately, we could not come up with a typesafe scheme for going 
 from mutable to immutable that was reasonable. The transition is up to 
 the user to do correctly, but it isn't a terrible burden, and is simple 
 to get right.


In fact, since we last discussed that, dmd technology has made enough 
strides (particularly wrt manipulation of copies) that we can define 
Unique!T and something a la Java's StringBuilder with ease. 
Unfortunately current bugs in constructor implementation delay 
definition of such artifacts.


Andrei
Mar 08 2009
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On Sun, 08 Mar 2009 20:51:39 -0700, Andrei Alexandrescu wrote:


 Walter Bright wrote:

 Steve Schveighoffer wrote:

 Walter: Use invariant when you can, it's the best! User: ok, how do I
 use it?
 Walter: You need to cast mutable data to invariant, but it's on you to
 make sure nobody changes the original mutable data.  Casting
 circumvents the typesystem, so the compiler can't help you. User: :(


 
 Unfortunately, we could not come up with a typesafe scheme for going
 from mutable to immutable that was reasonable. The transition is up to
 the user to do correctly, but it isn't a terrible burden, and is simple
 to get right.


 
 In fact, since we last discussed that, dmd technology has made enough
 strides (particularly wrt manipulation of copies) that we can define
 Unique!T and something a la Java's StringBuilder with ease.
 Unfortunately current bugs in constructor implementation delay
 definition of such artifacts.


Yes, please fix!

-Steve
Mar 09 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Steven Schveighoffer wrote:

 On Sun, 08 Mar 2009 20:51:39 -0700, Andrei Alexandrescu wrote:
 

 Walter Bright wrote:

 Steve Schveighoffer wrote:

 Walter: Use invariant when you can, it's the best! User: ok, how do I
 use it?
 Walter: You need to cast mutable data to invariant, but it's on you to
 make sure nobody changes the original mutable data.  Casting
 circumvents the typesystem, so the compiler can't help you. User: :(


 Unfortunately, we could not come up with a typesafe scheme for going
 from mutable to immutable that was reasonable. The transition is up to
 the user to do correctly, but it isn't a terrible burden, and is simple
 to get right.


 In fact, since we last discussed that, dmd technology has made enough
 strides (particularly wrt manipulation of copies) that we can define
 Unique!T and something a la Java's StringBuilder with ease.
 Unfortunately current bugs in constructor implementation delay
 definition of such artifacts.


 
 Yes, please fix!
 
 -Steve


Please vote up:

http://d.puremagic.com/issues/show_bug.cgi?id=2674


Andrei
Mar 09 2009
prev sibling parent grauzone <none example.net> writes:
Walter Bright wrote:

 Derek Parnell wrote:

 import std.stdio;
 void main()
 {
   int [] a = new int [1];

    a [0] = 1;

    invariant (int) [] b = cast (invariant (int) []) a;

    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
 }
    The problem is that we have declared 'b' as invariant, but the 
 program is
 allowed to change it. That is the issue.


 
 The following also compiles:
 
 char c;
 int* p = cast(int*)&c;
 *p = 5;
 
 and is clearly buggy code. Whenever you use a cast, the onus is on the 
 programmer to know what they are doing. The cast is an escape from the 
 typing system.


Sometimes a cast can break a program, sometimes a cast is absolutely 
harmless:

int w, h;
double ratio = (cast(double)w) / h;

Sure you don't want to introduce something like dangerous_cast() if a 
cast dangerous, because it e.g. reinterprets raw memory bytes, or 
aliases immutable memory to mutable memory?
Mar 08 2009
prev sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
Derek Parnell escribió:

 On Sat, 07 Mar 2009 23:40:52 -0800, Walter Bright wrote:
 

 I'm still not understanding you, because this is a contrived example 
 that I cannot see the point of nor can I see where it would be 
 legitimately used.


 
 I can see Burton's concern, and I'm very surprised that the compiler allows
 this to happen. Here is a slightly more explicit version of Burton's code.
 
 import std.stdio;
 void main()
 {
   int [] a = new int [1];
 
    a [0] = 1;
 
    invariant (int) [] b = cast (invariant (int) []) a;
 
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
    a [0] += b [0];
    writef ("a=%s b=%s\n", a [0], b[0]);
 }
    
 The problem is that we have declared 'b' as invariant, but the program is
 allowed to change it. That is the issue.


As I see it, the cast should fail saying "can't cast a to invariant 
because a is mutable", because otherwise you are producing a behaviour 
that doesn't match what the code says.

In:

char c;
int* p = cast(int*)&c;
*p = 5;

there's nothing wrong about the cast because a char can be viewed as an 
int. But a mutable data cannot be seen as immutable.

Walter, you say "Whenever you use a cast, the onus is on the programmer 
to know what they are doing." That's true. But if the programmer made a 
mistake and she wasn't knowing what she was doing, the compiler or the 
runtime should yell.

For example:

class A { }

class B : A { }

class C : A { }

void foo(A a) {
   C c = cast(C) a;
}

The programmer is sure "a" is really of type "C". If it is, ok, 
everything works. Now, assuming it is not, this yields a null "c" which 
fails at runtime if "c" is used (which will be used eventually because 
that's why we are casting it). The failure is a big one: the program 
halts. (well, I'd prefer an exception to be thrown, but that's another 
issue). Now, if you allow a cast from mutable to immutable to continue, 
the error at runtime will be very subtle and really hard to find, 
because the program will continue to work but with a wrong behaviour.
Mar 08 2009
parent Sergey Gromov <snake.scaly gmail.com> writes:
Sun, 08 Mar 2009 13:01:27 -0200, Ary Borenszweig wrote:


 As I see it, the cast should fail saying "can't cast a to invariant 
 because a is mutable", because otherwise you are producing a behaviour 
 that doesn't match what the code says.
 
 In:
 
 char c;
 int* p = cast(int*)&c;
 *p = 5;
 
 there's nothing wrong about the cast because a char can be viewed as an 
 int. But a mutable data cannot be seen as immutable.


It's not interpreting char as int.  It's interpreting c and part of
return address as an int on x86 essentially corrupting stack.  Or gets
an unaligned access hardware exception.  Or whatever.  It's anything but
a correct code.  It's exactly the same issue as casting mutable to
immutable: you're abusing the power the compiler gave you.

Though I agree that the line between safe and unsafe casts is rather
subtle.
Mar 08 2009
prev sibling parent reply Burton Radons <burton.radons gmail.com> writes:
Walter Bright Wrote:


 Burton Radons wrote:

 Walter Bright Wrote:
 

 Burton Radons wrote:

 This wouldn't be too bad because const is worthless when
 optimising, but if invariant is going to be given any weight then
 we must never cause data to be casted to invariant unless if it's
 actually invariant data. So, the sensible default is "const
 (char) []" for strings, a selection of aliases in object.d for
 the others, and safe casting templates in object.d.


 What I interpret from this is that you see strings as fundamentally
  mutable character arrays, and sometimes in special cases you can
 make them immutable. I propose turning that view on its head -
 regard strings as fundamentally immutable, and having a mutable
 char array is a rare thing that only appears in isolated places in
 the program.


 
 No, I don't. You are misunderstanding me, and I'm not sure why or
 how.


 
 I guess I just cannot figure out where you're coming from.
 

 Here's a (contrived) example of where my concern may come into
 play:
 
 int [] a = new int [1];
 
 a [0] = 1;
 
 auto b = cast (invariant (int) []) a;
 
 a [0] += b [0]; a [0] += b [0]; writef ("%s\n", a [0]); // Normal
 result: 4. // Optimiser which assumes invariant data can't change: 3
 
 Yes, the code is an abuse of the const system. THAT'S EXACTLY MY
 POINT. Casting mutable data to invariant leads to situations like
 these. Only data which will never change can be made invariant.
 Putting "alias invariant (char) [] string" in object.d induces these
 situations and makes it seem like it's a good idea.


 
 I'm still not understanding you, because this is a contrived example 
 that I cannot see the point of nor can I see where it would be 
 legitimately used.


Obviously I made it contrived so that it's as clear as possible what the issue
is. In reality, it will be going through more layers. Here's one layer:

   int [] a = new int [1];
   a [0] = 1;

   invariant (int) [] func (invariant (int) [] a) { return a; }

   auto b = func (cast (invariant (int) []) a); 

Notice this has the same pattern as std.string.replace; that's why I did that
cast.

   a [0] += b [0];
   a [0] += b [0];
   writef ("%s\n", a [0]);
   // Not optimised: 4.
   // Assuming b cannot be modified: 3.

When this actually crops up in bugs the reality will be far more complex and
practically impossible to discover.

I think I've stated this warning a half-dozen times in the last three days, and
that's it, I'm done.
Mar 08 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Burton Radons wrote:

 Obviously I made it contrived so that it's as clear as possible what
 the issue is. In reality, it will be going through more layers.
 Here's one layer:
 
 int [] a = new int [1]; a [0] = 1;
 
 invariant (int) [] func (invariant (int) [] a) { return a; }
 
 auto b = func (cast (invariant (int) []) a);
 
 Notice this has the same pattern as std.string.replace; that's why I
 did that cast.
 
 a [0] += b [0]; a [0] += b [0]; writef ("%s\n", a [0]); // Not
 optimised: 4. // Assuming b cannot be modified: 3.
 
 When this actually crops up in bugs the reality will be far more
 complex and practically impossible to discover.



The solution is, when you cast things to invariant, that's a one way 
ticket. You cannot continue to mutate the original. If you find you need 
to, the data must be duplicated, and the duplicate made invariant. If 
this is done a lot, it's time to refactor the program.

Casts are greppable, and should be reviewed anyway. In this case, 
casting an array to immutable should be followed by NO further 
references to the mutable version.
Mar 08 2009
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Burton Radons wrote:

    int [] a = new int [1];
 
    a [0] = 1;
 
    auto b = cast (invariant (int) []) a;
 
    a [0] += b [0];
    a [0] += b [0];
    writef ("%s\n", a [0]);
    // Normal result: 4.
    // Optimiser which assumes invariant data can't change: 3
 
 Yes, the code is an abuse of the const system. THAT'S EXACTLY MY POINT.
Casting mutable data to invariant leads to situations like these. Only data
which will never change can be made invariant. Putting "alias invariant (char)
[] string" in object.d induces these situations and makes it seem like it's a
good idea.


You're going into undefined territory and complain that it doesn't work 
as you expect. Perhaps that should issue a warning, but you're doing 
something wrong and bad: you're saying that the same array is both 
mutable and immutable.

Think of the other approach: once you cast an array to invariant, the 
compiler finds all aliases of that array and turns them invariant. You'd 
be even more upset in that case. It would have long-reaching effects 
that are hard to track down.

Or you could forbid the cast. But it's a useful cast, and you really 
can't get rid of it if you ever want to convert something that is 
mutable to something that is invariant without copying.
Mar 08 2009
parent Jason House <jason.james.house gmail.com> writes:
Christopher Wright Wrote:


 Burton Radons wrote:

    int [] a = new int [1];
 
    a [0] = 1;
 
    auto b = cast (invariant (int) []) a;
 
    a [0] += b [0];
    a [0] += b [0];
    writef ("%s\n", a [0]);
    // Normal result: 4.
    // Optimiser which assumes invariant data can't change: 3
 
 Yes, the code is an abuse of the const system. THAT'S EXACTLY MY POINT.
Casting mutable data to invariant leads to situations like these. Only data
which will never change can be made invariant. Putting "alias invariant (char)
[] string" in object.d induces these situations and makes it seem like it's a
good idea.


 
 You're going into undefined territory and complain that it doesn't work 
 as you expect. Perhaps that should issue a warning, but you're doing 
 something wrong and bad: you're saying that the same array is both 
 mutable and immutable.
 
 Think of the other approach: once you cast an array to invariant, the 
 compiler finds all aliases of that array and turns them invariant. You'd 
 be even more upset in that case. It would have long-reaching effects 
 that are hard to track down.
 
 Or you could forbid the cast. But it's a useful cast, and you really 
 can't get rid of it if you ever want to convert something that is 
 mutable to something that is invariant without copying.


A cast could be avoided if the compiler could track unique mutable references.
Then assignment to invariant could be done implicitly and make the mutable
reference no longer exist. This would require escape analysis for mutable
references. I like allowing implicit invariant casts, but I seem to be in the
minority. Doing that brings further language complexity.
Mar 08 2009
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Burton Radons wrote:

 Walter Bright Wrote:
 

 Burton Radons wrote:

 That's what we said about strings in 1.0. You modify it, you copy
 it, or you tell the user. The gentleman's agreement worked
 perfectly and that came without a mess of keywords, without
 implicit or explicit restrictions on behaviour, without having to
 condition templates.


 The one flaw in it was the behavior I consistently saw of "I'm
 copying the string just to be sure I own it and nobody else changes
 it." D was meant for copy-on-write, which means copy the string
 *only* if you change it. No defensive copying. No "just in case"
 copying. The gentleman's agreement failed as far as I could tell.
 
 With immutable strings, the gentleman's agreement is enforced.


 
 Am I going to become a broken record on this? Because "invariant
 (char) []" is the string type, data that is going to be mutable will
 always find its way into that type in order to deal with an API which
 WILL use string as its arguments, not writing out "const (char) []".
 It gives me no information about the future of the object while
 removing the apparent need for the gentleman's agreement. Therefore I
 have no way of knowing what the actual pedigree of this string I've
 been given has. It may be invariant, it may be mutable.
 
 I want this to be addressed directly. Exactly how am I wrong on this
 point? Is it not conceivable that mutable data gets casted to
 invariant in this case?


It is conceivable by means of a cast. I've explained that casts can 
break any of D's guarantees, so there is nothing new that you can 
masquerade a mutable string into an immutable one. If there was a means 
to implicitly convert a mutable string into an immutable one, you'd have 
a case. But it either looks like you're not understanding something, or 
are using a double standard when it comes about casting as applied to 
immutability in particular.

There is one point where we are forced to doing something gauche: 
assumeUnique. We could have avoided that by introducing a "unique" 
notion, but we thought we'd simplify the language by not doing so. So 
far the uses of assumeUnique seem to be idiomatic and contained enough 
to not be a threat, so it seems to have been a passable engineering 
decision.

To recap, if an API takes a string and all you have a char[], DO NOT 
CAST IT. Call .idup - better safe than sorry. The API may evolve and 
store a reference for later. Case in point: the up-and-coming 
std.stdio.File constructor initially was:

this(in char[] filename);

Later on I decided to save the filename for error message reporting and 
the such. Now I had two choices:

(1) Leave the signature unchanged and issue an idup:

this.filename = to!string(filename); // issues an idup

(2) Change the signature to

this(string filename);

Now all client code that DID pass a string in the first place (the vast 
majority) was safe _and_ efficient. The minority of client code was that 
that had a char[] or a const(char)[] at hand. That code did not compile, 
so it had to insert a to!string on the caller side.

As has been copiously shown in other languages, the need for 
character-level mutable string is rather rare. So most of the time you 
will not traffic in char[], but instead you'll have a immutable(char)[] 
to start with. This further erodes the legitimacy of your concern.

I have no idea how to make this any more clearer. I explained it so many 
times and in so many ways, even I understood it :o).


Andrei
Mar 07 2009
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-03-07 18:19:50 -0500, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Later on I decided to save the filename for error message reporting and 
 the such. Now I had two choices:
 
 (1) Leave the signature unchanged and issue an idup:
 
 this.filename = to!string(filename); // issues an idup
 
 (2) Change the signature to
 
 this(string filename);


Third choice: have two contructors.

	this(in char[] filename) { this(filename.idup); }
	this(string filename) { ... }

It's pretty easy to add a new version of a construcor or function when 
you change things and need to keep backward compatibility. The question 
is: should you?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Mar 07 2009
prev sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Sat, 07 Mar 2009 15:19:50 -0800, Andrei Alexandrescu wrote:


 To recap, if an API takes a string and all you have a char[], DO NOT 
 CAST IT. Call .idup - better safe than sorry. The API may evolve and 
 store a reference for later. Case in point: the up-and-coming 
 std.stdio.File constructor initially was:
 
 this(in char[] filename);
 
 Later on I decided to save the filename for error message reporting and 
 the such. Now I had two choices:
 
 (1) Leave the signature unchanged and issue an idup:
 
 this.filename = to!string(filename); // issues an idup
 
 (2) Change the signature to
 
 this(string filename);
 
 Now all client code that DID pass a string in the first place (the vast 
 majority) was safe _and_ efficient. The minority of client code was that 
 that had a char[] or a const(char)[] at hand. That code did not compile, 
 so it had to insert a to!string on the caller side.
 
 As has been copiously shown in other languages, the need for 
 character-level mutable string is rather rare. So most of the time you 
 will not traffic in char[], but instead you'll have a immutable(char)[] 
 to start with. This further erodes the legitimacy of your concern.


My file names are constructed most of the time.  And most of the time
they are simple char[]s.

It is not obvious that File should store the file name.  It's not
strictly necessary.  It's an *implementation detail.*  Now you expose
this implementation detail through the class interface, and you do this
without any good reason.  You save a 150 byte allocation per file.
Nice.

I can understand when a hash takes an immutable key.  It's in the hash's
contract.  Various lazy functions could take immutable input to
guarantee correct lazy execution.  But I think that overall use of
immutable types should be rare and thoroughly thought-out.  They should
be used only when it's absolutely, provably necessary.  That's why I
think aliasing string as immutable is a mistake.  It felt wrong when I
discovered D a year ago, and it feels wrong now.
Mar 07 2009
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Sergey Gromov wrote:

 Sat, 07 Mar 2009 15:19:50 -0800, Andrei Alexandrescu wrote:
 

 To recap, if an API takes a string and all you have a char[], DO NOT 
 CAST IT. Call .idup - better safe than sorry. The API may evolve and 
 store a reference for later. Case in point: the up-and-coming 
 std.stdio.File constructor initially was:

 this(in char[] filename);

 Later on I decided to save the filename for error message reporting and 
 the such. Now I had two choices:

 (1) Leave the signature unchanged and issue an idup:

 this.filename = to!string(filename); // issues an idup

 (2) Change the signature to

 this(string filename);

 Now all client code that DID pass a string in the first place (the vast 
 majority) was safe _and_ efficient. The minority of client code was that 
 that had a char[] or a const(char)[] at hand. That code did not compile, 
 so it had to insert a to!string on the caller side.

 As has been copiously shown in other languages, the need for 
 character-level mutable string is rather rare. So most of the time you 
 will not traffic in char[], but instead you'll have a immutable(char)[] 
 to start with. This further erodes the legitimacy of your concern.


 
 My file names are constructed most of the time.  And most of the time
 they are simple char[]s.


Ehm. Mine are also constructed, but somehow come in string format, e.g.:

string basename;
...
auto f = File(basename ~ ".txt");


 It is not obvious that File should store the file name.  It's not
 strictly necessary.  It's an *implementation detail.*  Now you expose
 this implementation detail through the class interface, and you do this
 without any good reason.  You save a 150 byte allocation per file.
 Nice.


It's just an example, the point being that there things are always fast 
and safe. In many cases there's much more at stake and you can't rely on 
idioms that allocate memory needlessly.


 I can understand when a hash takes an immutable key.  It's in the hash's
 contract.  Various lazy functions could take immutable input to
 guarantee correct lazy execution.  But I think that overall use of
 immutable types should be rare and thoroughly thought-out.  They should
 be used only when it's absolutely, provably necessary.  That's why I
 think aliasing string as immutable is a mistake.  It felt wrong when I
 discovered D a year ago, and it feels wrong now.


That may be because you are writing C in D. Immutable strings should 
allow solid coding without much friction.


Andrei
Mar 07 2009
parent reply Jason House <jason.james.house gmail.com> writes:
Andrei Alexandrescu Wrote:


 Sergey Gromov wrote:

 Sat, 07 Mar 2009 15:19:50 -0800, Andrei Alexandrescu wrote:
 

 To recap, if an API takes a string and all you have a char[], DO NOT 
 CAST IT. Call .idup - better safe than sorry. The API may evolve and 
 store a reference for later. Case in point: the up-and-coming 
 std.stdio.File constructor initially was:

 this(in char[] filename);

 Later on I decided to save the filename for error message reporting and 
 the such. Now I had two choices:

 (1) Leave the signature unchanged and issue an idup:

 this.filename = to!string(filename); // issues an idup

 (2) Change the signature to

 this(string filename);

 Now all client code that DID pass a string in the first place (the vast 
 majority) was safe _and_ efficient. The minority of client code was that 
 that had a char[] or a const(char)[] at hand. That code did not compile, 
 so it had to insert a to!string on the caller side.

 As has been copiously shown in other languages, the need for 
 character-level mutable string is rather rare. So most of the time you 
 will not traffic in char[], but instead you'll have a immutable(char)[] 
 to start with. This further erodes the legitimacy of your concern.


 
 My file names are constructed most of the time.  And most of the time
 they are simple char[]s.


 
 Ehm. Mine are also constructed, but somehow come in string format, e.g.:
 
 string basename;
 ...
 auto f = File(basename ~ ".txt");
 

 It is not obvious that File should store the file name.  It's not
 strictly necessary.  It's an *implementation detail.*  Now you expose
 this implementation detail through the class interface, and you do this
 without any good reason.  You save a 150 byte allocation per file.
 Nice.


 
 It's just an example, the point being that there things are always fast 
 and safe. In many cases there's much more at stake and you can't rely on 
 idioms that allocate memory needlessly.


Your example above does allocate memory. A mutable string could potentially
avoid allocating to append ".txt"

 

 I can understand when a hash takes an immutable key.  It's in the hash's
 contract.  Various lazy functions could take immutable input to
 guarantee correct lazy execution.  But I think that overall use of
 immutable types should be rare and thoroughly thought-out.  They should
 be used only when it's absolutely, provably necessary.  That's why I
 think aliasing string as immutable is a mistake.  It felt wrong when I
 discovered D a year ago, and it feels wrong now.


 
 That may be because you are writing C in D. Immutable strings should 
 allow solid coding without much friction.
 
 
 Andrei
Mar 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jason House wrote:

 Andrei Alexandrescu Wrote:
 

 Sergey Gromov wrote:

 Sat, 07 Mar 2009 15:19:50 -0800, Andrei Alexandrescu wrote:


 To recap, if an API takes a string and all you have a char[], DO NOT 
 CAST IT. Call .idup - better safe than sorry. The API may evolve and 
 store a reference for later. Case in point: the up-and-coming 
 std.stdio.File constructor initially was:

 this(in char[] filename);

 Later on I decided to save the filename for error message reporting and 
 the such. Now I had two choices:

 (1) Leave the signature unchanged and issue an idup:

 this.filename = to!string(filename); // issues an idup

 (2) Change the signature to

 this(string filename);

 Now all client code that DID pass a string in the first place (the vast 
 majority) was safe _and_ efficient. The minority of client code was that 
 that had a char[] or a const(char)[] at hand. That code did not compile, 
 so it had to insert a to!string on the caller side.

 As has been copiously shown in other languages, the need for 
 character-level mutable string is rather rare. So most of the time you 
 will not traffic in char[], but instead you'll have a immutable(char)[] 
 to start with. This further erodes the legitimacy of your concern.


 My file names are constructed most of the time.  And most of the time
 they are simple char[]s.


 Ehm. Mine are also constructed, but somehow come in string format, e.g.:

 string basename;
 ...
 auto f = File(basename ~ ".txt");


 It is not obvious that File should store the file name.  It's not
 strictly necessary.  It's an *implementation detail.*  Now you expose
 this implementation detail through the class interface, and you do this
 without any good reason.  You save a 150 byte allocation per file.
 Nice.


 It's just an example, the point being that there things are always fast 
 and safe. In many cases there's much more at stake and you can't rely on 
 idioms that allocate memory needlessly.


 
 Your example above does allocate memory. A mutable string could potentially
avoid allocating to append ".txt"


It does, and for a good reason - File stores an alias of it. If it 
didn't have to, it would have accepted const, in which case a mutable 
string would have sufficed.

Andrei
Mar 07 2009
parent Jason House <jason.james.house gmail.com> writes:
Andrei Alexandrescu wrote:


 Jason House wrote:

 Andrei Alexandrescu Wrote:
 

 Sergey Gromov wrote:

 Sat, 07 Mar 2009 15:19:50 -0800, Andrei Alexandrescu wrote:


 To recap, if an API takes a string and all you have a char[], DO NOT
 CAST IT. Call .idup - better safe than sorry. The API may evolve and
 store a reference for later. Case in point: the up-and-coming
 std.stdio.File constructor initially was:

 this(in char[] filename);

 Later on I decided to save the filename for error message reporting
 and the such. Now I had two choices:

 (1) Leave the signature unchanged and issue an idup:

 this.filename = to!string(filename); // issues an idup

 (2) Change the signature to

 this(string filename);

 Now all client code that DID pass a string in the first place (the
 vast majority) was safe _and_ efficient. The minority of client code
 was that that had a char[] or a const(char)[] at hand. That code did
 not compile, so it had to insert a to!string on the caller side.

 As has been copiously shown in other languages, the need for
 character-level mutable string is rather rare. So most of the time you
 will not traffic in char[], but instead you'll have a
 immutable(char)[] to start with. This further erodes the legitimacy of
 your concern.


 My file names are constructed most of the time.  And most of the time
 they are simple char[]s.


 Ehm. Mine are also constructed, but somehow come in string format, e.g.:

 string basename;
 ...
 auto f = File(basename ~ ".txt");


 It is not obvious that File should store the file name.  It's not
 strictly necessary.  It's an *implementation detail.*  Now you expose
 this implementation detail through the class interface, and you do this
 without any good reason.  You save a 150 byte allocation per file.
 Nice.


 It's just an example, the point being that there things are always fast
 and safe. In many cases there's much more at stake and you can't rely on
 idioms that allocate memory needlessly.


 
 Your example above does allocate memory. A mutable string could
 potentially avoid allocating to append ".txt"


 
 It does, and for a good reason - File stores an alias of it. If it
 didn't have to, it would have accepted const, in which case a mutable
 string would have sufficed.



I think you missed my point, but I was partly being a devil's advocate...  I 
probably should not be fanning the flames for this thread, so I'll be quiet
Mar 07 2009
prev sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Sergey Gromov wrote:

 But I think that overall use of
 immutable types should be rare and thoroughly thought-out.  They should
 be used only when it's absolutely, provably necessary.


I suggest that that's exactly backwards <g>. Mutable types should be the 
rare, carefully considered ones.


 That's why I
 think aliasing string as immutable is a mistake.  It felt wrong when I
 discovered D a year ago, and it feels wrong now.


I know it feels wrong. That's the C background talking. I went through 
the same thing. It's sort of like OOP if you're used to C. It takes a 
while before it clicks, in the meantime, it feels wrong and stupid.
Mar 07 2009
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:

This is an interesting topic.
I like immutability, but sometimes I also like mutability.


languages that are considered very good at dealing with strings (like Perl) use
immutable strings. The fascinating thing about strings in such languages is:
"Nobody notices they are immutable, they just work."<


Languages that have immutable strings often have:
- "String interning", to improve performance...
- A good garbage collector, to cope with the increased allocation-deallocation
traffic.
- Sometimes the garbage collector is able to see that two unrelated strings are
equal, and keep only one of them. Experiments have shown this reduces a lot the
memory used by many Java programs.
- Strings often keep their hash value stored beside them, so it's computed only
once, the first time you actually need the hash value (this also means the hash
value is initialized to an unvalid value).

People notice such strings are immutable. Usually it's fine, but once in a
while it's a pain.

Note that in Python you usually try to avoid looping on single chars because
it's a too much slow thing to do, so you try to use string methods and regular
expressions as much as possible. But I like a lower level language because it
gives me the *freedom* to read and process the single chars efficiently.
(Python is implemented in C, and writing the Python interpreter itself with a
language that uses immutable strings only is probably a pain).

Such languages like Python also always offer you an escape, for example in the
standard library of Python there is a mutable char array (Python3 is different,
it has as built-ins immutable unicode strings + mutable arrays of bytes + maybe
an immutable array of bytes):

This is Python 2.5:


 from array import array

 s = array("c", "Hello")
 s






array('c', 'Hello')

 s[2] = "X"
 s






array('c', 'HeXlo')


 t = array("u", u"Hello")
 t






array('u', u'Hello')

 t[1]= u"Y"
 t






array('u', u'HYllo')


Also note that Ruby, that is a very good language for string processing, allows
mutable strings. So the situation isn't as clear cut as you think.

To compare such matters across languages you can also take a look at this page:
http://merd.sourceforge.net/pixel/language-study/various/mutability-and-sharing/



After all, it never occurs to anyone to think that the integer 123 could be a
"mutable" integer and perhaps be 133 the next time you look at it.<


Because they are small numbers. With the multi-precision GMP library you can
mutate numbers in place because this becomes useful when you manage huge
numbers.

Note that there are Python bindings for GMP, they manage numbers in an
immutable way to respect the Python style, but it's not much efficient, see
explanation here in the middle:
http://gmpy.sourceforge.net/



The way to do strings in D is to have them be immutable. If you are building a
string by manipulating its parts, start with mutable, when finished then
convert it to immutable and 'publish' it to the rest of the program.<


Seems acceptable.



you'll find you never need to defensively dup the string "just in case" and
things just seem to naturally work out.<


If you put strings in an associative array as keys, you usually want them to be
immutable to keep their correct place in the hash and avoid big troubles.
For such purpose Python has mutable and immutable arrays (named list and
tuple), where you can only use tuples as dictionary keys.
So built-in associative arrays of D too may appreciate immutable arrays more :-)

Bye,
bearophile
Mar 07 2009
parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


The way to do strings in D is to have them be immutable. If you are building a
string by manipulating its parts, start with mutable, when finished then
convert it to immutable and 'publish' it to the rest of the program.<


Most of the times this seems acceptable.
But if such text is very long (example, 20 MB) and you want to pass it around
for various functions to process&modify it, they you may want to keep it
mutable (this is a quite uncommon situation, but it's happened to me, during
genomic data processing).

Bye,
bearophile
Mar 07 2009
prev sibling parent jerry quinn <jlquinn optonline.net> writes:
Walter Bright Wrote:


 When we first got into what to do with strings and 
 const/immutable/mutable, I was definitely in the camp that strings 
 should be mutable char[], or at worst const(char)[]. The thing is, 
 Andrei pointed out to me, languages that are considered very good at 
 dealing with strings (like Perl) use immutable strings. The fascinating 
 thing about strings in such languages is:
 
 "Nobody notices they are immutable, they just work."
 
 So what is it about immutability that makes strings "just work" in a 
 natural and intuitive manner? The insight is that it enables strings, 
 which are reference types, to behave exactly as if they were value types.
 
 After all, it never occurs to anyone to think that the integer 123 could 
 be a "mutable" integer and perhaps be 133 the next time you look at it. 
 If you put 123 into a variable, it stays 123. It's immutable. People 
 intuitively expect strings to behave the same way. Only C programmers 
 expect that once they assign a string to a variable, that string may 
 change in place.
 
 C has it backwards by making strings mutable, and it's one of the main 
 reasons why dealing with strings in C is such a gigantic pain. But as a 
 longtime C programmer, I was so used to that I didn't notice what a pain 
 it was until I started using other languages where string manipulation 
 was a breeze.
 
 The way to do strings in D is to have them be immutable. If you are 
 building a string by manipulating its parts, start with mutable, when 
 finished then convert it to immutable and 'publish' it to the rest of 
 the program. Mutable char[] arrays should only exist as temporaries. 
 This is exactly the opposite of the way one does it in C, but if you do 
 it this way, you'll find you never need to defensively dup the string 
 "just in case" and things just seem to naturally work out.


So your suggestion is to do something like:

string manipulate() {
  char[] buf = read_20M_string_data();
  initialization_mangling(buf);
  return (string)buf;
}
string 20M_string = manipulate();

As long as we don't want to mangle the storage later in the program.
Mar 07 2009
prev sibling parent Derek Parnell <derek psych.ward> writes:
On Fri, 06 Mar 2009 14:56:04 -0800, Andrei Alexandrescu wrote:


 Steven Schveighoffer wrote:

 I think what Burton is saying is by annointing immutable(char)[] as the 
 type "string," you are essentially sending a message to developers that 
 all strings should be immutable, and all *string parameters* should be 
 declared immutable.





 Phobos is being changed to accept in char[] instead of string
 wherever applicable. As far as what the default "string" ought to be,
 immutable(char)[] is the safest of the three so I think it should be that.


I vaguely remember someone suggesting that "string" be the alias for
immutable character arrays and "text" be the alias for mutable character
arrays. For some people, it might be easier to relate the word "text" as
being something that can be edited in-place.

I'm not advocating or rejecting this ... just trying to recall the original
poster's suggestion.

-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell
Mar 06 2009
prev sibling parent reply Burton Radons <burton.radons shaw.ca> writes:
Andrei Alexandrescu Wrote:


 In D you will be able to break any design with a cast, unless you use 
 the not-yet-defined D2 which disallows all risky casts. So the fact that 
 you can cast const away is hardly changing anything.


Oh, and this is an idea which is almost exactly 20 years old, from back when
ANSI was developing the C standard. They had defined const but then made it so
that const cannot be casted off, making the language unimplementable.

We cannot avoid these APIs which led to this unimplementability because it's a
common problem: a mutable object goes through an environment where it is
treated as const, but afterwards needs to continue to be mutable. For example,
you might have:

  alias const (char) [] string;
  alias char [] mstring;

  // Return the first match within the string, or null if there is no match.
  string match (string text, RE expression);

  mstring text;

  auto submatch = match (text, expression);
  submatch [] = ' '; // Fails.

The only non-casting alternative is to define:

  mstring mmatch (mstring text, RE expression);

But there are two problems. One, it's the same exact function which does the
same exact thing. So you're wasting the programmer's time for moving match into
a template function and then overloading it twice, you're wasting the reader's
time for having to learn two functions and knowing when to apply either, and
you're wasting the processor's time for having to load both functions. More
critically, it's not descriptive; it in fact implies that mmatch may or will
modify the data within the text, when it won't, ever. So if the purpose is to
make const describe the nature of the implementation of the function, it has
completely failed and so can't be trusted anywhere.

Dennis Ritchie argued against const when it appeared
(http://www.lysator.liu.se/c/dmr-on-noalias.html).
Mar 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Burton Radons wrote:

 Andrei Alexandrescu Wrote:
 

 In D you will be able to break any design with a cast, unless you use 
 the not-yet-defined D2 which disallows all risky casts. So the fact that 
 you can cast const away is hardly changing anything.


 
 Oh, and this is an idea which is almost exactly 20 years old, from back when
ANSI was developing the C standard. They had defined const but then made it so
that const cannot be casted off, making the language unimplementable.
 
 We cannot avoid these APIs which led to this unimplementability because it's a
common problem: a mutable object goes through an environment where it is
treated as const, but afterwards needs to continue to be mutable. For example,
you might have:
 
   alias const (char) [] string;
   alias char [] mstring;
 
   // Return the first match within the string, or null if there is no match.
   string match (string text, RE expression);
 
   mstring text;
 
   auto submatch = match (text, expression);
   submatch [] = ' '; // Fails.
 
 The only non-casting alternative is to define:
 
   mstring mmatch (mstring text, RE expression);
 
 But there are two problems. One, it's the same exact function which does the
same exact thing. So you're wasting the programmer's time for moving match into
a template function and then overloading it twice, you're wasting the reader's
time for having to learn two functions and knowing when to apply either, and
you're wasting the processor's time for having to load both functions. More
critically, it's not descriptive; it in fact implies that mmatch may or will
modify the data within the text, when it won't, ever. So if the purpose is to
make const describe the nature of the implementation of the function, it has
completely failed and so can't be trusted anywhere.


This is a known problem for hich we will provide a solution.

Andrei
Mar 06 2009
parent reply Burton Radons <burton.radons shaw.ca> writes:
Andrei Alexandrescu Wrote:


 This is a known problem for hich we will provide a solution.


If you have something which works everywhere please tell us because we've been
trying to find one for a long time, but as far as I know there is no solution.
The best I've ever gotten to is:

   // "A" means that the constness of the return type depends upon the
constness of the argument. There are dozens of ways to specify the same thing.
   const (A) mstring match (const (A) mstring text, RE expression);

But setting aside whether that helps or hinders self-documentation, that's far
from the only place at which you put mutable data through a const section that
you need to modify later. What if the function were instead:

   struct REMatch
   {
      string match; /// The matched string.
      size_t offset; /// Offset within the string where the match occurs.
      string [] groups; /// Matched groups.

      this (string text);
   }

What am I going to do about this now without using templates? If you define a
special syntax to make this work, then I can give you something even further
which won't.
Mar 06 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Burton Radons wrote:

 Andrei Alexandrescu Wrote:
 

 This is a known problem for hich we will provide a solution.


 
 If you have something which works everywhere please tell us because
 we've been trying to find one for a long time, but as far as I know
 there is no solution. The best I've ever gotten to is:
 
 // "A" means that the constness of the return type depends upon the
 constness of the argument. There are dozens of ways to specify the
 same thing. const (A) mstring match (const (A) mstring text, RE
 expression);
 
 But setting aside whether that helps or hinders self-documentation,
 that's far from the only place at which you put mutable data through
 a const section that you need to modify later. What if the function
 were instead:
 
 struct REMatch { string match; /// The matched string. size_t offset;
 /// Offset within the string where the match occurs. string []
 groups; /// Matched groups.
 
 this (string text); }
 
 What am I going to do about this now without using templates? If you
 define a special syntax to make this work, then I can give you
 something even further which won't.


The problem is you set up artificially constrained rules, i.e. "without 
using templates". You can't use the same struct to store mutable types 
and non-mutable types mixed with always-mutable types, and for good 
reasons. No type system will allow 100% of the correct programs to run. 
Why the fuss. Use a gorram template and call it a day.

Andrei
Mar 06 2009
parent reply Burton Radons <burton.radons gmail.com> writes:
Andrei Alexandrescu Wrote:


 Burton Radons wrote:

 Andrei Alexandrescu Wrote:
 

 This is a known problem for hich we will provide a solution.


 
 If you have something which works everywhere please tell us because
 we've been trying to find one for a long time, but as far as I know
 there is no solution. The best I've ever gotten to is:
 
 // "A" means that the constness of the return type depends upon the
 constness of the argument. There are dozens of ways to specify the
 same thing. const (A) mstring match (const (A) mstring text, RE
 expression);
 
 But setting aside whether that helps or hinders self-documentation,
 that's far from the only place at which you put mutable data through
 a const section that you need to modify later. What if the function
 were instead:
 
 struct REMatch { string match; /// The matched string. size_t offset;
 /// Offset within the string where the match occurs. string []
 groups; /// Matched groups.
 
 this (string text); }
 
 What am I going to do about this now without using templates? If you
 define a special syntax to make this work, then I can give you
 something even further which won't.


 
 The problem is you set up artificially constrained rules, i.e. "without 
 using templates". You can't use the same struct to store mutable types 
 and non-mutable types mixed with always-mutable types, and for good 
 reasons. No type system will allow 100% of the correct programs to run. 
 Why the fuss. Use a gorram template and call it a day.


Ah, so you don't have a solution.

What you'll have instead are programs which are developed one way but then come
to an impasse where they need to cast off const but can't. So after some
cursing, the programmer starts wasting his time modifying all of his code to be
templated, because avoiding casting off const has exactly the same viral
progression as adding in const (I did it in C++ the last time we thought const
might be able to make code better-optimised).

This goes fine, until he comes up to a library which hasn't gone through the
same process, so it has a normal interface. Or he comes up to an interface
itself, which will not normally be templatable. He doesn't have the code, so he
can't change the library. What does he do then?

There are four options I can see. One, he could make a copy of the supposedly
const data that's been trapped by the library, which won't always work. Two, he
could take the pointer and length from the slice and figure out where in his
mutable data the slice exists at, which might be impossible depending upon
pointer arithmetic restrictions. Three, he could reimplement the library
functionality himself, which may be impossible. Four, he could move on to a
language which doesn't make his job hard just so that it can sometimes add
numbers faster.

This is actually reminding me of C++ the more I think about it. C++'s bad
features are so bad because they're far-reaching but they couldn't be
consistently applied. So if you read the specification, you'll find twenty or
so caveats that try to make the feature work, when the proper thing to have
done was to realise that a feature which doesn't naturally fit in a language
shouldn't be in that language. Yet here we have a feature that's not just C++,
but it's C++^2.

Multiple keywords. The threat that abuse will eventually lead to code being
compiled incorrectly, coupled with forcing abuse on common code (I stress that
any data which is marked as invariant at any point but is not actually
invariant will cause optimisation issues if it's given any weight whatsoever).
At least three different ways to define a const. "const int *foo ()" and "int
*foo () const" equivalency. A syntax which makes declarations hard to read. And
coming, you say, is enforced const-correctness, just to make things extra awful.
Mar 06 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Burton Radons wrote:

 Andrei Alexandrescu Wrote:

 The problem is you set up artificially constrained rules, i.e.
 "without using templates". You can't use the same struct to store
 mutable types and non-mutable types mixed with always-mutable
 types, and for good reasons. No type system will allow 100% of the
 correct programs to run. Why the fuss. Use a gorram template and
 call it a day.


 
 Ah, so you don't have a solution.


I do have a solution, the problem is sometimes no amount of convincing 
will do any good. Arguments focusing on niche cases can be formulated 
against every single restriction of a type system. Const and immutable 
are supposed to express realities about data. Sometimes said realities 
undergo dialectics that are difficult to express. The system is not 
perfect, and cannot be made perfect within the constraints at hand (e.g. 
without putting undue complexity on the programmer). If you're bent from 
the get-go against const and immutable, there is no good scenario that 
is good enough, no bad scenario that's infrequent and avoidable enough, 
and no way to purport a gainful dialog.

Andrei
Mar 06 2009
prev sibling parent Burton Radons <burton.radons gmail.com> writes:
Derek Parnell Wrote:


 On Sat, 07 Mar 2009 17:08:58 -0500, Burton Radons wrote:
 

 Am I going to become a broken record on this? Because
 "invariant (char) []" is the string type, data that
 is going to be mutable will always find its way into
 that type in order to deal with an API which WILL use
 string as its arguments, not writing out
 "const (char) []". 


 
 I'm starting to think that 'string' for function parameters should be a
 rare thing. For a function to insist that it only recieves immutable data
 sounds like the function is worried that it might accidently change data.
 And that sounds like a bug to me. It is shifting the responsibility to the
 caller for the data's integrity.
 

 It gives me no information about
 the future of the object while removing the apparent
 need for the gentleman's agreement. Therefore I have
 no way of knowing what the actual pedigree of this
 string I've been given has. It may be invariant, it
 may be mutable.


 
 But why would your function care about that? Let's assume your function's
 signature is 'const' for its parameters because it does not intend to
 modify any of them. If the caller passes invariant data then your function
 cannot modify the arguments. If the caller passes mutable data, the
 compiler won't allow your function to modify the parameters either, due to
 the const signature. So why is it important that the function should know
 the mutability of the passed data?


In my example the function's signature is like this:

   int cachedComputation (string x)
   {
      static int [string] array;

      return array [x] = hugeComputation (x);
   }

According to the strict interpretation of the standard I should be able to say
that the contents of x will never be modified or destroyed, but because it's
the default that's used everywhere in Phobos that forces everything to be
casted to string before it can be used anywhere, I can't assume it has more
weight than const.
Mar 07 2009