www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - -dip25 switch: time to make it always on?

reply Walter Bright <newshound2 digitalmars.com> writes:
It was added for 2.067 back in March of 2015:

https://dlang.org/changelog/2.067.0.html

It's been a strong success. Time to make it the default instead of enabled by a 
switch?
Jun 29 2016
next sibling parent reply Dicebot <public dicebot.lv> writes:
Please don't. There is a dire need to fix the implementation 
first, currently DIP25 is almost unusable for anyone who doesn't 
know exact limits of the feature - compiler diagnostics are 
pretty much non-existent. Right now it is miles away from what 
I'd call a success.

See this d.learn discussion 
(http://forum.dlang.org/thread/vtvtukooicwspxzzoxtt forum.dlang.org) for
example of diagnostics problems I am talking about.

There are also plenty of open issues in bugzilla about current 
implementation : 
https://issues.dlang.org/buglist.cgi?quicksearch=dip25&list_id=209219
Jun 29 2016
next sibling parent Martin Nowak <code dawg.eu> writes:
On Wednesday, 29 June 2016 at 10:02:08 UTC, Dicebot wrote:

 Please don't. There is a dire need to fix the implementation 
 first, currently DIP25 is almost unusable for anyone who 
 doesn't know exact limits of the feature - compiler diagnostics 
 are pretty much non-existent. Right now it is miles away from 
 what I'd call a success.


I actually doubt that anyone but dlang itself is using it.
Only enabling it with a dmd compiler switch means it unlikely to 
be used by any dub package. I also don't understand why we left 
our well established model of adding new errors as deprecations, 
would've integrated much better with existing build tools. Same 
goes for this -transition= thing.
Jun 29 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 3:02 AM, Dicebot wrote:

 See this d.learn discussion
 (http://forum.dlang.org/thread/vtvtukooicwspxzzoxtt forum.dlang.org) for
example
 of diagnostics problems I am talking about.


Please post bug reports to bugzilla. Brad set up bugzilla because using the
n.g. 
as a buglist was a complete failure.



 There are also plenty of open issues in bugzilla about current implementation :
 https://issues.dlang.org/buglist.cgi?quicksearch=dip25&list_id=209219


The current curated list of  safe issues:

https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&keywords=safe&keywords_type=allwords&list_id=209224&query_format=advanced
Jun 29 2016
parent reply Dicebot <public dicebot.lv> writes:
On 06/29/2016 02:53 PM, Walter Bright wrote:

 On 6/29/2016 3:02 AM, Dicebot wrote:

 See this d.learn discussion
 (http://forum.dlang.org/thread/vtvtukooicwspxzzoxtt forum.dlang.org)
 for example
 of diagnostics problems I am talking about.


 
 Please post bug reports to bugzilla. Brad set up bugzilla because using
 the n.g. as a buglist was a complete failure.


I have submitted a short summary of that ng.learn thread here :
https://issues.dlang.org/show_bug.cgi?id=16228

My overall opinion of DIP25 is that it must not become part of language
mainline for two reasons:

1) Implementation is not sufficiently mature (mostly diagnostics issues)
2) It targets a very niche use case and is too much of a language burden
on its own - there are no interesting designs one can do with DIP25
alone to "sell" the feature.

In my opinion it should sit in a preview state until `return scope` or
whatever replaces it gets finalized and tested and merged in main
language simultaneously as parts of same feature.
Jul 02 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/2/2016 2:55 AM, Dicebot wrote:

 I have submitted a short summary of that ng.learn thread here :
 https://issues.dlang.org/show_bug.cgi?id=16228


Thanks! I'll fix them.



 My overall opinion of DIP25 is that it must not become part of language
 mainline for two reasons:

 1) Implementation is not sufficiently mature (mostly diagnostics issues)


Of all the reports on it, there's nothing yet that makes it fundamentally
wrong. 
Just a few garden variety bugs. Furthermore, none of the bugs resulted in any 
sort of corruption that was not in the code already without -dip25, i.e. it did 
no harm.



 2) It targets a very niche use case and is too much of a language burden
 on its own - there are no interesting designs one can do with DIP25
 alone to "sell" the feature.


Since 'ref' is a critical feature, 'return ref' is needed to ensure that 
functions are not returning references to the stack. It's necessary for the 
fundamental 'identity' function to work safely.

You're right that there are no interesting designs that dip25 enables - there 
are no designs at all that dip25 enables. But that isn't it's purpose. It's 
purpose is to diagnose uses of ref that would cause corruption.

I don't understand why it would be a language burden. The idea behind this 
design is to require the user to add as few annotations as possible, and in
this 
it has been (inadvertently according to deadalnix!) rather successful.

'return ref' is intended to provide safety without turning D into a B+D
language.

All it really takes is finding one instance of memory corruption to make it 
worthwhile. I've lost many hours of my life searching for the causes of memory 
corruption. Recently on Reddit there were several tales of unsafe code costing 
corporations huge $$$$ in remedial action and innumerable customers suffered as 
malware blasted their systems. A solution is a big deal - even an incomplete 
solution is better than no solution. (Even though I fully intend to make D's 
solution complete.)



 In my opinion it should sit in a preview state until `return scope` or
 whatever replaces it gets finalized and tested and merged in main
 language simultaneously as parts of same feature.


'return scope' is coming soon. But that's a bigger cognitive leap, and is
indeed 
a separate problem.

BTW, my work in applying  safe to Phobos modules has uncovered one overflow bug 
so far (and fixed).  safe, while incomplete, is far from useless.

As for being in a preview state, this thread amply demonstrates that is 
ineffective, as few to nobody even tries it. Maybe it would be more effective 
with a larger community.
Jul 02 2016
parent Jacob Carlborg <doob me.com> writes:
On 02/07/16 14:00, Walter Bright wrote:


 I don't understand why it would be a language burden.


One can argue that every new keyword (this happens to reuse two existing 
keywords) or every new feature is burden.

-- 
/Jacob Carlborg
Jul 03 2016
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 29 June 2016 at 10:02:08 UTC, Dicebot wrote:

 Please don't. There is a dire need to fix the implementation 
 first, currently DIP25 is almost unusable for anyone who 
 doesn't know exact limits of the feature - compiler diagnostics 
 are pretty much non-existent. Right now it is miles away from 
 what I'd call a success.

 See this d.learn discussion 
 (http://forum.dlang.org/thread/vtvtukooicwspxzzoxtt forum.dlang.org) for
example of diagnostics problems I am talking about.

 There are also plenty of open issues in bugzilla about current 
 implementation : 
 https://issues.dlang.org/buglist.cgi?quicksearch=dip25&list_id=209219


I don't think DIP25 is salvageable. The bugs here can be fixed, 
but that won't mean much. The approach is unprincipled. If there 
is something to be learn from the  safe story is that it doesn't 
work as a general approach.

Additionally, the fact that is isn't used much tells us that it 
is probably too limited to be useful enough to pay for itself.
Jun 29 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 3:02 AM, Dicebot wrote:

 There are also plenty of open issues in bugzilla about current implementation :
 https://issues.dlang.org/buglist.cgi?quicksearch=dip25&list_id=209219


14238 awaits improvements to 'scope'.
15293 has an outstanding PR.

The rest either had fixes merged or were not actually -dip25 problems.
Jul 01 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 29/06/16 11:40, Walter Bright wrote:

 It was added for 2.067 back in March of 2015:

 https://dlang.org/changelog/2.067.0.html

 It's been a strong success. Time to make it the default instead of
 enabled by a switch?


How is it a success? Grepping on "return ref" gives back four results in 
Phobos and one in druntime.

-- 
/Jacob Carlborg
Jun 29 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 6:45 AM, Jacob Carlborg wrote:

 How is it a success? Grepping on "return ref" gives back four results in Phobos
 and one in druntime.


It was designed to solve the returning reference problem with as few
annotations 
as possible. Needing only 4 annotations in the entire runtime library and yet 
have plugged the holes is quite nice.

Also, there are more cases the grep doesn't show, i.e.:

     ref foo() return { ... }
Jun 29 2016
parent Meta <jared771 gmail.com> writes:
On Wednesday, 29 June 2016 at 20:51:56 UTC, Walter Bright wrote:

 On 6/29/2016 6:45 AM, Jacob Carlborg wrote:

 How is it a success? Grepping on "return ref" gives back four 
 results in Phobos
 and one in druntime.


 It was designed to solve the returning reference problem with 
 as few annotations as possible. Needing only 4 annotations in 
 the entire runtime library and yet have plugged the holes is 
 quite nice.


I don't claim to know all of the places it can or should be 
applied in Phobos, but this seems to me like it indicates either 
return is not strict enough or it is not used everywhere it 
should be. We know for sure from that learn thread that there is 
at least one case that it doesn't catch.
Jun 29 2016
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 29 June 2016 at 09:40:45 UTC, Walter Bright wrote:

 It's been a strong success.


tbh, I've never been able to figure out what it is even supposed 
to do and thus never actually used it, despite being interested 
in the subject.
Jun 29 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 8:26 AM, Adam D. Ruppe wrote:

 tbh, I've never been able to figure out what it is even supposed to do and thus
 never actually used it, despite being interested in the subject.


One way to find out is to turn it on and let it do the work for you.
Jun 29 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 29 June 2016 at 20:52:52 UTC, Walter Bright wrote:

 On 6/29/2016 8:26 AM, Adam D. Ruppe wrote:

 tbh, I've never been able to figure out what it is even 
 supposed to do and thus
 never actually used it, despite being interested in the 
 subject.


 One way to find out is to turn it on and let it do the work for 
 you.


but *what* work? i've read DIP many times, each time i thought 
that i get the idea, and then, next day... stop-stop-stop, what 
is that DIP25 again? i gave up after tenth time or so. i'd better 
go learn haskel instead of DIP25, that DIP is too hard for me to 
even remember, and impossible to learn and understand.
Jun 29 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 5:08 PM, ketmar wrote:

 On Wednesday, 29 June 2016 at 20:52:52 UTC, Walter Bright wrote:

 On 6/29/2016 8:26 AM, Adam D. Ruppe wrote:

 tbh, I've never been able to figure out what it is even supposed to do and thus
 never actually used it, despite being interested in the subject.


 One way to find out is to turn it on and let it do the work for you.


 but *what* work?


It'll give you error messages for unsafe attempts to return references. No
error 
messages, and you're good to go.
Jun 29 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Thursday, 30 June 2016 at 01:54:27 UTC, Walter Bright wrote:

 On 6/29/2016 5:08 PM, ketmar wrote:

 On Wednesday, 29 June 2016 at 20:52:52 UTC, Walter Bright 
 wrote:

 On 6/29/2016 8:26 AM, Adam D. Ruppe wrote:

 tbh, I've never been able to figure out what it is even 
 supposed to do and thus
 never actually used it, despite being interested in the 
 subject.


 One way to find out is to turn it on and let it do the work 
 for you.


 but *what* work?


 It'll give you error messages for unsafe attempts to return 
 references. No error messages, and you're good to go.


thank you. this is the first time i can really make at least some 
sense of it. so, basically, in  safe code i should explicitly 
mark returning `ref`s, or compiler will complain?
Jun 29 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/29/2016 7:02 PM, ketmar wrote:

 thank you. this is the first time i can really make at least some sense of it.
 so, basically, in  safe code i should explicitly mark returning `ref`s, or
 compiler will complain?


Yup. That's all there is to it.
Jun 29 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, June 29, 2016 02:40:45 Walter Bright via Digitalmars-d wrote:

 It was added for 2.067 back in March of 2015:

 https://dlang.org/changelog/2.067.0.html

 It's been a strong success. Time to make it the default instead of enabled
 by a switch?


Well, I don't know how you can really tell whether it's been a success or
not. The only real evidence we have has to do with druntime, Phobos, and
maybe dmd - which certainly isn't nothing, but we really don't know how much
anyone has been using it. I suspect that almost no one has been. And if we
want to test it more globally, we're pretty much going to have to make it
the default (presumably by initially making it just print warning messages
which aren't affected by -w so as to avoid breaking code) and see what
happens. I certainly wouldn't suggest just making what -dip25 does suddenly
apply to everyone with no transition period.

That being said, I think that we need to be very sure that this is where we
want to go, and there always seems to have been a fair bit of disagreement
over that. Personally, I don't know. One big problem I see is that having an
attribute that you have to use in  safe code but not in other code really
does not play well with attribute inference, and you were talking about
making attribute inference even more widespread as part of the ref-counting
improvements.

And while I know that there are not a lot of open bugs related to -dip25, it
didn't exactly make me feel confident in the implementation, when I played
around with it a few minutes ago and hit a bug right away:

https://issues.dlang.org/show_bug.cgi?id=16226

I honestly expect that almost no one has been using -dip25 (I certainly
haven't), and I think that saying that it's been a success is overstating
things considerably. If we want to see how well it works, we're really going
to need to make it the default behavior (presumably not as an error to begin
with), though obviously, that's going to be a problem if we then decide that
it was a mistake. :|

I think that dip25 may very well be the right way to go, but I also fear
that it's too narrow to really get us what we want - similar to how inout
helps with certain things but doesn't really get us what we want because
it's too narrow. So, I'm not opposed to moving forward with it, but I'm not
at all confident that it's the right choice.

- Jonathan M Davis
Jul 01 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/1/2016 6:28 PM, Jonathan M Davis via Digitalmars-d wrote:

 https://issues.dlang.org/show_bug.cgi?id=16226


https://github.com/dlang/dmd/pull/5900

DMD will go ahead and infer the 'return' attribute when it is inferring other 
attributes for a function. This helps a lot to reduce the number of extra 
annotations required without reducing effectiveness.

The bug was it didn't do the inference right.
Jul 02 2016