www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP 1008 Preliminary Review Round 1

reply Mike Parker <aldacron gmail.com> writes:
DIP 1008 is titled "Exceptions and  nogc".

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

All review-related feedback on and discussion of the DIP should 
occur in this thread. The review period will end at 11:59 PM ET 
on June 2 (3:59 AM GMT June 3), or when I make a post declaring 
it complete.

At the end of Round 1, if further review is deemed necessary, the 
DIP will be scheduled for another round. Otherwise, it will be 
queued for the formal review and evaluation by the language 
authors.

Extensive discussion of this DIP has already taken place in two 
threads, both linked from the document. You may find it 
beneficial to skim through those threads before posting any 
feedback here.

Thanks in advance to all who participate.

Destroy!
May 19 2017
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 19/05/2017 4:45 PM, Mike Parker wrote:

 DIP 1008 is titled "Exceptions and  nogc".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

 All review-related feedback on and discussion of the DIP should occur in
 this thread. The review period will end at 11:59 PM ET on June 2 (3:59
 AM GMT June 3), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the formal review and evaluation by the language authors.

 Extensive discussion of this DIP has already taken place in two threads,
 both linked from the document. You may find it beneficial to skim
 through those threads before posting any feedback here.

 Thanks in advance to all who participate.

 Destroy!


"Code that needs to leak the thrown exception object can clone the object."

Errors:
```D
import std.stdio;
void main() {
	auto e = new Exception("foo");
	e = e.dup;
	writeln(e.toString());	
}
```

Let's just say, I am unaware of a way to duplicate classes. Assuming I'm 
messing something, a code example would be very appreciated in the DIP.
May 19 2017
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, May 19, 2017 at 05:13:34PM +0100, rikki cattermole via Digitalmars-d
wrote:
[...]

 "Code that needs to leak the thrown exception object can clone the
 object."
 
 Errors:
 ```D
 import std.stdio;
 void main() {
 	auto e = new Exception("foo");
 	e = e.dup;
 	writeln(e.toString());	
 }
 ```
 
 Let's just say, I am unaware of a way to duplicate classes. Assuming
 I'm messing something, a code example would be very appreciated in the
 DIP.


AFAIK, there is no way to clone classes, unless the class writer
implemented it explicitly. Only arrays support .dup, no other type does
(unless user code explicitly implements it, or its equivalent).


T

-- 
This is not a sentence.
May 19 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:

 On Fri, May 19, 2017 at 05:13:34PM +0100, rikki cattermole via 
 Digitalmars-d wrote: [...]

 "Code that needs to leak the thrown exception object can clone 
 the object."
 
 Errors:
 ```D
 import std.stdio;
 void main() {
 	auto e = new Exception("foo");
 	e = e.dup;
 	writeln(e.toString());
 }
 ```
 
 Let's just say, I am unaware of a way to duplicate classes. 
 Assuming I'm messing something, a code example would be very 
 appreciated in the DIP.


 AFAIK, there is no way to clone classes, unless the class 
 writer implemented it explicitly. Only arrays support .dup, no 
 other type does (unless user code explicitly implements it, or 
 its equivalent).


 T


Well, not that it's a complete implementation or anything, but it 
definitely could be done with a library:

/// Make a shallow clone of a class instance
/// (members that are classes are not cloned)
C clone(C)(C src) if (is(C == class))
{
     // could probably just dynamic cast and copy
     // from most derived...
     assert(typeid(src) == typeid(C));
     // can use any memory allocator here,
     // assuming deallocation is handled correctly
     import core.memory : GC;
     import core.exception : onOutOfMemoryError;
     auto ptr = GC.malloc(__traits(classInstanceSize, C));
     ptr || onOutOfMemoryError;
     scope (failure) GC.free(ptr);

     C dst = cast(C)ptr;

     import std.traits : BaseClassesTuple;
     import std.meta : AliasSeq;
     import core.stdc.string : memcpy;
     foreach(T; AliasSeq!(C, BaseClassesTuple!C)) {
         T bsrc = src;
         T bdst = dst;
         foreach(i, ref _; bsrc.tupleof) {
             alias M = typeof(_);
             memcpy(&bdst.tupleof[i], &bsrc.tupleof[i], M.sizeof);
             static if (__traits(hasMember, M, "__postblit"))
                 bdst.tupleof[i].__postblit;
         }
     }

     return dst;
}

void main() {
     import std.stdio;
     struct Struct {
         int[] data;
         this(this) {
             data = data.dup;
             writeln("Struct is copied");
         }
     }

     class Klass {
         int value;
         Struct s;
         this(int v) {
             value = v;
             s = Struct([1, 2, 3, 4, 5]);
         }
     }

     auto c1 = new Klass(42);
     auto c2 = c1.clone;
     assert(c2);
     assert(c2 !is c1);
     assert(c2.value == 42);
     assert(c2.s.data);
     assert(c2.s.data !is c1.s.data);
     assert(c2.s.data == c1.s.data);
}
May 19 2017
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, May 19, 2017 at 05:48:55PM +0000, Stanislav Blinov via Digitalmars-d
wrote:

 On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:


[...]

 AFAIK, there is no way to clone classes, unless the class writer
 implemented it explicitly. Only arrays support .dup, no other type does
 (unless user code explicitly implements it, or its equivalent).
 
 
 T


 
 Well, not that it's a complete implementation or anything, but it
 definitely could be done with a library:
 
 /// Make a shallow clone of a class instance
 /// (members that are classes are not cloned)


[...]

That's the problem right there: you cannot guarantee that a shallow copy
will have the correct semantics.  For example, modifying a class member
through the original copy will also modify what is seen through the
clone, which may not be the intention.

Also, you have to take care that if the class member is ref-counted, the
clone function needs to increment the refcount, otherwise you could end
up with a dangling pointer. Meaning, in the general case, you need to be
careful about calling postblits and such.

Furthermore, a generic deep-copying clone function may not be possible
in the general case, because sometimes you don't know if a reference
member is intended to be a reference to external data, or an owned value
that needs to be deep-copied. For example, if a class C encapsulates a
container of references to data, then C.clone should not copy the data,
but if a class D is a container of references to data that it owns, then
the data should be copied. A generic function cannot decide which one it
is unless the intent is made clear, e.g. via a standard UDA, or some
other such means.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:
May 19 2017
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Friday, 19 May 2017 at 18:10:50 UTC, H. S. Teoh wrote:

 On Fri, May 19, 2017 at 05:48:55PM +0000, Stanislav Blinov via 
 Digitalmars-d wrote:

 On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:


 [...]

 AFAIK, there is no way to clone classes, unless the class 
 writer implemented it explicitly. Only arrays support .dup, 
 no other type does (unless user code explicitly implements 
 it, or its equivalent).
 
 
 T


 
 Well, not that it's a complete implementation or anything, but 
 it definitely could be done with a library:
 
 /// Make a shallow clone of a class instance
 /// (members that are classes are not cloned)


 [...]

 That's the problem right there: you cannot guarantee that a 
 shallow copy will have the correct semantics.  For example, 
 modifying a class member through the original copy will also 
 modify what is seen through the clone, which may not be the 
 intention.


That's true.


 Also, you have to take care that if the class member is 
 ref-counted, the clone function needs to increment the 
 refcount, otherwise you could end up with a dangling pointer.


It does call postblits. But of course, a complete implementation 
should do more than that. It probably has to lock the monitor for 
the duration, correctly annotate the GC block if memory is being 
GC-allocated, etc...


 Meaning, in the general case, you need to be careful about 
 calling postblits and such.

 Furthermore, a generic deep-copying clone function may not be 
 possible in the general case...


Of course. For such cloning to be officially supported by the 
language, at least one additional primitive is needed: __postblit 
for classes. This will let user code correctly handle cloning as 
needed. Surely, a much better solution would be actual 
owning/non-owning pointers and references, but I doubt we'd ever 
get there.
May 19 2017
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 ...


I have already made my objections known in the other threads, but 
I'll list them here for posterity.

Firstly, and most importantly IMO, this does not solve the 
surface level problem, which is the lack of  nogc in much of 
Phobos. While the DIP is correct that in a lot of the absence of 
 nogc is due to exceptions, most notably the auto decoding 
functions in std.range, it not the only reason. There are many 
things in Phobos which stop  nogc, such as array literals, array 
appending, AAs, and delegates. While many functions can be made 
 nogc via this change, there will be many functions left in 
Phobos still allocating, and this is a major problem in my 
estimation. The reason I take this all or nothing view of the 
situation is that  nogc exists largely for a group of people who 
want to avoid the GC __completely__. If lack of any GC usage is 
actually a requirement for a project, having many very useful 
parts of Phobos off limits (a few that spring to mind are bigint, 
digest, and a lot of stdio) is still a problem when our 
competitors in this GC free arena are Rust and C++ which will 
both have a lot more functionality in this limited scope.

Secondly, I'm not a fan of special casing syntax, especially when 
I don't think the given benefits outweigh the above listed costs. 
Making operator new do something completely different in one 
specific context is a continuation of the trend in the recent 
past of making the language more and more complicated, which I 
find troubling as one of D's main selling points is it's 
practicality and its low learning curve.

And while I don't think the code breakage will be a huge problem, 
it still is something to consider when asking what are we getting 
for all of these costs.

Thirdly, I don't think the DIP asks the right question. The DIP 
asks


 How can we make exceptions, and thereby many Phobos functions, 
  nogc?


IMO the right question to ask is a level of analysis above that. 
As already mentioned, there are many things which will still be 
allocated on the GC. We need to fix these as well if we want to 
market D as a GC optional language. Instead, we should be asking


 How can allocations in Phobos and user code be transferred to 
 an allocation strategy of the user's choosing/needs?


If the user values code simplicity and safety, then the GC is 
fine, and template functions which allocate can then be marked as 
 safe due to their use of GC. But if the user needs other speed 
or usage requirements, then the user needs to offer something 
else, and that will automatically be marked as  system. I think 
that integrating std.allocator across Phobos in a consistent 
manner will allow this, where functions can accept allocators, 
and their internal allocations can all be done on the given 
allocator. This would also allow exceptions and other objects can 
be manually freed after a function is called if needed, or just 
left to the GC to collect if the GC allocator was passed to the 
function.

In summary, I believe what Phobos needs is a wholistic approach 
to the problem of memory management, and not a bunch of small 
solutions over time. To be sure, this is a much harder task, but 
I think if it's done right, with D we can have the first language 
which has compete user control to those who need it while 
offering ease of use to everyone else.
May 19 2017
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, May 19, 2017 at 06:16:33PM +0000, Jack Stouffer via Digitalmars-d wrote:
[...]

 Instead, we should be asking
 

 How can allocations in Phobos and user code be transferred to an
 allocation strategy of the user's choosing/needs?


 
 If the user values code simplicity and safety, then the GC is fine,
 and template functions which allocate can then be marked as  safe due
 to their use of GC. But if the user needs other speed or usage
 requirements, then the user needs to offer something else, and that
 will automatically be marked as  system. I think that integrating
 std.allocator across Phobos in a consistent manner will allow this,
 where functions can accept allocators, and their internal allocations
 can all be done on the given allocator. This would also allow
 exceptions and other objects can be manually freed after a function is
 called if needed, or just left to the GC to collect if the GC
 allocator was passed to the function.


I agree with this, and having looked at std.experimental.allocator
recently, I think Andrei may even have made certain design decisions
with this in mind.  I think the ideal goal (I'm not sure how achievable
it is) would be to make theAllocator part of *druntime*, upon which you
can actually use array concatenations, AA's, etc., (almost) freely, and
the user would be able to control exactly how allocations would work. At
least in the current state of things, I don't see this as being
particularly possible due to various issues, the main one being that
code written with GC in mind generally does not track lifetimes, which
is a big obstacle to successfully being able to just assign a different
allocator to theAllocator and have things Just Work(tm).

Short of putting theAllocator in druntime outright, the next best thing
would be for all Phobos code to use it by default, eschewing any
constructs that might bypass it.  This seems more attainable, though it
does imply a LOT of churn in Phobos, and may require significant time
and effort to pull off.  Some Phobos code, of course, might be able to
take in allocators as template parameters or some such mechanism of
finer-grained control, but given the very large amount of code we're
dealing with, defaulting to theAllocator seems like a (slightly) more
reachable goal.



 In summary, I believe what Phobos needs is a wholistic approach to the
 problem of memory management, and not a bunch of small solutions over
 time.  To be sure, this is a much harder task, but I think if it's
 done right, with D we can have the first language which has compete
 user control to those who need it while offering ease of use to
 everyone else.


Yes, D needs to return to its original vision of getting the fundamental
design decisions right, rather than trying to retroactively patch over
existing flaws.  Otherwise, we risk becoming the next C++, which would
be a great pity.


T

-- 
"Computer Science is no more about computers than astronomy is about
telescopes." -- E.W. Dijkstra
May 19 2017
prev sibling next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Friday, 19 May 2017 at 18:16:33 UTC, Jack Stouffer wrote:

 On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 ...





 Secondly, I'm not a fan of special casing syntax, especially 
 when I don't think the given benefits outweigh the above listed 
 costs...


You're raising an extremely important point. Special-casing 
exceptions in such a way, while may help *some* code, is still a 
crutch, and a very ugly one at that. The bigger issue of  nogc is 
the implicit allocations. It would be much better to solve the 
fundamental problem, that is, give user code optional, but 
complete control over memory allocation.

All three main areas need to be addressed in concert: classes, 
arrays and delegates. Working on just one area, and even just a 
subset of it (exceptions) in the long run will further the 
problem, not help solve it.

As an example, operator new could accept additional syntax (outer 
[] mean "optional"):

new [(TAllocator)] Type [[count]] [(init)]

Where TAllocator is a class providing at least minimal allocator 
interface (allocate/deallocate).  nogc can be inferred from the 
allocator.
This lets user code decide on their class and array allocations. 
OTOH, this will incur storage penalty, since all 
dynamically-allocated objects (including arrays) will have to 
carry the allocator reference with them. It doesn't, however, 
solve the  nogc inference in general, since allocator is not part 
of the type. Attempting to concatenate two arrays at a distance 
from their allocation site would not be considered  nogc. That 
being said, arrays with complete inference support could be made 
library types, so perhaps that is not *that* big of an issue.

Delegates, as they are now, require a lot more attention. 
Firstly, there is the dubious design decision that any outer 
scope access is always by reference, which imposes the GC 
allocation of closures in the first place.  nogc cannot be solved 
for delegates without addressing control over captures. Then we 
have the allocation itself, in cases when it is desired. And that 
will completely destroy existing delegate syntax.

There is, however, another side to this problem, and that is 
interaction of exceptions with the runtime. Allowing users to 
employ any desired allocation scheme precludes reliable exception 
handling, since the runtime effectively loses control over 
exception lifetime. Catching garbage in a catch(...) block would 
never be a pleasant surprise. That being said, it's not that it 
is in control right now either:

void innocent() {
     throw new Exception("I am innocent");
}

void malicious() {
     try {
         innocent();
     } catch (Exception e) {
         // innocent, eh? well I am not...
         e.destroy;
         throw e;
     }
}

void oblivious() {
     try {
         malicious();
     } catch (Exception e) {
         writeln("Caught: ", e.msg);
     }
}

void main() {
     oblivious();
}

Hello, segmentation fault. If users want to be difficult, they'd 
find a way :)

So perhaps, instead of tackling exceptions on their own, we 
really should focus on allocation in general. Everything else 
should (hopefully) come automagically.
May 19 2017
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, May 19, 2017 11:35:54 AM PDT H. S. Teoh via Digitalmars-d wrote:

 I agree with this, and having looked at std.experimental.allocator
 recently, I think Andrei may even have made certain design decisions
 with this in mind.  I think the ideal goal (I'm not sure how achievable
 it is) would be to make theAllocator part of *druntime*, upon which you
 can actually use array concatenations, AA's, etc., (almost) freely, and
 the user would be able to control exactly how allocations would work. At
 least in the current state of things, I don't see this as being
 particularly possible due to various issues, the main one being that
 code written with GC in mind generally does not track lifetimes, which
 is a big obstacle to successfully being able to just assign a different
 allocator to theAllocator and have things Just Work(tm).


Because of the issue of lifetimes, some language features simply cannot be
implemented without the GC, and I think don't see any point in trying to
make it so that you can use all features of D without the GC. That simply
won't work. By the very nature of the language, completely avoiding the GC
means completely avoiding some features.

D's dynamic arrays fundamentally require the GC, because they do not manage
their own memory. They're just a pointer and a length and literally do not
care what memory backs them. As long as all you're doing is slicing them and
passing them around (i.e. restrict yourself to the range-based functions),
then the GC is not involved, and doesn't need to be, but as soon as you
concatenate or append, the GC has to be involved. For that not to be the
case, dynamic arrays would have to manage their own memory (e.g. be
ref-counted), which means that they could not be what they are now. A
different data structure would be required.

Similarly, stuff like closures require the GC. They need something to manage
their memory. They're designed to be automatic.

Honestly, I think that this push for  nogc and manual memory mangement is
toxic. Yes, we should strive to not require the GC where reasonable, but
some things simply are going to require the GC to work well, and avoiding
the GC very quickly gives you a lot of the problems that you have with
languages like C and C++.

For instance, at dconf, Atila talked about the D wrapper for excel that he
wrote. He decided to use  nogc and std.exception.allocator, and not only did
that make it much harder for him to come up with a good, workable design, it
meant that he suddenly had to deal with memory corruption bugs that you
simply never have with the GC. He felt like he was stuck programming in C++
again - only worse, because he had issues with valgrind that made it so that
he couldn't effectively use it to locate his memory corruption problems.

The GC makes it far easier to write clean, memory-safe code. It is a _huge_
boon for us to have the GC. Yes, there are cases where you can't afford to
use the GC, or you have to limit its use in order for your code to be as
performant as it needs to be, but that's the exception, not the norm. And
avoiding the GC comes at a real cost.

I have no problem whatsoever telling folks that some features of D require
the GC and that while D's GC is optional, if you avoid it, you avoid certain
features. There is no free lunch.

We do need to make sure that we do not accidentally or erroneously require
the GC so that code can work with  nogc when folks require that if it's
reasonable for that code to be  nogc. Where reasonable, allocators should be
an option so that folks can decide whether to use the GC or not for a
particular piece of code. But sometimes, it's just not reasonable to expect
the same functionality when not using the GC as you get when using the GC.

And the reality of the matter is that using the GC has real benefits, and
trying to avoid it comes at a real cost, much as a number of C++ progammers
want to complain and deride as soon as they hear that D has a GC. And
honestly, even having  nogc all over the place won't make many of them
happy, because the GC is still in the language.

- Jonathan M Davis
May 19 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2017 8:54 PM, Jonathan M Davis via Digitalmars-d wrote:

 And the reality of the matter is that using the GC has real benefits, and
 trying to avoid it comes at a real cost, much as a number of C++ progammers
 want to complain and deride as soon as they hear that D has a GC. And
 honestly, even having  nogc all over the place won't make many of them
 happy, because the GC is still in the language.


Also, have a GC makes CTFE real nice.
May 20 2017
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 20 May 2017 at 07:02:10 UTC, Walter Bright wrote:

 Also, have a GC makes CTFE real nice.


Having to implement a GC for newCTFE won't be nice though :o)
I agree though being able to allocate memory makes ctfe much more 
useful then it would otherwise be.
May 20 2017
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, May 20, 2017 at 07:53:58AM +0000, Stefan Koch via Digitalmars-d wrote:

 On Saturday, 20 May 2017 at 07:02:10 UTC, Walter Bright wrote:

 
 Also, have a GC makes CTFE real nice.


 
 Having to implement a GC for newCTFE won't be nice though :o)
 I agree though being able to allocate memory makes ctfe much more
 useful then it would otherwise be.


I think we might be able to get away without implementing a GC for
newCTFE.  All you need is a memory pool allocator, i.e., a
bump-the-pointer algorithm to a block of memory (or maybe a linked list
of blocks if we need to make it growable) allocated when you enter CTFE,
then upon exiting CTFE, copy out the return value(s) and free the entire
pool.

As long as we don't anticipate CTFE code that requires massive amounts
of allocation / deallocation before producing a result, this ought to be
good enough.


T

-- 
What do you get if you drop a piano down a mineshaft? A flat minor.
May 20 2017
prev sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, May 20, 2017 12:34:09 PM PDT H. S. Teoh via Digitalmars-d 
wrote:

 On Sat, May 20, 2017 at 07:53:58AM +0000, Stefan Koch via Digitalmars-d 


wrote:

 On Saturday, 20 May 2017 at 07:02:10 UTC, Walter Bright wrote:

 Also, have a GC makes CTFE real nice.


 Having to implement a GC for newCTFE won't be nice though :o)
 I agree though being able to allocate memory makes ctfe much more
 useful then it would otherwise be.


 I think we might be able to get away without implementing a GC for
 newCTFE.  All you need is a memory pool allocator, i.e., a
 bump-the-pointer algorithm to a block of memory (or maybe a linked list
 of blocks if we need to make it growable) allocated when you enter CTFE,
 then upon exiting CTFE, copy out the return value(s) and free the entire
 pool.

 As long as we don't anticipate CTFE code that requires massive amounts
 of allocation / deallocation before producing a result, this ought to be
 good enough.


Well, from the perspective of ther user's code, there really isn't any
difference. They can allocate memory with new, and the compiler takes care
of managing the memory.

- Jonathan M Davis
May 20 2017
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, May 20, 2017 12:02:10 AM PDT Walter Bright via Digitalmars-d 
wrote:

 On 5/19/2017 8:54 PM, Jonathan M Davis via Digitalmars-d wrote:

 And the reality of the matter is that using the GC has real benefits,
 and
 trying to avoid it comes at a real cost, much as a number of C++
 progammers want to complain and deride as soon as they hear that D has
 a GC. And honestly, even having  nogc all over the place won't make
 many of them happy, because the GC is still in the language.


 Also, have a GC makes CTFE real nice.


Yeah, especially when you're not allowed to manually allocate memory in
CTFE. :)

And given that Stephan thinks that ref is too hard to implement in newCTFE
(IIRC from what he said at dconf anyway), I'd hate to think what it would
take to allow something like malloc or free.

- Jonathan M Davis
May 20 2017
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 20 May 2017 at 13:06:01 UTC, Jonathan M Davis wrote:


 Yeah, especially when you're not allowed to manually allocate 
 memory in CTFE. :)

 And given that Stephan thinks that ref is too hard to implement 
 in newCTFE (IIRC from what he said at dconf anyway), I'd hate 
 to think what it would take to allow something like malloc or 
 free.


Did I say that ?

ref is working.

unions and other ABI-related things will be tricky.
May 20 2017
next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, May 20, 2017 1:36:14 PM PDT Stefan Koch via Digitalmars-d 
wrote:

 On Saturday, 20 May 2017 at 13:06:01 UTC, Jonathan M Davis wrote:

 Yeah, especially when you're not allowed to manually allocate
 memory in CTFE. :)

 And given that Stephan thinks that ref is too hard to implement
 in newCTFE (IIRC from what he said at dconf anyway), I'd hate
 to think what it would take to allow something like malloc or
 free.


 Did I say that ?

 ref is working.


I thought you did, but I could have misremembered. Regardless, if it's
working now, all the better. :)


 unions and other ABI-related things will be tricky.


unions are useful for certain things, but really, they're pretty terrible
once you get beyond stuff like int and float. They _can_ be used safely for
more complicated stuff, but it definitely does get tricky. And I'm sure that
it's that much worse with CTFE. :(

- Jonathan M Davis
May 20 2017
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote:

 unions and other ABI-related things will be tricky.


Isn't the unions issue quite easily solved by tagging behind the 
scenes?
May 20 2017
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 20 May 2017 at 14:59:37 UTC, Ola Fosheim Grøstad 
wrote:

 On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote:

 unions and other ABI-related things will be tricky.


 Isn't the unions issue quite easily solved by tagging behind 
 the scenes?


Ah tagging behind the scene is an option, it comes with runtime 
cost though.
And tagging would disallow the tricky and very common usecase of 
overlaying and int and a float.

This I understand, is heavily used.
May 20 2017
next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, May 20, 2017 3:05:44 PM PDT Stefan Koch via Digitalmars-d 
wrote:

 On Saturday, 20 May 2017 at 14:59:37 UTC, Ola Fosheim Grøstad

 wrote:

 On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote:

 unions and other ABI-related things will be tricky.


 Isn't the unions issue quite easily solved by tagging behind
 the scenes?


 Ah tagging behind the scene is an option, it comes with runtime
 cost though.
 And tagging would disallow the tricky and very common usecase of
 overlaying and int and a float.

 This I understand, is heavily used.


std.bitmanip definitely uses that sort of trick (it also does it between a
static array of ubytes) and integer types for stuff like swapping
endianness).

- Jonathan M Davis
May 20 2017
prev sibling next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 20 May 2017 at 15:05:44 UTC, Stefan Koch wrote:

 Ah tagging behind the scene is an option, it comes with runtime 
 cost though.


(I guess you meant compile-time cost)


 And tagging would disallow the tricky and very common usecase 
 of overlaying and int and a float.

 This I understand, is heavily used.


So this is allowed by the language spec? Anyway, since 
compile-time is harder to debug than run-time that seems to be a 
reasonable restriction.

(C++ has this restriction as a general rule: one are only allowed 
to read from a union field if it was the most recently written 
to.)
May 20 2017
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, May 20, 2017 at 03:05:44PM +0000, Stefan Koch via Digitalmars-d wrote:

 On Saturday, 20 May 2017 at 14:59:37 UTC, Ola Fosheim Grøstad wrote:

 On Saturday, 20 May 2017 at 13:36:14 UTC, Stefan Koch wrote:

 unions and other ABI-related things will be tricky.


 
 Isn't the unions issue quite easily solved by tagging behind the
 scenes?


 
 Ah tagging behind the scene is an option, it comes with runtime cost
 though.  And tagging would disallow the tricky and very common usecase
 of overlaying and int and a float.
 
 This I understand, is heavily used.


We only need this for floating-point operations.  The current CTFE
engine already special-cases floats for repainting, so we could
potentially just special-case unions involving floats and leave
everything else unsupported.  Supporting unions in CTFE can be a bear.


T

-- 
Computers aren't intelligent; they only think they are.
May 20 2017
prev sibling parent Jack Stouffer <jack jackstouffer.com> writes:
On Saturday, 20 May 2017 at 13:06:01 UTC, Jonathan M Davis wrote:

 ...


Let's take the CTFE discussion to a different thread
May 20 2017
prev sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Saturday, 20 May 2017 at 03:54:43 UTC, Jonathan M Davis wrote:


 Because of the issue of lifetimes, some language features 
 simply cannot be implemented without the GC, and I think don't 
 see any point in trying to make it so that you can use all 
 features of D without the GC. That simply won't work. By the 
 very nature of the language, completely avoiding the GC means 
 completely avoiding some features.


Even with the GC we have guns to shoot us in the foot with, which 
are .destroy() and GC.free(). The GC itself is not an issue at 
all, it's the lack of choice in the language that is the problem. 
 nogc attribute alone is not enough.


 D's dynamic arrays fundamentally require the GC, because they 
 do not manage their own memory. They're just a pointer and a 
 length and literally do not care what memory backs them. As 
 long as all you're doing is slicing them and passing them 
 around (i.e. restrict yourself to the range-based functions), 
 then the GC is not involved, and doesn't need to be, but as 
 soon as you concatenate or append, the GC has to be involved. 
 For that not to be the case, dynamic arrays would have to 
 manage their own memory (e.g. be ref-counted), which means that 
 they could not be what they are now. A different data structure 
 would be required.


That is not necessary. See my previous comment. We can amend the 
type system so it understands when it can't use the GC.

// Syntax is temporary, for illustration purposes. It is 
currently ambiguous with the language

int[] (nogc) myArray;


auto a = myArray ~ [1,2,3]; // error, cannot concatenate nogc and 
gc arrays.
auto b = myArray ~ [1,2,3] (myAllocator);
// implies compiler-generated:
// auto block = myAllocator.reallocate(myArray, (myArray.length + 
3)*int.sizeof);
// handle out-of-memory, etc...
// int[] (nogc) result = 
(cast(int[])block.ptr)[0..myArray.length+3];
// return result;

Yes, verbose, and yes, ugly. Manual memory management is that. 
But just flat-out forbidding users to use certain features is no 
less verbose and no less ugly.


 Similarly, stuff like closures require the GC. They need 
 something to manage their memory. They're designed to be 
 automatic.


They were designed long ago, perhaps that design needs 
revisiting. They absolutely do not *have* to manage their memory. 
It is convenient when they do and very pleasant when working with 
GC. But that makes them a niche feature at best. If the user is 
given a little bit more control over captures, we'd get more 
cases when allocation is not needed. If the user is given control 
of the allocation itself, even better, as it gives them back a 
feature taken away by the GC.

Explicit (nogc) requirement can be devised for the closures too, 
we just need to put effort into that instead of silently ignoring 
it.


 Honestly, I think that this push for  nogc and manual memory 
 mangement is toxic. Yes, we should strive to not require the GC 
 where reasonable, but some things simply are going to require 
 the GC to work well, and avoiding the GC very quickly gives you 
 a lot of the problems that you have with languages like C and 
 C++.

 For instance, at dconf, Atila talked about the D wrapper for 
 excel that he wrote. He decided to use  nogc and 
 std.exception.allocator, and not only did that make it much 
 harder for him to come up with a good, workable design, it 
 meant that he suddenly had to deal with memory corruption bugs 
 that you simply never have with the GC. He felt like he was 
 stuck programming in C++ again - only worse, because he had 
 issues with valgrind that made it so that he couldn't 
 effectively use it to locate his memory corruption problems.


That is *mostly* due to the lack of facilities in the language 
*and* standard library. It's not written with manual memory 
management in mind and so does not provide any ready-made 
primitives for that, which means you have to write your own, 
which means you will have bugs. At least more bugs than you 
would've had have you had the help.


 The GC makes it far easier to write clean, memory-safe code. It 
 is a _huge_ boon for us to have the GC. Yes, there are cases 
 where you can't afford to use the GC, or you have to limit its 
 use in order for your code to be as performant as it needs to 
 be, but that's the exception, not the norm. And avoiding the GC 
 comes at a real cost.


All is true except the last sentence. The cost should not be 
huge, but for that the language has to work with us. Explicitly, 
without any "special cases".


 And the reality of the matter is that using the GC has real 
 benefits, and trying to avoid it comes at a real cost, much as 
 a number of C++ progammers want to complain and deride as soon 
 as they hear that D has a GC. And honestly, even having  nogc 
 all over the place won't make many of them happy, because the 
 GC is still in the language.


If people simply want to assume the GC is bad and turn away, let 
them. The community or the language won't suffer from it.
OTOH, for people who do have legitimate nogc use cases, we should 
strive to keep as much language facilities as possible.
May 20 2017
prev sibling next sibling parent reply nkm1 <t4nk074 openmailbox.org> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 DIP 1008 is titled "Exceptions and  nogc".


As someone who iis interested in  nogc (more precisely: in 
avoiding GC pauses), I like this proposal... But it looks like 
people are concerned about 'new' becoming contextual keyword 
(that in some contexts it allocates with GC and in others it does 
something else). So maybe different syntax can be used? How about 
"throw scope Exception"? ("scope" already does various things :)


 Instead, we should be asking... How can allocations in Phobos 
 and user code be transferred to an allocation strategy of the 
 user's choosing/needs?


But memory management is not only about allocations, it's also 
about deallocations! Manually deallocating stuff (like freeing 
exception objects) is error prone and bothersome. Refcounted 
exceptions seems to me a pretty good solution (and I think the 
general plan for D is to use more reference counting? At least, 
that's my impression...)
May 19 2017
next sibling parent Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 19 May 2017 at 19:46:07 UTC, nkm1 wrote:

 As someone who iis interested in  nogc (more precisely: in 
 avoiding GC pauses), I like this proposal... But it looks like 
 people are concerned about 'new' becoming contextual keyword 
 (that in some contexts it allocates with GC and in others it 
 does something else). So maybe different syntax can be used? 
 How about "throw scope Exception"? ("scope" already does 
 various things :)


This syntax was already proposed and mostly rejected by the 
community. See the original thread listed in the DIP.


 But memory management is not only about allocations, it's also 
 about deallocations! Manually deallocating stuff (like freeing 
 exception objects) is error prone and bothersome.


That's why we have the GC.


 Refcounted exceptions seems to me a pretty good solution (and I 
 think the general plan for D is to use more reference counting? 
 At least, that's my impression...)


That was my impression too. However Walter has made it clear that 
 safe ref-counting of classes is not planned as he doesn't think 
it's possible. I'm unclear on the details of that, so you'll have 
to ask him.
May 19 2017
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 19 May 2017 at 19:46:07 UTC, nkm1 wrote:

 I like this proposal... But it looks like people are concerned 
 about 'new' becoming contextual keyword (that in some contexts 
 it allocates with GC and in others it does something else)


It *already* does that. `new` can be overloaded and modified with 
`scope`.

http://dlang.org/spec/expression.html

"NewExpressions are used to allocate memory on the garbage 
collected heap (default) or using a class or struct specific 
allocator. "

"If a NewExpression is used as an initializer for a function 
local variable with scope storage class, and the ArgumentList to 
new is empty, then the instance is allocated on the stack rather 
than the heap or using the class specific allocator. "


I also believe the compiler *should* be free to optimize it to a 
different allocation scheme if it proves it safely can.
May 19 2017
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Friday, 19 May 2017 at 21:24:51 UTC, Adam D. Ruppe wrote:


 "NewExpressions are used to allocate memory on the garbage 
 collected heap (default) or using a class or struct specific 
 allocator. "

 "If a NewExpression is used as an initializer for a function 
 local variable with scope storage class, and the ArgumentList 
 to new is empty, then the instance is allocated on the stack 
 rather than the heap or using the class specific allocator. "


IMHO, this has to go. Having alignment control now, and with 
DIP1000 solving reference escaping, there's absolutely no need in 
this special syntax; stack-allocated classes are possible as 
library types.

But it may be beneficial to reconsider the 'new (AllocatorOpts)' 
syntax, with more thought on interaction with the type system. As 
in, it's not necessary to have type-specific allocation 
functions. But some sort of type system flag is required if we 
want to make the language aware of our allocation schemes. To 
expand on my previous reply, something like this comes to mind:

// Delcaring class/struct not supporting being allocated by a 
non- nogc allocator:
class [(nogc)] ClassName [(template parameters)] { ... }
struct [(nogc)] StructName [(template parameters)] { ... }

// Declaring arrays:
T[][(nogc)] arr;

So it could look like this:

class Allocator
{
     void[] allocate(size_t, TypeInfo ti = null)  nogc { ... }
     void deallocate(void[])  nogc { ... }
}

Allocator myAllocator = /* however is desired */;

class (nogc) MyClass
{
     int[] gcArray;
     int[] (nogc) nonGCArray;

     // note the ctor itself is not  nogc, since it allocates
     // gcArray, so interoperability is possible
     this()
     {
         gcArray = [1, 2]; // fine
         nonGCArray = [1, 2]; // error
         nonGCArray = new (myAllocator) int[2];
     }

     ~this()
     {
         myAllocator.dispose(nonGCArray);
     }
}

auto a = new MyClass; // error, no allocator provided, GC 
assumed, MyClass cannot be allocated by GC

auto b = new (myAllocator) MyClass; // fine
auto c = new (theAllocator) MyClass; // error, theAllocator is 
not  nogc

---

Not a very pretty syntax, but I can't think of a way of making it 
any prettier...

Bringing this back to exceptions, it should "just work":

class (nogc) NoGCException : Exception { ... }

throw new (myAllocator) Exception("Argh!");

//...

catch (NoGCException e) {
} // error, e is not rethrown or disposed

catch (NoGCException e) {
     myAllocator.dispose(e);
} // fine, e was disposed


This, however, means teaching the language a few extra library 
constructs. And of course, there's the danger of deallocating 
with the wrong allocator, but being careful comes with the 
territory as far as memory management is concerned.
May 19 2017
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 DIP 1008 is titled "Exceptions and  nogc".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on June 2 (3:59 AM GMT June 3), or when I make a post declaring 
 it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the formal review and evaluation by the language 
 authors.

 Extensive discussion of this DIP has already taken place in two 
 threads, both linked from the document. You may find it 
 beneficial to skim through those threads before posting any 
 feedback here.

 Thanks in advance to all who participate.

 Destroy!


Like others, I do not like the special-casing of `throw new`. 
However, several designs have already been explored and found 
lacking. This proposal also has the advantage that it (hopefully) 
doesn't break existing code and all existing code gets the 
benefit for free. I think this benefit has been hugely 
underestimated; we must consider that large swathes of code that 
were previously non- nogc due to exception allocation can now be 
*inferred automatically* to be  nogc. That's a huge advantage and 
may be worth the special case.

Because of the transitive nature of attributes, even one function 
in the call graph that is non- nogc "paints" connecting nodes 
which in turn paint *their* connecting nodes, etc.
May 19 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2017 6:23 PM, Meta wrote:

 Like others, I do not like the special-casing of `throw new`. However, several
 designs have already been explored and found lacking. This proposal also has
the
 advantage that it (hopefully) doesn't break existing code and all existing code
 gets the benefit for free. I think this benefit has been hugely underestimated;
 we must consider that large swathes of code that were previously non- nogc due
 to exception allocation can now be *inferred automatically* to be  nogc. That's
 a huge advantage and may be worth the special case.


We'll be doing more invisible special casing in the future. For example,

   void foo(scope string s);
   ...
   string s;
   ...
   foo(s ~ "abc");

This array concatenation does not need to be done with the GC. It's not 
fundamentally different from what the compiler does now with:

   s = "abc" ~ "def";

not doing a GC allocation, either. The compiler also detects when it can 
allocate a closure on the stack rather than on the GC. We expect the compiler
to 
do such transformations.

In general, if the compiler can determine the lifetime of an allocation, and
can 
control "leakage" of any references to that allocation, then it is fair game to 
not use the GC for it.

The recent 'scope' improvements have uncovered a lot more opportunities for 
this, and DIP 1008 is the first fruit of it.
May 19 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Saturday, 20 May 2017 at 02:25:45 UTC, Walter Bright wrote:


 We'll be doing more invisible special casing in the future. For 
 example,

   void foo(scope string s);
   ...
   string s;
   ...
   foo(s ~ "abc");

 This array concatenation does not need to be done with the GC. 
 It's not fundamentally different from what the compiler does 
 now with:

   s = "abc" ~ "def";

 not doing a GC allocation, either.


string s = callCAPIAndAllocateString();
foo(s ~ "abc");

What will happen? The compiler will generate different code? 
Won't compile? The former means invisible performance gap. The 
latter means an unpleasant surprise. Neither are good. Do we 
really need such special cases?


 The compiler also detects when it can allocate a closure on the 
 stack rather than on the GC. We expect the compiler to do such 
 transformations.


You're saying "detects" as if it always does that, which is not 
true. And this avoids the issue, not addresses it. We have *no* 
control over how the closure is allocated. It's either none, or 
GC. Which means we can't have dynamically-allocated closures in 
 nogc code. Which forces user to write verbose code (i.e. structs 
with opCall), losing the benefit of anonymous functions 
altogether.


 In general, if the compiler can determine the lifetime of an 
 allocation, and can control "leakage" of any references to that 
 allocation, then it is fair game to not use the GC for it.


*If*. And if it can't, it won't compile. Which is:
a) Frustrating when it should've detected it.
b) Makes you think of allocating things manually from the start, 
to not deal with sudden failure to compile.


 The recent 'scope' improvements have uncovered a lot more 
 opportunities for this, and DIP 1008 is the first fruit of it.


That is true, benefits of 'scope' are indeed huge. But please, 
consider how fragile all the "special-casing" is. It's based on 
preconditions and assumptions made by the compiler, and is beyond 
user control. It hides potential maintenance problems.
May 20 2017
parent Nick Treleaven <nick geany.org> writes:
On Saturday, 20 May 2017 at 09:35:34 UTC, Stanislav Blinov wrote:

 On Saturday, 20 May 2017 at 02:25:45 UTC, Walter Bright wrote:

   void foo(scope string s);


 string s = callCAPIAndAllocateString();
 foo(s ~ "abc");

 What will happen? The compiler will generate different code? 
 Won't compile? The former means invisible performance gap. The 
 latter means an unpleasant surprise. Neither are good. Do we 
 really need such special cases?


It's no different from when s is GC allocated, s[] has to be 
copied for the concatenation. I'm not sure how Walter wants to 
lower this, but maybe it could use a region/stacked allocator. 
That could mean allocation in constant time when the region is 
already big enough, and deallocation would be constant time 
(except if freeing an unused region ahead of the current region).
May 22 2017
prev sibling next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, May 19, 2017 3:45:28 PM PDT Mike Parker via Digitalmars-d wrote:

 DIP 1008 is titled "Exceptions and  nogc".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

 All review-related feedback on and discussion of the DIP should
 occur in this thread. The review period will end at 11:59 PM ET
 on June 2 (3:59 AM GMT June 3), or when I make a post declaring
 it complete.

 At the end of Round 1, if further review is deemed necessary, the
 DIP will be scheduled for another round. Otherwise, it will be
 queued for the formal review and evaluation by the language
 authors.

 Extensive discussion of this DIP has already taken place in two
 threads, both linked from the document. You may find it
 beneficial to skim through those threads before posting any
 feedback here.

 Thanks in advance to all who participate.


Overall, I think that this is a decent proposal for making exceptions  nogc
- and exceptions are definitely one of the few things that prevents code 
that
really should be  nogc from being  nogc. However, I do have two objections:

1. I thought that we were going to come up with a general solution for
ref-counted classes, not something for just exceptions. If that's the case,
then this makes a lot less sense. However, that being said, if this solution
can be transparently transitioned to a more general ref-counting solution,
then this is probably a good stop-gap solution (and if we can't get
ref-counted classes for some reason, then at least we'd have it for
exceptions).

2. This really isn't going to fix the  nogc problem with exceptions without
either seriously overhauling how exceptions are generated and printed or by
having less informative error messages. The problem is with how exception
messages are generated. They take a string, and that pretty much means that
either they're given a string literal (which can be  nogc but does not allow
for customizing the error message with stuff like what the bad input was),
or they're given a constructed string (usually by using format) - and that
can't be  nogc.

And you can't even create an alternate constructor to get around the
problem. Everything relies on the msg member which is set by the
constructor. Code that wants the message accesses msg directly, and when the
exception is printed when it isn't caught, it's msg that is used. Not even
overiding toString gets around the issue. For instance, this code

class E : Exception
{
    this(int i, string file = __FILE__, size_t line = __LINE__)
    {
        super("no message", file, line);
        _i = i;
    }

    override string toString()
    {
        import std.format;
        return format("The value was %s", _i);
    }

    int _i;
}

void main()
{
    throw new E(42);
}

prints

foo.E foo.d(20): no message
----------------
??:? _Dmain [0xd0d473ce]
??:? _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFNlZv 
[0xd0d526db]
??:? scope void rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).tryExec(scope void delegate()) [0xd0d52607]
??:? scope void rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).runAll() [0xd0d52684]
??:? scope void rt.dmain2._d_run_main(int, char**, extern (C) int 
function(char[][])*).tryExec(scope void delegate()) [0xd0d52607]
??:? _d_run_main [0xd0d52577]
??:? main [0xd0d5040b]
??:? __libc_start_main [0xf798a3f0]

toString wasn't used anywhere. toString would only be used if the exception
were caught and printed, e.g.

void main()
{
    import std.stdio;
    writeln(new E(42));
}

would print

The value was 42

whereas without the toString override, it would print

foo.E foo.d(15): no message

So, as is stands, we need to pass strings to exception constructors, and if
we want those messsages to be informative, they cannot be string literals
and thus have to be GC-allocated if you don't want to risk memory leaks or
accessing memory that's no longer valid. And Walter's proposal does nothing
to fix that, so even with his proposal, it's not going to work well to have
code that throws exceptions be  nogc and have the error messages be useful.

What we would probably need would be to change msg is a function which
generates a message so that derived classes can override that rather than
passing a message string. Then at least the allocation of the string would
just happen when the exception was printed. And if it's okay to make getting
the exception  system, then it could even return a const(char)[] to a member
variable (be it a static array or a string) and potentially avoid allocating
altogether. Alternatively, if we're willing to have only one output range
type work with it (so that the function can be virtual), then we could
replace msg with a function that took an output range.

Regardless, with how exceptions are currently constructed, I don't think
that Walter's proposal goes far enough to actually fix the problem with
 nogc and exceptions, even if it does solve a critical piece of the problem.

- Jonathan M Davis
May 19 2017
next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Saturday, 20 May 2017 at 02:05:21 UTC, Jonathan M Davis wrote:

 2. This really isn't going to fix the  nogc problem with 
 exceptions without either seriously overhauling how exceptions 
 are generated and printed or by having less informative error 
 messages. The problem is with how exception messages are 
 generated. They take a string, and that pretty much means that 
 either they're given a string literal (which can be  nogc but 
 does not allow for customizing the error message with stuff 
 like what the bad input was), or they're given a constructed 
 string (usually by using format) - and that can't be  nogc.

 And you can't even create an alternate constructor to get 
 around the problem. Everything relies on the msg member which 
 is set by the constructor. Code that wants the message accesses 
 msg directly, and when the exception is printed when it isn't 
 caught, it's msg that is used. Not even overiding toString gets 
 around the issue. For instance, this code

 class E : Exception
 {
     this(int i, string file = __FILE__, size_t line = __LINE__)
     {
         super("no message", file, line);
         _i = i;
     }

     override string toString()
     {
         import std.format;
         return format("The value was %s", _i);
     }

     int _i;
 }

 void main()
 {
     throw new E(42);
 }

 prints

 foo.E foo.d(20): no message
 [...]


---
class E : Exception
{
     this(int i, string file = __FILE__, size_t line = __LINE__)
     {
         super("no message", file, line);
         _i = i;
     }

     override void toString(scope void delegate(in char[]) sink) 
const
     {
         import std.format;
         sink(format("The value was %s", _i));
     }

     int _i;
}

void main()
{
     throw new E(42);
}
---

prints "The value was 42".
Personally, I use a string literal for msg in the constructor, 
add some value members to the exception, and then override the 
above toString.
May 20 2017
prev sibling parent reply Nick Treleaven <nick geany.org> writes:
On Saturday, 20 May 2017 at 02:05:21 UTC, Jonathan M Davis wrote:

 What we would probably need would be to change msg is a 
 function which generates a message so that derived classes can 
 override that rather than passing a message string.


Further to Moritz's reply showing the existing toString overload 
taking a delegate. This delegate is not  nogc. Otherwise I was 
thinking of doing something like this:

//FIXME: uniqueToString should return  nogc UniquePtr!(const 
char[])
import std.conv;
alias uniqueToString = to!(const char[]);

class MessageEx(E, sinkArgs...) : E
{
     this(A...)(A args)
     {
         super(args);
     }

     //FIXME: delegate not  nogc
     /* nogc*/ override void toString(scope void delegate(in 
char[]) sink) const
     {
         foreach (a; sinkArgs)
             sink(uniqueToString(a));
     }
}

unittest
{
     auto x = 7;
     throw new MessageEx!(Exception, "x = ", x)(null);
}

The result of uniqueToString would free any memory allocated 
after the call to sink.
May 22 2017
parent Jack Stouffer <jack jackstouffer.com> writes:
On Monday, 22 May 2017 at 12:00:30 UTC, Nick Treleaven wrote:

 //FIXME: uniqueToString should return  nogc UniquePtr!(const 
 char[])
 import std.conv;
 alias uniqueToString = to!(const char[]);

 class MessageEx(E, sinkArgs...) : E
 {
     this(A...)(A args)
     {
         super(args);
     }

     //FIXME: delegate not  nogc
     /* nogc*/ override void toString(scope void delegate(in 
 char[]) sink) const
     {
         foreach (a; sinkArgs)
             sink(uniqueToString(a));
     }
 }

 unittest
 {
     auto x = 7;
     throw new MessageEx!(Exception, "x = ", x)(null);
 }

 The result of uniqueToString would free any memory allocated 
 after the call to sink.


Heh, I actually ran into this problem earlier today and just saw 
this post https://issues.dlang.org/show_bug.cgi?id=17420

Weird coincidence.
May 23 2017
prev sibling next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 Destroy!




 In catch blocks, e is regarded as scope so that it cannot 
 escape the catch block.
 ...
 Code that needs to leak the thrown exception object can clone 
 the object.


There's a contradiction here. Generic cloning cannot be 
implemented without storing exceptions for rethrowing later (in 
case member destructors throw).

Furthermore:


 2. Disallowing Exception objects with postblit fields.


What about fields with destructors? I detect a double-free. If we 
need to clone, we need postblits and destructors, or neither.

It looks as if that clause is added specifically to deal with 
cloning. Yet no information on cloning implementation is provided.
May 20 2017
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 Extensive discussion of this DIP has already taken place in two 
 threads, both linked from the document. You may find it 
 beneficial to skim through those threads before posting any 
 feedback here.

 Thanks in advance to all who participate.

 Destroy!


The proposal is a very mechanical fix, throwing several special 
cases at one specific problem.
Why does it have to be refcounted? Seems like there is only ever 
one reference to the current exception (the catch variable).
The only thing that seems necessary is to require scope on catch 
variable declarations, so that people do not escape the class 
reference, then this info might be used by the runtime to free 
exception objects after the catch handler is done.

Amaury put a bit more words into that.
http://forum.dlang.org/post/gtqsojgqqaorubcsneie forum.dlang.org

Has staticError been considered? It has a potential issue with 
multiple nested exceptions, but otherwise works fine.
https://github.com/dlang/druntime/blob/bc832b18430ce1c85bf2dded07bbcfe348ff0813/src/core/exception.d#L683
May 23 2017
next sibling parent MysticZach <reachzach ggmail.com> writes:
On Tuesday, 23 May 2017 at 22:40:43 UTC, Martin Nowak wrote:

 The proposal is a very mechanical fix, throwing several special 
 cases at one specific problem.
 Why does it have to be refcounted? Seems like there is only 
 ever one reference to the current exception (the catch 
 variable).
 The only thing that seems necessary is to require scope on 
 catch variable declarations, so that people do not escape the 
 class reference, then this info might be used by the runtime to 
 free exception objects after the catch handler is done.

 Amaury put a bit more words into that.
 http://forum.dlang.org/post/gtqsojgqqaorubcsneie forum.dlang.org

 Has staticError been considered? It has a potential issue with 
 multiple nested exceptions, but otherwise works fine.
 https://github.com/dlang/druntime/blob/bc832b18430ce1c85bf2dded07bbcfe348ff0813/src/core/exception.d#L683


I'm trying to understand your and Amaury's point. Normally, when 
you say `new` you get memory from the GC allocator. The  nogc 
attribute is supposed to prevent this, if I understand it 
correctly. Are you saying that ` nogc` as such is misconceived, 
because what a good language feature should really be doing is 
identifying and preventing new memory that _can't_ be 
deterministically destroyed? Is the problem here with the  nogc 
attribute? Because I think Walter's goal with this DIP is to make 
it so that you can put  nogc on _called_ functions that throw 
using `new`. Whereas your solution is to ignore that `new 
Exception` allocation, on account of the fact that you can 
deterministically destroy the Exception, provided you use `scope` 
catch blocks at, or above, the call site. Your solution might 
actually have its priorities straight, and ` nogc` may be 
designed badly because it clumps all GC allocations into one big 
basket. However, getting new memory from the GC still could 
trigger a collection cycle, which is what  nogc was created for, 
and simply knowing that you can reliably destroy the allocated 
memory doesn't change that.

Thus, if I understand correctly, you and Amaury are arguing that 
` nogc` as currently designed is a false goal to be chasing, that 
the more important goal is memory that can be deterministically 
destroyed, and therefore it distresses you that the language may 
be altered to chase the false prize of ` nogc` everywhere, 
instead of focusing on a real prize worth attaining?
May 24 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/23/2017 3:40 PM, Martin Nowak wrote:

 Why does it have to be refcounted? Seems like there is only ever one reference 
 to the current exception (the catch variable).


Rethrowing the catch variable makes for 2 references.



 Has staticError been considered? It has a potential issue with multiple nested 
 exceptions, but otherwise works fine.
 https://github.com/dlang/druntime/blob/bc832b18430ce1c85bf2dded07bbcfe348ff0813/src/c
re/exception.d#L683 


Doesn't work for chained exceptions.
May 24 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/25/17 6:24 AM, Walter Bright wrote:

 On 5/23/2017 3:40 PM, Martin Nowak wrote:

 Why does it have to be refcounted? Seems like there is only ever one 
 reference to the current exception (the catch variable).


 
 Rethrowing the catch variable makes for 2 references.


Why doesn't the rethrow count as a move? There is no way it can be 
reused in the scope that rethrows. -- Andrei
May 25 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/25/2017 12:30 AM, Andrei Alexandrescu wrote:

 On 5/25/17 6:24 AM, Walter Bright wrote:

 On 5/23/2017 3:40 PM, Martin Nowak wrote:

 Why does it have to be refcounted? Seems like there is only ever one 
 reference to the current exception (the catch variable).


 Rethrowing the catch variable makes for 2 references.


 
 Why doesn't the rethrow count as a move? There is no way it can be reused in
the 
 scope that rethrows. -- Andrei


It could be - it's just that there's nothing currently in the compiler to 
support the notion of moves. So the destructor will still wind up getting
called 
on it.

It's a good idea for a future enhancement, though.
May 26 2017
parent Martin Nowak <code dawg.eu> writes:
On Friday, 26 May 2017 at 08:45:40 UTC, Walter Bright wrote:

 Why doesn't the rethrow count as a move? There is no way it 
 can be reused in the scope that rethrows. -- Andrei


 It could be - it's just that there's nothing currently in the 
 compiler to support the notion of moves. So the destructor will 
 still wind up getting called on it.

 It's a good idea for a future enhancement, though.


Even worth a hack right now to avoid ref counting for those 
exceptions IMO.
Jun 18 2017
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 DIP 1008 is titled "Exceptions and  nogc".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on June 2 (3:59 AM GMT June 3), or when I make a post declaring 
 it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the formal review and evaluation by the language 
 authors.

 Extensive discussion of this DIP has already taken place in two 
 threads, both linked from the document. You may find it 
 beneficial to skim through those threads before posting any 
 feedback here.

 Thanks in advance to all who participate.

 Destroy!


As many others, I dislike special-casing "throw new" and enough 
has been said about that.

I also don't like adding hooks to druntime, or that by doing so 
the allocation strategy is baked in (but I guess only when using 
the `new` operator so there's that).

I think maybe the problem isn't with `throw` but with `catch`. 
What if instead we make it so that:

catch(scope T ex) { /*...*/; }

Means:

catch(scope T ex) { /*...*/; ex.__dtor; }


That way whoever is throwing the exception allocates however he 
or she sees fit and the memory is handled by the class's 
destructor. Sort of proof of concept using malloc (if I did this 
for real I'd use std.allocator):

class MallocException: Exception {

     char* buffer;
     size_t length;

     // has to be public or `emplace` won't compile
     this(string msg)  nogc {
         import core.stdc.stdlib: malloc;
         length = msg.length;
         buffer = cast(char*)malloc(length);
         buffer[0 .. length] = msg[];
         super(cast(string)buffer[0 .. length]);
     }

     ~this()  nogc {
         import core.stdc.stdlib: free;
         free(buffer);
         free(cast(void*)this);
     }

     static MallocException create(string msg)  nogc {
         import core.stdc.stdlib: malloc;
         import std.conv: emplace;
         enum size = __traits(classInstanceSize, MallocException);
         auto buf = malloc(size);
         return emplace!MallocException(buf[0 .. size], msg);
     }
}

void main()  nogc {
     try {
         import core.stdc.stdlib: malloc;
         throw MallocException.create("oops");
     } catch(MallocException ex) {
         ex.__dtor;
     }
}

The code above works today, the annoying thing is having to 
invoke the destructor manually (and also knowing it's called 
`__dtor`).

This would however mean delaying making phobos  nogc since each 
`throw` site would have to be changed.

And if someone never remembers to catch by scope somewhere and is 
allocating memory themselves, well... use the GC, Luke.

Atila
May 25 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/25/2017 4:54 PM, Atila Neves wrote:

 I think maybe the problem isn't with `throw` but with `catch`. What if instead 
 we make it so that:
 
 catch(scope T ex) { /*...*/; }
 
 Means:
 
 catch(scope T ex) { /*...*/; ex.__dtor; }


The trouble comes in when one starts copying exception references around. Who 
then is responsible for destroying it?
May 26 2017
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/26/17 4:49 AM, Walter Bright wrote:

 On 5/25/2017 4:54 PM, Atila Neves wrote:

 I think maybe the problem isn't with `throw` but with `catch`. What if
 instead we make it so that:

 catch(scope T ex) { /*...*/; }

 Means:

 catch(scope T ex) { /*...*/; ex.__dtor; }


 The trouble comes in when one starts copying exception references
 around. Who then is responsible for destroying it?


This isn't the trouble. The trouble is that `new Exception` is not 
 nogc, and there isn't a way to fix all existing exception code easily.

But to answer your question, don't mark the exception scope in that 
case. The compiler should complain if you copy it outside the scope, no?

My $0.02: Either we are going to make `new Exception` be  nogc, or we 
are going to require people to type something different. IMO, I feel if 
we can create a library solution, and use dfix or similar tool to allow 
people to update their code, then we are in a much better place.

I need to review the DIP again, but last time I read it, there were some 
issues I saw. I will post those in a while.

-Steve
May 26 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/26/2017 4:50 AM, Steven Schveighoffer wrote:

 But to answer your question, don't mark the exception scope in that case. The 
 compiler should complain if you copy it outside the scope, no?


I suspect if you proceed with that line of reasoning, you'll wind up in the
same 
place I did with DIP 1008.
May 26 2017
parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 26 May 2017 at 16:36:16 UTC, Walter Bright wrote:

 On 5/26/2017 4:50 AM, Steven Schveighoffer wrote:

 But to answer your question, don't mark the exception scope in 
 that case. The compiler should complain if you copy it outside 
 the scope, no?


 I suspect if you proceed with that line of reasoning, you'll 
 wind up in the same place I did with DIP 1008.


Could you explain the line of reasoning that led you do dip1008? 
Thanks,

Atila
May 26 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/26/2017 11:58 AM, Atila Neves wrote:

 On Friday, 26 May 2017 at 16:36:16 UTC, Walter Bright wrote:

 On 5/26/2017 4:50 AM, Steven Schveighoffer wrote:

 But to answer your question, don't mark the exception scope in that case. The 
 compiler should complain if you copy it outside the scope, no?


 I suspect if you proceed with that line of reasoning, you'll wind up in the 
 same place I did with DIP 1008.


 
 Could you explain the line of reasoning that led you do dip1008? Thanks,


Follow every construct that enables a reference to an Exception to escape and 
figure out what you're going to do about it. After all, the call to free() must 
happen exactly once per Exception that was allocated with malloc(), and one
must 
also deal with GC allocated Exceptions in the mix. A typical omission is not 
considering with the rethrow case nor the chaining case.

Pragmatically speaking:

Counter-proposals that don't do this are legion and I've critiqued a lot of
them 
for missing crucial cases. I'm getting more reluctant to continue doing that,
so 
I ask submitters of them to be both much more complete, and if they are
complete 
solutions, provide a rationale as to why they are better than DIP 1008 rather 
than being arguably equivalent.

Also keep in mind that DIP 1008 has an implementation sitting in the PR queue, 
and it's fairly simple. Solutions that require redesigning D or the compiler or 
both are not likely to get much traction. The list of things I have to do next 
reminds me of trying to run up the down escalator while it constantly increases 
speed.
May 26 2017
parent Atila Neves <atila.neves gmail.com> writes:
On Saturday, 27 May 2017 at 01:27:24 UTC, Walter Bright wrote:

 On 5/26/2017 11:58 AM, Atila Neves wrote:

 On Friday, 26 May 2017 at 16:36:16 UTC, Walter Bright wrote:

 On 5/26/2017 4:50 AM, Steven Schveighoffer wrote:

 But to answer your question, don't mark the exception scope 
 in that case. The compiler should complain if you copy it 
 outside the scope, no?


 I suspect if you proceed with that line of reasoning, you'll 
 wind up in the same place I did with DIP 1008.


 
 Could you explain the line of reasoning that led you do 
 dip1008? Thanks,


 Follow every construct that enables a reference to an Exception 
 to escape and figure out what you're going to do about it.


The idea would be that this only happens with `catch(scope T 
ex)`, so that the exception _can't_ escape. It can't be stored 
somewhere else, it can't be rethrown. It's going away so it's ok 
to call the destructor on it.


 After all, the call to free() must happen exactly once per 
 Exception that was allocated with malloc(), and one must also 
 deal with GC allocated Exceptions in the mix.


GC exceptions would just not do any work in the destructor.


 A typical omission is not considering with the rethrow case nor 
 the chaining case.


I omitted the chaining case in my example for brevity. A 
production-grade implementation would call the destructors of the 
exceptions in the chain. Sorry for not making that clear.

My idea is that every exception knows how to get rid of itself if 
by chance it's going away soon. And that if anyone wants to store 
a ` nogc` exception then that exception instance provides a 
`.dup` function.

I got to this idea by considering what it would like if we could 
catch smart pointers:

catch(RefCounted!Exception ex)

Then I made it simpler. TBH I'd rather be able to catch a smart 
pointer.


 Counter-proposals that don't do this are legion and I've 
 critiqued a lot of them for missing crucial cases. I'm getting 
 more reluctant to continue doing that, so I ask submitters of 
 them to be both much more complete, and if they are complete 
 solutions, provide a rationale as to why they are better than 
 DIP 1008 rather than being arguably equivalent.



I understand that, I'd have the same attitude towards other 
proposals if I were you. I know and confess I tend to be overly 
optimistic and fail to consider everything that can go wrong. The 
devil is indeed in the details.

Why do I think this scheme is better?

1. The programmer controls how allocation is done
2. Less of a special case (I admit it's a bit "special-casey")
3. Doesn't require another 2 druntime functions
4. Arguably easier to explain how it works


 Also keep in mind that DIP 1008 has an implementation sitting 
 in the PR queue, and it's fairly simple. Solutions that require 
 redesigning D or the compiler or both are not likely to get 
 much traction. The list of things I have to do next reminds me 
 of trying to run up the down escalator while it constantly 
 increases speed.


I could be completely wrong, but my intuition says that what I 
proposed wouldn't be too hard to implement either.

Atila
May 26 2017
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 26 May 2017 at 11:50:40 UTC, Steven Schveighoffer 
wrote:

 On 5/26/17 4:49 AM, Walter Bright wrote:

 On 5/25/2017 4:54 PM, Atila Neves wrote:

 I think maybe the problem isn't with `throw` but with 
 `catch`. What if
 instead we make it so that:

 catch(scope T ex) { /*...*/; }

 Means:

 catch(scope T ex) { /*...*/; ex.__dtor; }


 The trouble comes in when one starts copying exception 
 references
 around. Who then is responsible for destroying it?


 This isn't the trouble. The trouble is that `new Exception` is 
 not  nogc, and there isn't a way to fix all existing exception 
 code easily.


True, but a hypothetical `NoGcException.create` _is_ ` nogc` 
(same as my MallocException), and is there really any difference 
between reading/writing `new Foo` and `Foo.create`?

Also, there's `enforce`, which is where I suspect a lot of 
throwing happens anyway. It could use DbI to figure out from the 
exception type what to do.


 My $0.02: Either we are going to make `new Exception` be  nogc, 
 or we are going to require people to type something different.


Or be able to customise `new T`, which feels less hacky than it 
magically meaning something different if the type is a Throwable. 
I know that there were class allocators in D1 and that they're 
depecreated but I wasn't around back then to know why.

Atila
May 26 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/26/17 2:58 PM, Atila Neves wrote:

 On Friday, 26 May 2017 at 11:50:40 UTC, Steven Schveighoffer wrote:

 On 5/26/17 4:49 AM, Walter Bright wrote:

 On 5/25/2017 4:54 PM, Atila Neves wrote:

 I think maybe the problem isn't with `throw` but with `catch`. What if
 instead we make it so that:

 catch(scope T ex) { /*...*/; }

 Means:

 catch(scope T ex) { /*...*/; ex.__dtor; }


 The trouble comes in when one starts copying exception references
 around. Who then is responsible for destroying it?


 This isn't the trouble. The trouble is that `new Exception` is not
  nogc, and there isn't a way to fix all existing exception code easily.


 True, but a hypothetical `NoGcException.create` _is_ ` nogc` (same as my
 MallocException), and is there really any difference between
 reading/writing `new Foo` and `Foo.create`?


There isn't, but Foo.create doesn't exist in current code. Someone has 
to go through it all and update.

Note that the implication in your strawman is that you need a special 
exception to be nogc. I'd rather leave the (de)allocation of the 
exception up to the thrower/catcher, and have the exception not care 
about its own location.


 Also, there's `enforce`, which is where I suspect a lot of throwing
 happens anyway. It could use DbI to figure out from the exception type
 what to do.


Good point, we could change enforce, and that would fix a lot of the 
usage of exceptions.


 My $0.02: Either we are going to make `new Exception` be  nogc, or we
 are going to require people to type something different.


 Or be able to customise `new T`, which feels less hacky than it
 magically meaning something different if the type is a Throwable. I know
 that there were class allocators in D1 and that they're depecreated but
 I wasn't around back then to know why.


Deprecated, but still there.

However, it's the auto-destruction that isn't there. There isn't a way 
to tell the compiler to destroy on the catch scope automatically. 
Currently, when you override class allocation, you need to explicitly 
delete.

-Steve
May 26 2017
parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 26 May 2017 at 19:31:26 UTC, Steven Schveighoffer 
wrote:

 Note that the implication in your strawman is that you need a 
 special exception to be nogc. I'd rather leave the 
 (de)allocation of the exception up to the thrower/catcher, and 
 have the exception not care about its own location.


That the thrower should decide how to allocate, I agree with. Or 
at least have a sane default that can be optionally changed. But 
deallocation should just automagically work, otherwise people 
will forget to do it and end up with leaks at the very least.

Atila
May 26 2017
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Friday, 26 May 2017 at 08:49:42 UTC, Walter Bright wrote:

 On 5/25/2017 4:54 PM, Atila Neves wrote:

 I think maybe the problem isn't with `throw` but with `catch`. 
 What if instead we make it so that:
 
 catch(scope T ex) { /*...*/; }
 
 Means:
 
 catch(scope T ex) { /*...*/; ex.__dtor; }


 The trouble comes in when one starts copying exception 
 references around. Who then is responsible for destroying it?


Since it's `scope`, where would it be copied to? This is assuming 
dip1000, of course.

Atila
May 26 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/26/2017 11:51 AM, Atila Neves wrote:

 Since it's `scope`, where would it be copied to? This is assuming dip1000, of 
 course.


The rethrow case must be allowed:

     throw ex;
May 26 2017
parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 27 May 2017 at 02:40:47 UTC, Walter Bright wrote:

 On 5/26/2017 11:51 AM, Atila Neves wrote:

 Since it's `scope`, where would it be copied to? This is 
 assuming dip1000, of course.


 The rethrow case must be allowed:

     throw ex;


Then either:

1. Elide the destructor call in that situation
2. Make the developer write `throw ex.dup`


`throw` escapes the ex IMHO.

My $0.02

Atila
May 26 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/26/2017 11:50 PM, Atila Neves wrote:

 On Saturday, 27 May 2017 at 02:40:47 UTC, Walter Bright wrote:

 On 5/26/2017 11:51 AM, Atila Neves wrote:

 Since it's `scope`, where would it be copied to? This is assuming dip1000, of 
 course.


 The rethrow case must be allowed:

     throw ex;


 
 Then either:
 
 1. Elide the destructor call in that situation


That's Andrei's "move semantics" idea, and it involves major code effort in the 
compiler.

There's the "chain" case, too.


 2. Make the developer write `throw ex.dup`


The point is to not require them to rewrite their code.




escapes 
 the ex IMHO.
 
 My $0.02
 
 Atila
May 27 2017
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/19/17 11:45 AM, Mike Parker wrote:

 DIP 1008 is titled "Exceptions and  nogc".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

 All review-related feedback on and discussion of the DIP should occur in
 this thread. The review period will end at 11:59 PM ET on June 2 (3:59
 AM GMT June 3), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the formal review and evaluation by the language authors.

 Extensive discussion of this DIP has already taken place in two threads,
 both linked from the document. You may find it beneficial to skim
 through those threads before posting any feedback here.

 Thanks in advance to all who participate.

 Destroy!


Some comments:

"The destructor for Throwable will, if the refcount is 1, call 
_d_delThrowable(e.next), i.e. on the head of the chained list of 
exceptions."

How does this work if the throwable is being destroyed by the GC? In 
this case, e.next may be a dangling pointer.

"it will call the new function _d_newThrowable() which will allocate E 
and intialize it for refcounting."

Where is it allocated? If on the GC heap, how can it be  nogc? If not, 
then how does it interact with GC pointers? e.g. the e.next pointer may 
point to a GC-allocated exception, how to stop the GC from collecting it 
early?

Part of this may mean clarifying what  nogc actually means. Does it mean 
no interaction with the GC system, or does it mean "cannot run a 
collection cycle"?

"In catch blocks, e is regarded as scope so that it cannot escape the 
catch block."

If e is scope, then the destructor is called. However, since e is a 
reference type, the data it points at isn't really stack allocated, and 
the scope-ness ends at the reference, no? So what if you have something 
like this?

void *foo;

try
{
   func();
}
catch(MyException e)
{
    foo = e.voidptr; // get pointer to some member
}

Should foo be allowed to capture pieces of e? How does the compiler stop 
that if not? What expectations can e assume in terms of its members that 
are references? Can it assume that it is in charge of the management of 
that memory if it's ref counted, or does it need to assume GC usage for 
those items?

-Steve
May 26 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/26/2017 2:31 PM, Steven Schveighoffer wrote:

 "The destructor for Throwable will, if the refcount is 1, call 
 _d_delThrowable(e.next), i.e. on the head of the chained list of exceptions."
 
 How does this work if the throwable is being destroyed by the GC? In this
case, 
 e.next may be a dangling pointer.


The GC calls a finalizer which calls _d_delThrowable on the e.next value.


 "it will call the new function _d_newThrowable() which will allocate E and 
 intialize it for refcounting."
 
 Where is it allocated?


My implementation uses malloc().


 If on the GC heap,


It isn't.


 how can it be  nogc?


And that's how it can be  nogc.


 If not, then how does it interact with GC pointers?


The refcount of 0 means it is GC allocated.


 e.g. the e.next pointer may point to a 
 GC-allocated exception, how to stop the GC from collecting it early?


The usual way - register the roots with the GC. Take a look at the PRs for
this, 
they do that.


 Part of this may mean clarifying what  nogc actually means. Does it mean no 
 interaction with the GC system, or does it mean "cannot run a collection
cycle"?


 nogc means no GC allocations occur in the  nogc code, the same as usual, no 
more, no less.


 "In catch blocks, e is regarded as scope so that it cannot escape the catch
block."
 
 If e is scope, then the destructor is called. However, since e is a reference 
 type, the data it points at isn't really stack allocated, and the scope-ness 
 ends at the reference, no? So what if you have something like this?
 
 void *foo;
 
 try
 {
    func();
 }
 catch(MyException e)
 {
     foo = e.voidptr; // get pointer to some member
 }
 
 Should foo be allowed to capture pieces of e? How does the compiler stop that
if 
 not?


That's what DIP 1000 addresses. DIP 1008 relies on DIP 1000.


 What expectations can e assume in terms of its members that are references? 
 Can it assume that it is in charge of the management of that memory if it's
ref 
 counted, or does it need to assume GC usage for those items?


See DIP 1000.
May 26 2017
next sibling parent via Digitalmars-d <digitalmars-d puremagic.com> writes:
what do you mean by this:

"Look into http: vs https: link handling. Some errors with https"

are you trying to publish https links or go to https://ink.vu links?
May 26 2017
prev sibling parent via Digitalmars-d <digitalmars-d puremagic.com> writes:
sorry i replied to the wrong email in my inbox and didn't notice
until it was already sent

plz ignore me
May 26 2017
prev sibling parent Nick Treleaven <nick geany.org> writes:
On Friday, 26 May 2017 at 21:31:20 UTC, Steven Schveighoffer 
wrote:

 Part of this may mean clarifying what  nogc actually means. 
 Does it mean no interaction with the GC system, or does it mean 
 "cannot run a collection cycle"?


I was pleased to find GC.addRange is now  nogc, so it seems 
potential interaction with the GC is OK so long as allocation and 
collection don't happen. Non- nogc addRange was quite a blocker 
for smart pointer implementation.

This does seem compatible with Walter's idea of not requiring 
linking of the GC runtime, at least in that the GC functions 
(such as addRange) can be stubbed out.
Jun 11 2017
prev sibling parent MysticZach <reachzach ggmail.com> writes:
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:

 DIP 1008 is titled "Exceptions and  nogc".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md


I would like the DIP to fully articulate the choice that it's 
facing, that special-casing the language for `throw new` comes 
with some downsides, but that the alternative is to force 
programmers to rewrite all their exceptions if they want to mark 
their functions ` nogc`. This requires an analysis of the 
similarities and differences between `throw new` and other uses 
of `new`. My personal impression is that exceptions do indeed 
fall into a distinct category, where they are easier to contain 
and shorter-lived than ordinary `new` memory. But the downside is 
that the language loses a little elegance by having another 
special case, and there might be corner cases where the special 
case is undesirable. I can only assume that at minimum, should 
this DIP be _rejected_, it should be replaced by some official 
documentation and methods on how to properly throw exceptions in 
` nogc` code.
Jun 05 2017