www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Rust moving away from GC into reference counting

reply "Paulo Pinto" <pjmlp progtools.org> writes:
Even as GC fanboy, I have to admit that reference counting is in 
trend for system languages.

Rust developers are thinking to move GC support to the language 
library while keeping reference counting as the main way to deal 
with memory management.

http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/

Quite in sync with the latest discussions going on.

--
Paulo
Jun 03 2013
next sibling parent "Dicebot" <m.strashun gmail.com> writes:
Is it an official position or just a blog post / proposal from 
one of developers?

Anyway, given  the recent findings by Adam, D is not _that_ far 
away here. Add configurable global allocators, finally implement 
"scope", move GC to library and reduce runtime a bit - and result 
may be pretty awesome. Hardest part is still "global allocator" 
and assumptions compiler does about runtime.

However, the fact that Rust developers already think about this 
and in D community most real attention to such stuff only came 
together with Manu gives them some advantage.

On Monday, 3 June 2013 at 09:10:14 UTC, Paulo Pinto wrote:
 Even as GC fanboy, I have to admit that reference counting is 
 in trend for system languages.

 Rust developers are thinking to move GC support to the language 
 library while keeping reference counting as the main way to 
 deal with memory management.

 http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/

 Quite in sync with the latest discussions going on.

 --
 Paulo

Jun 03 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
I think we could do the whole owned pointer concept in D too, by 
making a struct with a disabled copy constructor. I don't know 
much about Rust's specifics though so not sure if that's an exact 
match.

But I'm already running with the idea that all slices are 
"borrowed", so we could say the same thing about pointers, if you 
have one, assume it is borrowed and you shouldn't store it nor 
free it.
Jun 03 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 3 June 2013 at 14:04:23 UTC, Adam D. Ruppe wrote:
 I think we could do the whole owned pointer concept in D too, 
 by making a struct with a disabled copy constructor. I don't 
 know much about Rust's specifics though so not sure if that's 
 an exact match.

I don't see how struct with a disabled copy constructor is relevant to owned pointers. Most similar thing D has is "scope" qualifier concept (and that is why I do want it so hard :)) - hard guarantee that no pointer to your data will live longer than data itself. All normal pointers become managed pointers in that sense.
Jun 03 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 3 June 2013 at 14:20:06 UTC, Dicebot wrote:
 I don't see how struct with a disabled copy constructor is 
 relevant to owned pointers.

My thought is then you can control access to the inner pointer better that way. You couldn't even pass this struct to a function without calling some kind of method, which could return a different type to indicate that it is a lent pointer. It also wouldn't need to be reference counted, since the refcount is always going to be 1 because copying it is impossible.
 Most similar thing D has is "scope" qualifier concept (and that 
 is why I do want it so hard :)) - hard guarantee that no 
 pointer to your data will live longer than data itself. All 
 normal pointers become managed pointers in that sense.

Yes, I'm just trying to do what we can with the language today.
Jun 03 2013
prev sibling next sibling parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 3 June 2013 at 14:33:12 UTC, Adam D. Ruppe wrote:
 My thought is then you can control access to the inner pointer 
 better that way. You couldn't even pass this struct to a 
 function without calling some kind of method, which could 
 return a different type to indicate that it is a lent pointer.

Better - sure. But without type system support it will always be inferior. One issues that immediately comes to my mind is that it is yet another case when you either have right qualifier as a default or need to resort to automatic inference (owning pointers/references are useless if majority of Phobos does not accept borrowed ones). Another one is template bloat, of course. If we need to resort to workarounds instead of providing good solid approach for interested users, it just won't work. This will be interesting to hack around as a proof-of-concept experiment but won't suit for a production usage, IMHO.
Jun 03 2013
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
--047d7b676104d1f09a04de421091
Content-Type: text/plain; charset=UTF-8

Haha, wow. Indeed, isn't that well timed with respect to recent
discussions! ;)
Post-dconf, I gave it some serious thought and I'm kinda convincing myself
more and more each day that it's the way to go.

On 3 June 2013 19:10, Paulo Pinto <pjmlp progtools.org> wrote:

 Even as GC fanboy, I have to admit that reference counting is in trend for
 system languages.

 Rust developers are thinking to move GC support to the language library
 while keeping reference counting as the main way to deal with memory
 management.

 http://pcwalton.github.io/**blog/2013/06/02/removing-**
 garbage-collection-from-the-**rust-language/<http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/>

 Quite in sync with the latest discussions going on.

 --
 Paulo

--047d7b676104d1f09a04de421091 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div style>Haha, wow. Indeed, isn&#39;t that well timed wi= th respect to recent discussions! ;)</div><div style>Post-dconf, I gave it = some serious thought and I&#39;m kinda convincing myself more and more each= day that it&#39;s the way to go.</div> <br><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 3 June 2013 19= :10, Paulo Pinto <span dir=3D"ltr">&lt;<a href=3D"mailto:pjmlp progtools.or= g" target=3D"_blank">pjmlp progtools.org</a>&gt;</span> wrote:<br><blockquo= te class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc so= lid;padding-left:1ex"> Even as GC fanboy, I have to admit that reference counting is in trend for = system languages.<br> <br> Rust developers are thinking to move GC support to the language library whi= le keeping reference counting as the main way to deal with memory managemen= t.<br> <br> <a href=3D"http://pcwalton.github.io/blog/2013/06/02/removing-garbage-colle= ction-from-the-rust-language/" target=3D"_blank">http://pcwalton.github.io/= <u></u>blog/2013/06/02/removing-<u></u>garbage-collection-from-the-<u></u>r= ust-language/</a><br> <br> Quite in sync with the latest discussions going on.<br> <br> --<br> Paulo<br> </blockquote></div><br></div></div> --047d7b676104d1f09a04de421091--
Jun 03 2013
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 03.06.2013 18:00, schrieb Manu:
 Haha, wow. Indeed, isn't that well timed with respect to recent
 discussions! ;)
 Post-dconf, I gave it some serious thought and I'm kinda convincing
 myself more and more each day that it's the way to go.

As I mentioned I prefer GC based solutions, but then again I live in the JVM/.NET world, so I don't have the memory/timing pressure you have to deal with. But when looking at systems programming languages that offer reference counting as the main memory management, ATS, Parasail, Objective-C, and now Rust, all of them share a common feature: - Compiler support to remove extra increment/decrement operations. I guess on D's case this support would also be required. -- Paulo
Jun 03 2013
parent evansl <cppljevans suddenlink.net> writes:
On 06/03/13 12:41, Paulo Pinto wrote:
 Am 03.06.2013 18:00, schrieb Manu:
 Haha, wow. Indeed, isn't that well timed with respect to recent
 discussions! ;)
 Post-dconf, I gave it some serious thought and I'm kinda convincing
 myself more and more each day that it's the way to go.

As I mentioned I prefer GC based solutions, but then again I live in the JVM/.NET world, so I don't have the memory/timing pressure you have to deal with. But when looking at systems programming languages that offer reference counting as the main memory management, ATS, Parasail, Objective-C, and now Rust, all of them share a common feature: - Compiler support to remove extra increment/decrement operations. I guess on D's case this support would also be required. -- Paulo

IIUC, a system programming language would use threading at least a fair amount. In that case, I'm not sure reference counting is appropriate, at least according to the following: .> However, I think you are also talking about RC in multi-threaded .> (not distributed) systems. Here, the cost of making RC thread-safe .> (both RC and pointer manipulations must be atomic) can be expensive .> if done naively. It's worth taking a moment to think about .> this. Consider a scenario with 3 objects, A, B and C, and two .> threads, T1 and T2. Suppose that T1 writes a reference to B into a .> slot in A and T2 writes a reference to C into the same slot. (You .> can say that such a program is broken but memory managers but cope .> with broken code). Both the reference count operations and the .> pointer updates must be synchronised so that the reference counts .> do reflect the final state of the reference held in A. This is .> prohibitively expensive if used for general memory management .> (i.e. rather than just for managing a few special objects). This is from a private email(which Jones said I could forward to c.l.c++.m, which, I assume means I can quote here also): .> Date: Fri, 15 Apr 2005 10:50:50 +0100 .> From: Richard Jones <R.E.Jones kent.ac.uk> Of course maybe RC has improved since 2005. I'm no expert :( -regards, Larry
Jun 04 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jun 03, 2013 at 04:33:10PM +0200, Adam D. Ruppe wrote:
 On Monday, 3 June 2013 at 14:20:06 UTC, Dicebot wrote:
I don't see how struct with a disabled copy constructor is
relevant to owned pointers.

My thought is then you can control access to the inner pointer better that way. You couldn't even pass this struct to a function without calling some kind of method, which could return a different type to indicate that it is a lent pointer.

This is an old idea. It's the same as C++'s auto_ptr, and the same as an old idea I independently came up with many years ago. Basically, you have two kinds of pointers: a reference pointer and an owner pointer. An owner pointer can be freely copied into a reference pointer, but never the other way around. An owner pointer is unique, so it has destructive copy semantics (passing it into a function invalidates it in the caller's scope, for example -- if a function doesn't need ownership of the passed object, it should take a reference argument, to which an owner pointer implicitly converts). When it goes out of scope, it's freed. Reference pointers are never freed. Of course, this scheme doesn't correctly deal with dangling reference pointers, but in a manual memory management system, you have to deal with that manually anyway, so it doesn't really matter. Distinguishing between these two kinds of pointers for the most part eliminates a lot of pointer-related bugs. For the most part, this scheme is sufficient to handle a majority of pointer uses. The remaining cases involve multiple references to objects with no clear ownership designation or lifetime. IME, this is relatively confined to specific areas of usage, for which the GC is a better memory management scheme anyway.
 It also wouldn't need to be reference counted, since the refcount is
 always going to be 1 because copying it is impossible.

One should always be aware that there's a rough hierarchy of memory management schemes: - direct malloc/free: maximum control, most error-prone - auto_ptr (the above scheme): less error-prone - reference counting: more convenient - GC: very little control, least error-prone The further down you go, the less you have control over the memory management process, but also the more convenient it is to write code. The further up you go, the more control you have, but also the harder it is to write correct code (and the more error-prone it is). Given D's design of correctness-first, it would seem that GC by default is the correct choice. But it should definitely also allow moving up the hierarchy for applications that require finer control over memory management. Currently D allows this in theory, but in practice, much of Phobos assumes the GC, which greatly reduces its usefulness in such cases.
Most similar thing D has is "scope" qualifier concept (and that is
why I do want it so hard :)) - hard guarantee that no pointer to
your data will live longer than data itself. All normal pointers
become managed pointers in that sense.

Yes, I'm just trying to do what we can with the language today.

auto_ptr can be easily implemented with the language today. It's more a question of Phobos / certain language constructs *using* it. T -- Some ideas are so stupid that only intellectuals could believe them. -- George Orwell
Jun 03 2013
prev sibling next sibling parent "Rob T" <alanb ucora.com> writes:
On Monday, 3 June 2013 at 16:01:29 UTC, H. S. Teoh wrote:

C++11 deprecated auto_ptr in favor of unique_ptr, but it's 
basically the same concept, and it works very well in cases where 
you prefer to manage your own memory. D should have the same 
thing in the std lib, it's not difficult to implement. The ref 
counted pointers are another similar thing, a bit more difficult 
to implement correctly and there's a circular ref issue with them.

The "hierarchy of memory management schemes" is something we 
should embrace instead of shun in favor of the GC. I dislike 
being forced to use the GC, or having to jump through hoops to 
avoid it, and it's insane to have no control over it even when I 
really do want to make use out of it.

The GC can of course be made a lot better, and at least some 
important manual  control can be given to the programmer, for 
example we previously discussed ideas like specifying a maximum 
time limit for each GC run, and also specifying when the GC gets 
called. Currently we have virtually zero contol over the GC 
(other than enable and disable which is far too trivial) but I 
see no reason at all why this must be the case. Allowing some 
significant control over the GC should be independent of the GC 
implementation, so having a better GC design should in no way 
reduce or remove the requirement for having control.

Also a better GC in no way invalidates the need for other memory 
management schemes because there will always be situations where 
a GC is not an appropriate solution, at least not until someone 
invents the perfect one size fits all GC.

--rt
Jun 03 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 3 June 2013 at 16:47:35 UTC, Rob T wrote:
 On Monday, 3 June 2013 at 16:01:29 UTC, H. S. Teoh wrote:

 C++11 deprecated auto_ptr in favor of unique_ptr, but it's 
 basically the same concept, and it works very well in cases 
 where you prefer to manage your own memory. D should have the 
 same thing in the std lib, it's not difficult to implement. The 
 ref counted pointers are another similar thing, a bit more 
 difficult to implement correctly and there's a circular ref 
 issue with them.

There is this: http://dlang.org/phobos/std_typecons.html#.Unique I can't comment how good it is though.
Jun 03 2013
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
On Monday, 3 June 2013 at 16:47:35 UTC, Rob T wrote:
 Also a better GC in no way invalidates the need for other 
 memory management schemes because there will always be 
 situations where a GC is not an appropriate solution, at least 
 not until someone invents the perfect one size fits all GC.

phobos has Unique http://dlang.org/phobos/std_typecons.html#.Unique and RefCounted http://dlang.org/phobos/std_typecons.html#.RefCounted
Jun 03 2013
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 3 June 2013 at 18:10:50 UTC, Kagamin wrote:
 On Monday, 3 June 2013 at 16:47:35 UTC, Rob T wrote:
 Also a better GC in no way invalidates the need for other 
 memory management schemes because there will always be 
 situations where a GC is not an appropriate solution, at least 
 not until someone invents the perfect one size fits all GC.

phobos has Unique http://dlang.org/phobos/std_typecons.html#.Unique and RefCounted http://dlang.org/phobos/std_typecons.html#.RefCounted

RefCounted notably does not work with classes yet (just because no one has taken the time to add support).
Jun 03 2013
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e013c709addd8bd04de492b6a
Content-Type: text/plain; charset=UTF-8

On 4 June 2013 03:41, Paulo Pinto <pjmlp progtools.org> wrote:

 Am 03.06.2013 18:00, schrieb Manu:

  Haha, wow. Indeed, isn't that well timed with respect to recent
 discussions! ;)
 Post-dconf, I gave it some serious thought and I'm kinda convincing
 myself more and more each day that it's the way to go.

JVM/.NET world, so I don't have the memory/timing pressure you have to deal with.

I don't expect the GC would go anywhere. It would remain in use in typical computing environments. It would just be an option for realtime users/embedded platforms. But when looking at systems programming languages that offer reference
 counting as the main memory management, ATS, Parasail, Objective-C, and now
 Rust, all of them share a common feature:

 - Compiler support to remove extra increment/decrement operations.

 I guess on D's case this support would also be required.

Yeah, indeed. I have no idea how much work that would be. I'm not holding my breath that it'll appear any time soon :) --089e013c709addd8bd04de492b6a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">On 4 June 2013 03:41, Paulo Pinto <span dir=3D"ltr">&lt;<a= href=3D"mailto:pjmlp progtools.org" target=3D"_blank">pjmlp progtools.org<= /a>&gt;</span> wrote:<br><div class=3D"gmail_extra"><div class=3D"gmail_quo= te"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-lef= t:1px #ccc solid;padding-left:1ex"> Am 03.06.2013 18:00, schrieb Manu:<div class=3D"im"><br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Haha, wow. Indeed, isn&#39;t that well timed with respect to recent<br> discussions! ;)<br> Post-dconf, I gave it some serious thought and I&#39;m kinda convincing<br> myself more and more each day that it&#39;s the way to go.<br> <br> </blockquote> <br></div> As I mentioned I prefer GC based solutions, but then again I live in the<br=

deal with.<br></blockquote><div><br></div><div style>I don&#39;t expect the= GC would go anywhere. It would remain in use in typical computing environm= ents.</div> <div style>It would just be an option for realtime users/embedded platforms= .</div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0= 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> But when looking at systems programming languages that offer reference coun= ting as the main memory management, ATS, Parasail, Objective-C, and now Rus= t, all of them share a common feature:<br> <br> - Compiler support to remove extra increment/decrement operations.<br> <br> I guess on D&#39;s case this support would also be required.<br></blockquot= e><div><br></div><div style>Yeah, indeed. I have no idea how much work that= would be. I&#39;m not holding my breath that it&#39;ll appear any time soo= n :)</div> </div></div></div> --089e013c709addd8bd04de492b6a--
Jun 03 2013
prev sibling next sibling parent "Jonathan A Dunlap" <jdunlap outlook.com> writes:
Interesting read on the subject of ARC and GC: 
http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

It does seem that ARC would be a preferred strategy for D to 
pursue (even if it's a first pass before resorting to GC sweeps).
Jul 10 2013
prev sibling parent "Jonathan A Dunlap" <jdunlap outlook.com> writes:
Opps, just saw that this link was already posted here:
http://forum.dlang.org/thread/krhjq8$2r8l$1 digitalmars.com
Jul 10 2013