www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Some simple ideas about GC

reply Martin Nowak <code dawg.eu> writes:
I'd like to share some thoughts on improving D's GC, nothing radically 
different though.

A few observations

- Pause times seem to be a much bigger problem than CPU usage or memory 
bandwith. Focus on reducing the pause times.

- The GC code is already fairly optimized, so there is a very low 
profitability in small-scale code optimizations. Improve the algorithms 
not the code.

- The current GC code is not hackable. First rewrite then improve.

and corresponding ideas.

- Marking could be parallelized, sweeping should be done in the 
background, GC could serve allocations during sweep from a separate pool 
(e.g. thread-local).

- The current GC does a lot of bookkeeping work due to how the pools are 
organized (heterogeneous bin sizes). I suspect (but don't know) that 
there are big gains in organizing this differently.

- A testable and more modular rewrite (using recent D practices) would 
encourage more contribution and is necessary for experimentation.


-Martin
May 12 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/12/14, 4:09 PM, Martin Nowak wrote:
 I'd like to share some thoughts on improving D's GC, nothing radically
 different though.

 A few observations

I'll keep those with which std.allocator is likely to help:
 - The current GC code is not hackable. First rewrite then improve.

 - A testable and more modular rewrite (using recent D practices) would
 encourage more contribution and is necessary for experimentation.

I think std.allocator is some 15 work-hours from reviewable form, and std.typed_allocator (with tracing and all) some 50 more work-hours. Unfortunately these numbers grow due to fragmentation - and OMG I made a pun too. Andrei
May 12 2014
next sibling parent "Martin Nowak" <code dawg.eu> writes:
On Monday, 12 May 2014 at 23:44:09 UTC, Andrei Alexandrescu wrote:
 I'll keep those with which std.allocator is likely to help:

 - The current GC code is not hackable. First rewrite then 
 improve.

 - A testable and more modular rewrite (using recent D 
 practices) would
 encourage more contribution and is necessary for 
 experimentation.

I think std.allocator is some 15 work-hours from reviewable form, and std.typed_allocator (with tracing and all) some 50 more work-hours. Unfortunately these numbers grow due to fragmentation - and OMG I made a pun too.

There are already some existing allocators, e.g. vibe.d. If you make it possible to try out the allocator, report bugs and contribute fixes, this should help to polish the implementation. You could do this by moving your work to a separate repo and registering a dub package, instead of using a phobos branch. I haven't yet looked at typed_allocator, but the heap layers concept is just about right for a GC rewrite. Maybe we'll use multiple specialized GCs in the future, instead of one generic GC.
May 12 2014
prev sibling parent "w0rp" <devw0rp gmail.com> writes:
On Tuesday, 13 May 2014 at 06:20:36 UTC, Martin Nowak wrote:
 On Monday, 12 May 2014 at 23:44:09 UTC, Andrei Alexandrescu 
 wrote:
 I'll keep those with which std.allocator is likely to help:

 - The current GC code is not hackable. First rewrite then 
 improve.

 - A testable and more modular rewrite (using recent D 
 practices) would
 encourage more contribution and is necessary for 
 experimentation.

I think std.allocator is some 15 work-hours from reviewable form, and std.typed_allocator (with tracing and all) some 50 more work-hours. Unfortunately these numbers grow due to fragmentation - and OMG I made a pun too.

There are already some existing allocators, e.g. vibe.d. If you make it possible to try out the allocator, report bugs and contribute fixes, this should help to polish the implementation. You could do this by moving your work to a separate repo and registering a dub package, instead of using a phobos branch. I haven't yet looked at typed_allocator, but the heap layers concept is just about right for a GC rewrite. Maybe we'll use multiple specialized GCs in the future, instead of one generic GC.

I think this could be a good approach. This approach was taken for Java. It might be the case that one collector happens to work very well in environments with lots of memory to spend, and another works well where memory is limited, etc.
May 13 2014
prev sibling parent "safety0ff" <safety0ff.dev gmail.com> writes:
On Monday, 12 May 2014 at 23:09:07 UTC, Martin Nowak wrote:
 - The current GC code is not hackable. First rewrite then 
 improve.

I believe this might be one of the bigger factors for why we still (3 years later) do not have a GC that allows allocation during finalization. Allocation during finalization isn't an over ambitious project, but between the uglyness of the GC code and the low demand for this feature it is unlikely that somebody with tackle it.
May 12 2014