www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Thread-local GC?

reply "logicchains" <jonathan.t.barnard gmail.com> writes:
I was wondering if thread-local GC had been considered an option 
for making D's GC work better in threaded code? Erlang has this 
(well, process-local GC, which is closer to fibre-local in D 
terms) and it seems to work okay, although I don't think Erlang 
allows shared memory between processes.

If this were possible, it would be particularly useful if it 
could be combined with nogc to allow the spawning of nogc 
threads. These could be used for latency-sensitive work, with the 
guarantee that work done in a nogc thread would never be paused 
by garbage collection done in other threads.
Jul 09 2014
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
 Sent: Wednesday, July 09, 2014 at 7:51 PM
 From: "logicchains via Digitalmars-d" <digitalmars-d puremagic.com>
 To: digitalmars-d puremagic.com
 Subject: Thread-local GC?

 I was wondering if thread-local GC had been considered an option
 for making D's GC work better in threaded code? Erlang has this
 (well, process-local GC, which is closer to fibre-local in D
 terms) and it seems to work okay, although I don't think Erlang
 allows shared memory between processes.

 If this were possible, it would be particularly useful if it
 could be combined with nogc to allow the spawning of nogc
 threads. These could be used for latency-sensitive work, with the
 guarantee that work done in a nogc thread would never be paused
 by garbage collection done in other threads.
It's a good idea in principle, but the fact that you can cast to and from shared and immutable throws a major kink in it. Those casts would have to have hooks of some kind to allow for that to work. I don't know how feasible that is, but the fact that you can overload opCast with regards to shared and immutable makes hooks like that rather problematic. So, if we can figure out how to do it, great, but the fact that D's a systems language has a tendancy to make some stuff like that not work as easily as would be nice. - Jonathan M Davis
Jul 09 2014
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 10 July 2014 at 03:21:03 UTC, Jonathan M Davis via 
Digitalmars-d wrote:
...
 So, if we can figure out how to do it, great, but the fact that 
 D's a systems
 language has a tendancy to make some stuff like that not work 
 as easily as
 would be nice.

 - Jonathan M Davis
Sure, but maybe there is too much flexibility that isn't really needed, rather a more GC friendly one, which still allows for typical systems programming use cases. A good approach would be to do a bit of archeology and try to research how GC enabled systems programming languages like Cedar, across the available compilers. And respective OS. To certain extent, they were good enough to produce workable desktop OS. -- Paulo
Jul 10 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 11 July 2014 at 06:44:11 UTC, Paulo Pinto wrote:
 On Thursday, 10 July 2014 at 03:21:03 UTC, Jonathan M Davis via 
 Digitalmars-d wrote:
...
 So, if we can figure out how to do it, great, but the fact 
 that D's a systems
 language has a tendancy to make some stuff like that not work 
 as easily as
 would be nice.

 - Jonathan M Davis
Sure, but maybe there is too much flexibility that isn't really needed, rather a more GC friendly one, which still allows for typical systems programming use cases. A good approach would be to do a bit of archeology and try to research how GC enabled systems programming languages like implemented across the available compilers. And respective OS. To certain extent, they were good enough to produce workable desktop OS. -- Paulo
The action that cause trouble for the GC are already defined as undefined by spec and system. For once, we did things right with the spec.
Jul 11 2014
prev sibling next sibling parent =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
On Thursday, 10 July 2014 at 02:51:05 UTC, logicchains wrote:
 I was wondering if thread-local GC had been considered an 
 option for making D's GC work better in threaded code? Erlang 
 has this (well, process-local GC, which is closer to 
 fibre-local in D terms) and it seems to work okay, although I 
 don't think Erlang allows shared memory between processes.

 If this were possible, it would be particularly useful if it 
 could be combined with nogc to allow the spawning of nogc 
 threads. These could be used for latency-sensitive work, with 
 the guarantee that work done in a nogc thread would never be 
 paused by garbage collection done in other threads.
Very interesting points. There is currently work going on this topic. Search the forums and I believe you'll find it. Otherwise I can see if I find something.
Jul 10 2014
prev sibling next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 10 July 2014 at 02:51:05 UTC, logicchains wrote:
 I was wondering if thread-local GC had been considered an 
 option for making D's GC work better in threaded code? Erlang 
 has this (well, process-local GC, which is closer to 
 fibre-local in D terms) and it seems to work okay, although I 
 don't think Erlang allows shared memory between processes.

 If this were possible, it would be particularly useful if it 
 could be combined with nogc to allow the spawning of nogc 
 threads. These could be used for latency-sensitive work, with 
 the guarantee that work done in a nogc thread would never be 
 paused by garbage collection done in other threads.
I really like this idea. It would neatly address my issues with a long-standing pull request for adding provisions to core.thread for creating threads that should not be blocked by the GC on collections. Just add detection in the Thread ctors for the nogc label and set the appropriate internal flag. It's still kind of a veneer over potentially risky code, since nogc code can manipulate references to data in the GC and so be the sole owner of data that will then vanish on the next collection, but it seems a good way to document what's desired and enforce some degree of correctness.
Jul 10 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 10 July 2014 at 16:20:43 UTC, Sean Kelly wrote:
 On Thursday, 10 July 2014 at 02:51:05 UTC, logicchains wrote:
 I was wondering if thread-local GC had been considered an 
 option for making D's GC work better in threaded code? Erlang 
 has this (well, process-local GC, which is closer to 
 fibre-local in D terms) and it seems to work okay, although I 
 don't think Erlang allows shared memory between processes.

 If this were possible, it would be particularly useful if it 
 could be combined with nogc to allow the spawning of nogc 
 threads. These could be used for latency-sensitive work, with 
 the guarantee that work done in a nogc thread would never be 
 paused by garbage collection done in other threads.
I really like this idea. It would neatly address my issues with a long-standing pull request for adding provisions to core.thread for creating threads that should not be blocked by the GC on collections. Just add detection in the Thread ctors for the nogc label and set the appropriate internal flag. It's still kind of a veneer over potentially risky code, since nogc code can manipulate references to data in the GC and so be the sole owner of data that will then vanish on the next collection, but it seems a good way to document what's desired and enforce some degree of correctness.
https://github.com/D-Programming-Language/druntime/pull/493 Given the changes, it may be worth rejecting the pull request above and creating a new one for this approach, but this is the history anyway.
Jul 10 2014
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 10 July 2014 at 02:51:05 UTC, logicchains wrote:
 I was wondering if thread-local GC had been considered an 
 option for making D's GC work better in threaded code? Erlang 
 has this (well, process-local GC, which is closer to 
 fibre-local in D terms) and it seems to work okay, although I 
 don't think Erlang allows shared memory between processes.

 If this were possible, it would be particularly useful if it 
 could be combined with nogc to allow the spawning of nogc 
 threads. These could be used for latency-sensitive work, with 
 the guarantee that work done in a nogc thread would never be 
 paused by garbage collection done in other threads.
I'm beating that drum for 2 years. SDC do not give me enough bandwidth to handle that one.
Jul 10 2014