www.digitalmars.com         C & C++   DMDScript  

digitalmars.dip.ideas - Alternative garbage collector

reply Matthias =?UTF-8?B?Um/Dn215?= <matti731140 gmx.net> writes:
Hi,

the D garbage collector does a great job for desktop apps, but it 
can leak when processing really big data (>100GB) or in 
long-running processes (>1 month). And here is the reason why: 
The GC scans the memory for the bit patterns of the known 
pointers, and if there is no such pattern, the pointer is not 
used and can be deleted. But there is a tiny risk, that the bit 
pattern of the pointer occurs randomly inside some data. And then 
the pointer will not be deleted although it is not used anymore. 
That is negligible for desktop apps, but the risk increases with 
RAM consumption and program duration. That is not a theoretical 
risk, it really happened to a friend of mine when processing 
100GB of climate simulation data.
I suggest developing an alternative GC which does real refcounting. Then the developer can choose between the current approach (faster) and refcounting (no leaks).
Mar 29
parent reply Nick Treleaven <nick geany.org> writes:
On Sunday, 29 March 2026 at 17:58:31 UTC, Matthias Roßmy wrote:
 Hi,

 the D garbage collector does a great job for desktop apps, but 
 it can leak when processing really big data (>100GB) or in 
 long-running processes (>1 month). And here is the reason why: 
 The GC scans the memory for the bit patterns of the known 
 pointers, and if there is no such pattern, the pointer is not 
 used and can be deleted. But there is a tiny risk, that the bit 
 pattern of the pointer occurs randomly inside some data. And 
 then the pointer will not be deleted although it is not used 
 anymore.
Can you try enabling precise GC? https://dlang.org/spec/garbage.html#gc_config https://dlang.org/spec/garbage.html#precise_gc
 That is negligible for desktop apps, but the risk increases 
 with RAM consumption and program duration. That is not a 
 theoretical risk, it really happened to a friend of mine when 
 processing >100GB of climate simulation data.

 I suggest developing an alternative GC which does real 
 refcounting. Then the developer can choose between the current 
 approach (faster) and refcounting (no leaks).
Refcounting can have cycles which need a GC scan to clean up. Also it is far from straightforward to support memory-safe automatic reference counting anywhere GC is supported in D. I think people have spent a lot of time looking at doing that, including changing the language. AFAIK the only clear success with that is a wrapper type like: https://dlang.org/phobos/std_typecons.html#SafeRefCounted That requires `-dip1000` to be memory-safe.
Mar 29
parent Matthias =?UTF-8?B?Um/Dn215?= <matti731140 gmx.net> writes:
On Sunday, 29 March 2026 at 21:07:41 UTC, Nick Treleaven wrote:
 Can you try enabling precise GC?
 https://dlang.org/spec/garbage.html#gc_config
 https://dlang.org/spec/garbage.html#precise_gc
Thank you, that will solve the problem. However, I think that "precise" should be the default to avoid bad surprises.
Mar 30