www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Blog: GC

reply NonNull <non-null use.startmail.com> writes:
Regarding the depressing GC blog article, dated 2021

https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/

At the foot it says
Content Copyright © 2016-2018 by the D Language Foundation, All 
Rights Reserved

Easy to guess the current content about the GC. If a person knows 
anything about the situation and has a small amount of 
imagination, all of this is totally obvious. What a mess!

Do we really have to reverse engineer the implementation to get 
here? And a very incomplete reverse engineering indeed. The 
blog's conclusion: The situation is bad, therefore avoid 
interacting with it --- this is my first order approximation to 
the article.

So about D itself. No design here then, no intent, no 
anticipation of the consequences of putting the pieces together, 
no strategic reasoning about the consequences of the construction 
of D with a garbage collector. All this long-ago stuff is coming 
home to roost.

The article admits it is incomplete. But what can follow except 
workarounds if someone needs to get past this? And most won't 
work. This is not just a roadblock. It reveals the entropy inside 
D. The core of D is fatally flawed. D is not a language where 
system garbage is taken care of, both concretely and abstractly. 
The idea that we all system-program ourselves around these 
difficulties suggests both labor and depression.

So disappointed. I wanted D to win. But this is where the dirt 
is, and I do not believe a strategic fix will ever be in the 
works. Entropy rules.
Mar 05 2021
next sibling parent mipri <mipri minimaltype.com> writes:
On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:

 Regarding the depressing GC blog article, dated 2021

 https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/


Is this a fair summary of your position?

   You want D to have very simple object lifetime rules or
   maybe deterministic object finalization. You are envious of
   Rust's Drop trait or similar. You dislike the tutorial tone
   of the blog, its "let me help you understand lifetimes in D",
   because it accepts D's current rules as a given when what
   you would prefer to read is a grand proposal to change those
   rules. Reading the blog has helped you to lose hope that D
   will one day change into a language you'd prefer.

Also:

   The copyright notice is old and this shows a lack of care or
   attention to the website.
Mar 05 2021
prev sibling next sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:

 Regarding the depressing GC blog article, dated 2021

 https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/

 At the foot it says
 Content Copyright © 2016-2018 by the D Language Foundation, All 
 Rights Reserved

 Easy to guess the current content about the GC. If a person 
 knows anything about the situation and has a small amount of 
 imagination, all of this is totally obvious. What a mess!

 Do we really have to reverse engineer the implementation to get 
 here? And a very incomplete reverse engineering indeed. The 
 blog's conclusion: The situation is bad, therefore avoid 
 interacting with it --- this is my first order approximation to 
 the article.

 So about D itself. No design here then, no intent, no 
 anticipation of the consequences of putting the pieces 
 together, no strategic reasoning about the consequences of the 
 construction of D with a garbage collector. All this long-ago 
 stuff is coming home to roost.

 The article admits it is incomplete. But what can follow except 
 workarounds if someone needs to get past this? And most won't 
 work. This is not just a roadblock. It reveals the entropy 
 inside D. The core of D is fatally flawed. D is not a language 
 where system garbage is taken care of, both concretely and 
 abstractly. The idea that we all system-program ourselves 
 around these difficulties suggests both labor and depression.

 So disappointed. I wanted D to win. But this is where the dirt 
 is, and I do not believe a strategic fix will ever be in the 
 works. Entropy rules.


I vaguely agree that the Garbage Collector as a global concept is 
dated, however: Be realistic - every programming language has 
guidelines (http://dev.stephendiehl.com/hask/ Even for Haskell 
they can be enormous) and dos and don'ts.

Mike's article could probably use being a little more abstract in 
it's discussion of garbage collection as a concept, but this 
isn't reverse engineering the garbage collector, is it? It's also 
the second of (I think) three articles.

Fundamentally Garbage Collection is an optimization opportunity 
(sometimes at runtime, usually for the developer) is a trick for 
when you don't need determinism. All this article does is make 
clear what you shouldn't be tricked into believing.
Mar 05 2021
next sibling parent claptrap <clap trap.com> writes:
On Saturday, 6 March 2021 at 03:07:32 UTC, Max Haughton wrote:

 On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:


 I vaguely agree that the Garbage Collector as a global concept 
 is dated, however: Be realistic - every programming language 
 has guidelines (http://dev.stephendiehl.com/hask/ Even for 
 Haskell they can be enormous) and dos and don'ts.


"Number of guidelines" is inversely proportional to quality of 
design.



 Mar 06 2021



prev sibling  parent  James Lu <jamtlu gmail.com>  writes:


On Saturday, 6 March 2021 at 03:07:32 UTC, Max Haughton wrote:

 I vaguely agree that the Garbage Collector as a global concept 
 is dated, however: Be realistic - every programming language 
 has guidelines (http://dev.stephendiehl.com/hask/ Even for 
 Haskell they can be enormous) and dos and don'ts.


Modern garbage collection is region-based memory management with 
only two regions: One region where you can allocate for objects 
that will soon die, and one region for old objects, that if you 
leave garbage in, your application will pause because every 
object allocated forces an eventual Sweep phase, forcing the 
runtime to look at every single object, making memory allocation 
an O(N*G) operation, where G is the amount of old space garbage, 
and N is the amount of total objects. That easily becomes O(N^2).

(Yes, I'm aware of ZGC and Shenandoah, but most GCs don't work 
that way.)



 Mar 07 2021



prev sibling next sibling parent reply Rumbu <rumbu rumbu.ro>  writes:


On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:

 Regarding the depressing GC blog article, dated 2021

 https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/

 At the foot it says
 Content Copyright © 2016-2018 by the D Language Foundation, All 
 Rights Reserved

 Easy to guess the current content about the GC. If a person 
 knows anything about the situation and has a small amount of 
 imagination, all of this is totally obvious. What a mess!

 Do we really have to reverse engineer the implementation to get 
 here? And a very incomplete reverse engineering indeed. The 
 blog's conclusion: The situation is bad, therefore avoid 
 interacting with it --- this is my first order approximation to 
 the article.

 So about D itself. No design here then, no intent, no 
 anticipation of the consequences of putting the pieces 
 together, no strategic reasoning about the consequences of the 
 construction of D with a garbage collector. All this long-ago 
 stuff is coming home to roost.

 The article admits it is incomplete. But what can follow except 
 workarounds if someone needs to get past this? And most won't 
 work. This is not just a roadblock. It reveals the entropy 
 inside D. The core of D is fatally flawed. D is not a language 
 where system garbage is taken care of, both concretely and 
 abstractly. The idea that we all system-program ourselves 
 around these difficulties suggests both labor and depression.

 So disappointed. I wanted D to win. But this is where the dirt 
 is, and I do not believe a strategic fix will ever be in the 
 works. Entropy rules.


I took the article with a grain of salt when I saw the author 
claiming that finalisation of classes is a problem in all garbage 
collected languages. It seems that the author failed to update 
himself or - what's worse - wants to justify D failure in having 
deterministic class destruction by pointing to others.


for deterministic destruction.

I bet that the next article will end with the conclusion "use 
structs, classes are bad, you don't really need OOP, it's 
obsolete. But if you insist, here is our half baked library 
solution". I would be happy to be wrong on this.

If we want to be honest here, D failed to update/correct its OOP 
support, trying to please the C++ gang afraid of the garbage 
collector. I remember when D had built-in allocator support and 
delete for deterministic destructors. Now they are deprecated and 
we have a library solution :)



 Mar 06 2021



 next sibling parent reply Adam D. Ruppe <destructionator gmail.com>  writes:


On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 It seems that the author failed to update himself or - what's 
 worse - wants to justify D failure in having deterministic 
 class destruction by pointing to others.


D *does* have deterministic class destruction. `scope Object o = 
new Object;`. Also `scope(exit) .destroy(o);`.

The blog is kinda awkward because it says "pretend class 
destructors don't exist" now, but the next parts are going to 
explain how he's lying to you.

Class destructors do exist in D, just you need to take care to 
use them properly (especially since the old `scope class` was 
deprecated. Note this is distinct from the not-deprecated `scope 
Object` construct.). So the early advice is to pretend they don't 
exist. Then once you level up a little you can learn the truth.



 Mar 06 2021



 next sibling parent  Imperatorn <johan_forsberg_86 hotmail.com>  writes:


On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:

 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 [...]


 D *does* have deterministic class destruction. `scope Object o 
 = new Object;`. Also `scope(exit) .destroy(o);`.

 The blog is kinda awkward because it says "pretend class 
 destructors don't exist" now, but the next parts are going to 
 explain how he's lying to you.

 Class destructors do exist in D, just you need to take care to 
 use them properly (especially since the old `scope class` was 
 deprecated. Note this is distinct from the not-deprecated 
 `scope Object` construct.). So the early advice is to pretend 
 they don't exist. Then once you level up a little you can learn 
 the truth.


Oh, it's like scientology? 😱

Xenu



 Mar 06 2021



prev sibling next sibling parent reply Rumbu <rumbu rumbu.ro>  writes:


On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:

 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 It seems that the author failed to update himself or - what's 
 worse - wants to justify D failure in having deterministic 
 class destruction by pointing to others.


 D *does* have deterministic class destruction. `scope Object o 
 = new Object;`. Also `scope(exit) .destroy(o);`.

 The blog is kinda awkward because it says "pretend class 
 destructors don't exist" now, but the next parts are going to 
 explain how he's lying to you.

 Class destructors do exist in D, just you need to take care to 
 use them properly (especially since the old `scope class` was 
 deprecated. Note this is distinct from the not-deprecated 
 `scope Object` construct.). So the early advice is to pretend 
 they don't exist. Then once you level up a little you can learn 
 the truth.


A library solution is not a feature of the language. Also the 
scope Object declaration is not needed. At least I didn't found 
any reference in the language specification except the one of 
scoped classes, which, of course, will be deprecated.



 Mar 06 2021



 next sibling parent  Adam D. Ruppe <destructionator gmail.com>  writes:


On Saturday, 6 March 2021 at 21:12:51 UTC, Rumbu wrote:

 A library solution is not a feature of the language.


I'm not talking about a library solution.


 Also the scope Object declaration is not needed.


That's how you get scoped, deterministic destruction of class 
objects in the pure language.


 At least I didn't found any reference in the language 
 specification except the one of scoped classes, which, of 
 course, will be deprecated.


That's different. That's what I meant by `scope class`, but 
`scope Object` is referring to it as a storage class

https://dlang.org/deprecate.html#scope%20as%20a%20type%20constraint

"scope as a storage class attributed to variables, function 
parameters, etc. is not deprecated."

But this is why it is complicated: as the class author, you can't 
know how it will be used. The same is also true of structs, which 
the next part of the blog will talk about. So you need to take 
some appropriate care.

It can be done right though.



 Mar 06 2021



prev sibling  parent  Nick Treleaven <nick geany.org>  writes:


On Saturday, 6 March 2021 at 21:12:51 UTC, Rumbu wrote:

 At least I didn't found any reference in the language 
 specification except the one of scoped classes, which, of 
 course, will be deprecated.


"For local declarations, scope implements the RAII (Resource 
Acquisition Is Initialization) protocol. This means that the 
destructor for an object is automatically called when the 
reference to it goes out of scope."

https://dlang.org/spec/attribute.html#scope



 Mar 07 2021



prev sibling next sibling parent  Rumbu <rumbu rumbu.ro>  writes:


On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:

 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:



 D *does* have deterministic class destruction. `scope Object o 
 = new Object;`.


Thank you Adam for opening my eyes, I lived with the impression 
for years that scope initialization for classes is deprecated. In 
this case, the documentation must be clear that only "scope 
classes" are scheduled for deprecation, and this does not include 
"scope initialization".

Sorry, but when the chapter of scope classes starts with the note 
of deprecation, I assume that it is referring the entire chapter:

https://dlang.org/spec/class.html#auto

Yes, if you go further in deprecation details, you will find it 
written as a comment in a code example:

https://dlang.org/deprecate.html#scope%20as%20a%20type%20constraint


Believe it or not, a code comment is not the best place to put 
such important note.



 Mar 06 2021



prev sibling  parent  tsbockman <thomas.bockman gmail.com>  writes:


On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:

 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 It seems that the author failed to update himself or - what's 
 worse - wants to justify D failure in having deterministic 
 class destruction by pointing to others.


 D *does* have deterministic class destruction. `scope Object o 
 = new Object;`. Also `scope(exit) .destroy(o);`.


Be warned that there is currently a bug where even `scope C c = 
new C;` will not actually call the destructor for extern(C++) 
classes:
     https://issues.dlang.org/show_bug.cgi?id=21693

There doesn't seem to be any fundamental reason for this, though; 
it's just an oversight that will hopefully be fixed soon.



 Mar 10 2021



prev sibling next sibling parent  Mike Parker <aldacron gmail.com>  writes:


On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:


 I bet that the next article will end with the conclusion "use 
 structs, classes are bad, you don't really need OOP, it's 
 obsolete. But if you insist, here is our half baked library 
 solution". I would be happy to be wrong on this.


Then you're going to be happy.



 Mar 06 2021



prev sibling next sibling parent reply Dukc <ajieskola gmail.com>  writes:


On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 I took the article with a grain of salt when I saw the author 
 claiming that finalisation of classes is a problem in all 
 garbage collected languages. It seems that the author failed to 
 update himself or - what's worse - wants to justify D failure 
 in having deterministic class destruction by pointing to others.


 mechanisms for deterministic destruction.



to https://dlang.org/phobos/std_typecons.html#scoped. Mike warned 
only about indeterministic destructors -those that are only 
invoked when/if the object gets collected.

To be precise, class destructors are not any more problematic 
than struct desturctors if they are always called 
deterministically -see 
https://dlang.org/phobos/object.html#.destroy. But if the 
destructor needs to be called manually anyway, why should it be a 
destructor at all? It's probably better to put the finalizing in 
a normal function, or better yet to a destructor of a RAII struct.

Problems with GC-called destructors are not unique to D -see 
https://theartofmachinery.com/2018/12/05/gc_not_enough.html for a 
Ruby example.


 I remember when D had built-in allocator support and delete for 
 deterministic destructors. Now they are deprecated and we have 
 a library solution :)


I don't see why library solutions would be inherently inferior. 
What do we lose in case of deterministic destruction?



 Mar 06 2021



  parent reply Rumbu <rumbu rumbu.ro>  writes:


On Saturday, 6 March 2021 at 23:57:25 UTC, Dukc wrote:

 I don't see why library solutions would be inherently inferior. 
 What do we lose in case of deterministic destruction?


First, library solutions show the lack of commitment from 
language creators. There is a big difference between "This is how 
to do X in D" and "This is one idea how to do X in D. Use this 
library. Or maybe not, probably there are other options". The 
latter makes programmers insecure regarding the *correct* 
approach when they want to do something.

Secondly, basic language constructs must be available in the core 
language not in a library, even if it's just druntime in this 
case. Putting them in the library makes the language unusable 
without that library.



 Mar 06 2021



 next sibling parent reply Dukc <ajieskola gmail.com>  writes:


On Sunday, 7 March 2021 at 07:10:30 UTC, Rumbu wrote:

 First, library solutions show the lack of commitment from 
 language creators. There is a big difference between "This is 
 how to do X in D" and "This is one idea how to do X in D. Use 
 this library. Or maybe not, probably there are other options". 
 The latter makes programmers insecure regarding the *correct* 
 approach when they want to do something.

 Secondly, basic language constructs must be available in the 
 core language not in a library, even if it's just druntime in 
 this case. Putting them in the library makes the language 
 unusable without that library.


I'd agree with these if we were talking about a DUB package, but 
the standard library is called standard for a reason. It's one of 
the language maintainers (Atila) that also decides what will get 
to Phobos, so Phobos solutions are not outsourced solutions.

As for the language being unusable (or rather, less usable) 
without Phobos, I highly prefer it against more stuff being in 
the compiler. First off, it's not like you either use all of 
Phobos or no Phobos at all. Even a runtimeless device driver 
could easily use parts std.algorithm and std.range, since they 
are templated and do not depend on being linked to anything.

Second, if something does not work in Phobos, you can just avoid 
using it and work around the problem. Often it's even practical 
to give your workaround the same API as the Phobos equivalent. I 
know because I have worked with compiling to WASM. If that 
non-working feature were in the Druntime, you suddently get 
obscure errors, often from the linker, due to required DRuntime 
symbol missing. And if the compiler had the non-working feature, 
you'd have to patch and recompile the compiler before you can 
even start the real work!



 Mar 07 2021



  parent reply Siemargl <inqnone gmail.com>  writes:


On Sunday, 7 March 2021 at 12:11:26 UTC, Dukc wrote:

 On Sunday, 7 March 2021 at 07:10:30 UTC, Rumbu wrote:


 As for the language being unusable (or rather, less usable) 
 without Phobos, I highly prefer it against more stuff being in 
 the compiler. First off, it's not like you either use all of 
 Phobos or no Phobos at all. Even a runtimeless device driver 
 could easily use parts std.algorithm and std.range, since they 
 are templated and do not depend on being linked to anything.


Is anywhere a list of such features ?
I've seen many talk about them, but no concretic.


 Second, if something does not work in Phobos, you can just 
 avoid using it and work around the problem. Often it's even 
 practical to give your workaround the same API as the Phobos 
 equivalent. I know because I have worked with compiling to 
 WASM. If that non-working feature were in the Druntime, you 
 suddently get obscure errors, often from the linker, due to 
 required DRuntime symbol missing. And if the compiler had the 
 non-working feature, you'd have to patch and recompile the 
 compiler before you can even start the real work!


I think, druntime and phobos were divided a couple of years ago. 
So i can use druntime, but without phobos.

Hmm, translator jokes - fixes druntime as drunktime =)



 Mar 07 2021



  parent  Dukc <ajieskola gmail.com>  writes:


On Sunday, 7 March 2021 at 20:08:20 UTC, Siemargl wrote:

 On Sunday, 7 March 2021 at 12:11:26 UTC, Dukc wrote:

 Even a runtimeless device driver could easily use parts 
 std.algorithm and std.range, since they are templated and do 
 not depend on being linked to anything.


 Is anywhere a list of such features ?
 I've seen many talk about them, but no concretic.


No, just test yourself. In principle everything that does not 
need the GC, io, multitasking or the C standard library should be 
available in everywhere. But since that style of programming has 
less users and less testing, there is often the need to work 
around problems.


 I think, druntime and phobos were divided a couple of years 
 ago. So i can use druntime, but without phobos.


Yes, you can. And it also works vice-versa - you can use many 
parts of Phobos with your own almost-nonexistently small 
Druntime. See https://github.com/skoppe/spasm for example.



 Mar 10 2021



prev sibling  parent reply Jacob Carlborg <doob me.com>  writes:


On Sunday, 7 March 2021 at 07:10:30 UTC, Rumbu wrote:


 Secondly, basic language constructs must be available in the 
 core language not in a library, even if it's just druntime in 
 this case. Putting them in the library makes the language 
 unusable without that library.


FYI, there are a bunch of language features that require library 
support. One of the most obvious ones would be allocating memory 
using `new`.

--
/Jacob Carlborg



 Mar 17 2021



  parent reply rsy <rsy456 hotmail.com>  writes:


This is against that the D GC competes with:

https://malloc.se/blog/zgc-jdk16

Maximum: 1ms pause for the new Java GC (zgc)



 Mar 23 2021



  parent  Paulo Pinto <pjmlp progtools.org>  writes:


On Tuesday, 23 March 2021 at 23:12:22 UTC, rsy wrote:

 This is against that the D GC competes with:

 https://malloc.se/blog/zgc-jdk16

 Maximum: 1ms pause for the new Java GC (zgc)


That is just one of many options on the Java world, there is also 
Azul, while PTC and Aicas have real-time ones, deployed into 
production including military weapons systems in battleships and 
missile radar controls, where a GC pause in the wrong moment is a 
no-go.



 Mar 24 2021



prev sibling  parent  Jesse Phillips <Jesse.K.Phillips+D gmail.com>  writes:


On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 

 mechanisms for deterministic destruction.


https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/implementing-dispose#dispose-and-disposebool

If that is what solved looks like, D fairs pretty good.


every library that supports it and not part of the language.

```
scope(exit) obj.destroy();
```

This provides the same semantics as the using declaration.

https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-8.0/using

Yes, D is different, but we shouldn't pretend other language 
solutions actually solved it.



 Mar 09 2021



prev sibling  parent  Guillaume Piolat <first.name spam.org>  writes:


On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:

 The article admits it is incomplete. But what can follow except 
 workarounds if someone needs to get past this? And most won't 
 work. This is not just a roadblock. It reveals the entropy 
 inside D. The core of D is fatally flawed. D is not a language 
 where system garbage is taken care of, both concretely and 
 abstractly. The idea that we all system-program ourselves 
 around these difficulties suggests both labor and depression.


I have alternative guidelines:

1. For class instances where destruction order is important (they 
transitively hold something that isn't only GC memory), use 
deterministic destruction. Can be with destroy(), scope, but not 
with a release() function.

2. If you fear those destructors to be called by the GC, avoid it 
with https://p0nce.github.io/d-idioms/#GC-proof-resource-class
If you accidentally rely on the GC for destruction, it will warn 
you.

That's it. It's super simple to have everything deterministic 
that way.



 Mar 07 2021