www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Blog: GC

reply NonNull <non-null use.startmail.com> writes:
Regarding the depressing GC blog article, dated 2021

https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/

At the foot it says
Content Copyright © 2016-2018 by the D Language Foundation, All 
Rights Reserved

Easy to guess the current content about the GC. If a person knows 
anything about the situation and has a small amount of 
imagination, all of this is totally obvious. What a mess!

Do we really have to reverse engineer the implementation to get 
here? And a very incomplete reverse engineering indeed. The 
blog's conclusion: The situation is bad, therefore avoid 
interacting with it --- this is my first order approximation to 
the article.

So about D itself. No design here then, no intent, no 
anticipation of the consequences of putting the pieces together, 
no strategic reasoning about the consequences of the construction 
of D with a garbage collector. All this long-ago stuff is coming 
home to roost.

The article admits it is incomplete. But what can follow except 
workarounds if someone needs to get past this? And most won't 
work. This is not just a roadblock. It reveals the entropy inside 
D. The core of D is fatally flawed. D is not a language where 
system garbage is taken care of, both concretely and abstractly. 
The idea that we all system-program ourselves around these 
difficulties suggests both labor and depression.

So disappointed. I wanted D to win. But this is where the dirt 
is, and I do not believe a strategic fix will ever be in the 
works. Entropy rules.
Mar 05
next sibling parent mipri <mipri minimaltype.com> writes:
On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:
 Regarding the depressing GC blog article, dated 2021

 https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/
Is this a fair summary of your position? You want D to have very simple object lifetime rules or maybe deterministic object finalization. You are envious of Rust's Drop trait or similar. You dislike the tutorial tone of the blog, its "let me help you understand lifetimes in D", because it accepts D's current rules as a given when what you would prefer to read is a grand proposal to change those rules. Reading the blog has helped you to lose hope that D will one day change into a language you'd prefer. Also: The copyright notice is old and this shows a lack of care or attention to the website.
Mar 05
prev sibling next sibling parent reply Max Haughton <maxhaton gmail.com> writes:
On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:
 Regarding the depressing GC blog article, dated 2021

 https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/

 At the foot it says
 Content Copyright © 2016-2018 by the D Language Foundation, All 
 Rights Reserved

 Easy to guess the current content about the GC. If a person 
 knows anything about the situation and has a small amount of 
 imagination, all of this is totally obvious. What a mess!

 Do we really have to reverse engineer the implementation to get 
 here? And a very incomplete reverse engineering indeed. The 
 blog's conclusion: The situation is bad, therefore avoid 
 interacting with it --- this is my first order approximation to 
 the article.

 So about D itself. No design here then, no intent, no 
 anticipation of the consequences of putting the pieces 
 together, no strategic reasoning about the consequences of the 
 construction of D with a garbage collector. All this long-ago 
 stuff is coming home to roost.

 The article admits it is incomplete. But what can follow except 
 workarounds if someone needs to get past this? And most won't 
 work. This is not just a roadblock. It reveals the entropy 
 inside D. The core of D is fatally flawed. D is not a language 
 where system garbage is taken care of, both concretely and 
 abstractly. The idea that we all system-program ourselves 
 around these difficulties suggests both labor and depression.

 So disappointed. I wanted D to win. But this is where the dirt 
 is, and I do not believe a strategic fix will ever be in the 
 works. Entropy rules.
I vaguely agree that the Garbage Collector as a global concept is dated, however: Be realistic - every programming language has guidelines (http://dev.stephendiehl.com/hask/ Even for Haskell they can be enormous) and dos and don'ts. Mike's article could probably use being a little more abstract in it's discussion of garbage collection as a concept, but this isn't reverse engineering the garbage collector, is it? It's also the second of (I think) three articles. Fundamentally Garbage Collection is an optimization opportunity (sometimes at runtime, usually for the developer) is a trick for when you don't need determinism. All this article does is make clear what you shouldn't be tricked into believing.
Mar 05
next sibling parent claptrap <clap trap.com> writes:
On Saturday, 6 March 2021 at 03:07:32 UTC, Max Haughton wrote:
 On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:

 I vaguely agree that the Garbage Collector as a global concept 
 is dated, however: Be realistic - every programming language 
 has guidelines (http://dev.stephendiehl.com/hask/ Even for 
 Haskell they can be enormous) and dos and don'ts.
"Number of guidelines" is inversely proportional to quality of design.
Mar 06
prev sibling parent James Lu <jamtlu gmail.com> writes:
On Saturday, 6 March 2021 at 03:07:32 UTC, Max Haughton wrote:
 I vaguely agree that the Garbage Collector as a global concept 
 is dated, however: Be realistic - every programming language 
 has guidelines (http://dev.stephendiehl.com/hask/ Even for 
 Haskell they can be enormous) and dos and don'ts.
Modern garbage collection is region-based memory management with only two regions: One region where you can allocate for objects that will soon die, and one region for old objects, that if you leave garbage in, your application will pause because every object allocated forces an eventual Sweep phase, forcing the runtime to look at every single object, making memory allocation an O(N*G) operation, where G is the amount of old space garbage, and N is the amount of total objects. That easily becomes O(N^2). (Yes, I'm aware of ZGC and Shenandoah, but most GCs don't work that way.)
Mar 07
prev sibling next sibling parent reply Rumbu <rumbu rumbu.ro> writes:
On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:
 Regarding the depressing GC blog article, dated 2021

 https://dlang.org/blog/2021/03/04/symphony-of-destruction-structs-classes-and-the-gc-part-one/

 At the foot it says
 Content Copyright © 2016-2018 by the D Language Foundation, All 
 Rights Reserved

 Easy to guess the current content about the GC. If a person 
 knows anything about the situation and has a small amount of 
 imagination, all of this is totally obvious. What a mess!

 Do we really have to reverse engineer the implementation to get 
 here? And a very incomplete reverse engineering indeed. The 
 blog's conclusion: The situation is bad, therefore avoid 
 interacting with it --- this is my first order approximation to 
 the article.

 So about D itself. No design here then, no intent, no 
 anticipation of the consequences of putting the pieces 
 together, no strategic reasoning about the consequences of the 
 construction of D with a garbage collector. All this long-ago 
 stuff is coming home to roost.

 The article admits it is incomplete. But what can follow except 
 workarounds if someone needs to get past this? And most won't 
 work. This is not just a roadblock. It reveals the entropy 
 inside D. The core of D is fatally flawed. D is not a language 
 where system garbage is taken care of, both concretely and 
 abstractly. The idea that we all system-program ourselves 
 around these difficulties suggests both labor and depression.

 So disappointed. I wanted D to win. But this is where the dirt 
 is, and I do not believe a strategic fix will ever be in the 
 works. Entropy rules.
I took the article with a grain of salt when I saw the author claiming that finalisation of classes is a problem in all garbage collected languages. It seems that the author failed to update himself or - what's worse - wants to justify D failure in having deterministic class destruction by pointing to others. for deterministic destruction. I bet that the next article will end with the conclusion "use structs, classes are bad, you don't really need OOP, it's obsolete. But if you insist, here is our half baked library solution". I would be happy to be wrong on this. If we want to be honest here, D failed to update/correct its OOP support, trying to please the C++ gang afraid of the garbage collector. I remember when D had built-in allocator support and delete for deterministic destructors. Now they are deprecated and we have a library solution :)
Mar 06
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 It seems that the author failed to update himself or - what's 
 worse - wants to justify D failure in having deterministic 
 class destruction by pointing to others.
D *does* have deterministic class destruction. `scope Object o = new Object;`. Also `scope(exit) .destroy(o);`. The blog is kinda awkward because it says "pretend class destructors don't exist" now, but the next parts are going to explain how he's lying to you. Class destructors do exist in D, just you need to take care to use them properly (especially since the old `scope class` was deprecated. Note this is distinct from the not-deprecated `scope Object` construct.). So the early advice is to pretend they don't exist. Then once you level up a little you can learn the truth.
Mar 06
next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:
 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 [...]
D *does* have deterministic class destruction. `scope Object o = new Object;`. Also `scope(exit) .destroy(o);`. The blog is kinda awkward because it says "pretend class destructors don't exist" now, but the next parts are going to explain how he's lying to you. Class destructors do exist in D, just you need to take care to use them properly (especially since the old `scope class` was deprecated. Note this is distinct from the not-deprecated `scope Object` construct.). So the early advice is to pretend they don't exist. Then once you level up a little you can learn the truth.
Oh, it's like scientology? 😱 Xenu
Mar 06
prev sibling next sibling parent reply Rumbu <rumbu rumbu.ro> writes:
On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:
 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 It seems that the author failed to update himself or - what's 
 worse - wants to justify D failure in having deterministic 
 class destruction by pointing to others.
D *does* have deterministic class destruction. `scope Object o = new Object;`. Also `scope(exit) .destroy(o);`. The blog is kinda awkward because it says "pretend class destructors don't exist" now, but the next parts are going to explain how he's lying to you. Class destructors do exist in D, just you need to take care to use them properly (especially since the old `scope class` was deprecated. Note this is distinct from the not-deprecated `scope Object` construct.). So the early advice is to pretend they don't exist. Then once you level up a little you can learn the truth.
A library solution is not a feature of the language. Also the scope Object declaration is not needed. At least I didn't found any reference in the language specification except the one of scoped classes, which, of course, will be deprecated.
Mar 06
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 6 March 2021 at 21:12:51 UTC, Rumbu wrote:
 A library solution is not a feature of the language.
I'm not talking about a library solution.
 Also the scope Object declaration is not needed.
That's how you get scoped, deterministic destruction of class objects in the pure language.
 At least I didn't found any reference in the language 
 specification except the one of scoped classes, which, of 
 course, will be deprecated.
That's different. That's what I meant by `scope class`, but `scope Object` is referring to it as a storage class https://dlang.org/deprecate.html#scope%20as%20a%20type%20constraint "scope as a storage class attributed to variables, function parameters, etc. is not deprecated." But this is why it is complicated: as the class author, you can't know how it will be used. The same is also true of structs, which the next part of the blog will talk about. So you need to take some appropriate care. It can be done right though.
Mar 06
prev sibling parent Nick Treleaven <nick geany.org> writes:
On Saturday, 6 March 2021 at 21:12:51 UTC, Rumbu wrote:
 At least I didn't found any reference in the language 
 specification except the one of scoped classes, which, of 
 course, will be deprecated.
"For local declarations, scope implements the RAII (Resource Acquisition Is Initialization) protocol. This means that the destructor for an object is automatically called when the reference to it goes out of scope." https://dlang.org/spec/attribute.html#scope
Mar 07
prev sibling next sibling parent Rumbu <rumbu rumbu.ro> writes:
On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:
 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 D *does* have deterministic class destruction. `scope Object o 
 = new Object;`.
Thank you Adam for opening my eyes, I lived with the impression for years that scope initialization for classes is deprecated. In this case, the documentation must be clear that only "scope classes" are scheduled for deprecation, and this does not include "scope initialization". Sorry, but when the chapter of scope classes starts with the note of deprecation, I assume that it is referring the entire chapter: https://dlang.org/spec/class.html#auto Yes, if you go further in deprecation details, you will find it written as a comment in a code example: https://dlang.org/deprecate.html#scope%20as%20a%20type%20constraint Believe it or not, a code comment is not the best place to put such important note.
Mar 06
prev sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 6 March 2021 at 20:45:00 UTC, Adam D. Ruppe wrote:
 On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 It seems that the author failed to update himself or - what's 
 worse - wants to justify D failure in having deterministic 
 class destruction by pointing to others.
D *does* have deterministic class destruction. `scope Object o = new Object;`. Also `scope(exit) .destroy(o);`.
Be warned that there is currently a bug where even `scope C c = new C;` will not actually call the destructor for extern(C++) classes: https://issues.dlang.org/show_bug.cgi?id=21693 There doesn't seem to be any fundamental reason for this, though; it's just an oversight that will hopefully be fixed soon.
Mar 10
prev sibling next sibling parent Mike Parker <aldacron gmail.com> writes:
On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:

 I bet that the next article will end with the conclusion "use 
 structs, classes are bad, you don't really need OOP, it's 
 obsolete. But if you insist, here is our half baked library 
 solution". I would be happy to be wrong on this.
Then you're going to be happy.
Mar 06
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 I took the article with a grain of salt when I saw the author 
 claiming that finalisation of classes is a problem in all 
 garbage collected languages. It seems that the author failed to 
 update himself or - what's worse - wants to justify D failure 
 in having deterministic class destruction by pointing to others.


 mechanisms for deterministic destruction.
to https://dlang.org/phobos/std_typecons.html#scoped. Mike warned only about indeterministic destructors -those that are only invoked when/if the object gets collected. To be precise, class destructors are not any more problematic than struct desturctors if they are always called deterministically -see https://dlang.org/phobos/object.html#.destroy. But if the destructor needs to be called manually anyway, why should it be a destructor at all? It's probably better to put the finalizing in a normal function, or better yet to a destructor of a RAII struct. Problems with GC-called destructors are not unique to D -see https://theartofmachinery.com/2018/12/05/gc_not_enough.html for a Ruby example.
 I remember when D had built-in allocator support and delete for 
 deterministic destructors. Now they are deprecated and we have 
 a library solution :)
I don't see why library solutions would be inherently inferior. What do we lose in case of deterministic destruction?
Mar 06
parent reply Rumbu <rumbu rumbu.ro> writes:
On Saturday, 6 March 2021 at 23:57:25 UTC, Dukc wrote:
 I don't see why library solutions would be inherently inferior. 
 What do we lose in case of deterministic destruction?
First, library solutions show the lack of commitment from language creators. There is a big difference between "This is how to do X in D" and "This is one idea how to do X in D. Use this library. Or maybe not, probably there are other options". The latter makes programmers insecure regarding the *correct* approach when they want to do something. Secondly, basic language constructs must be available in the core language not in a library, even if it's just druntime in this case. Putting them in the library makes the language unusable without that library.
Mar 06
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Sunday, 7 March 2021 at 07:10:30 UTC, Rumbu wrote:
 First, library solutions show the lack of commitment from 
 language creators. There is a big difference between "This is 
 how to do X in D" and "This is one idea how to do X in D. Use 
 this library. Or maybe not, probably there are other options". 
 The latter makes programmers insecure regarding the *correct* 
 approach when they want to do something.

 Secondly, basic language constructs must be available in the 
 core language not in a library, even if it's just druntime in 
 this case. Putting them in the library makes the language 
 unusable without that library.
I'd agree with these if we were talking about a DUB package, but the standard library is called standard for a reason. It's one of the language maintainers (Atila) that also decides what will get to Phobos, so Phobos solutions are not outsourced solutions. As for the language being unusable (or rather, less usable) without Phobos, I highly prefer it against more stuff being in the compiler. First off, it's not like you either use all of Phobos or no Phobos at all. Even a runtimeless device driver could easily use parts std.algorithm and std.range, since they are templated and do not depend on being linked to anything. Second, if something does not work in Phobos, you can just avoid using it and work around the problem. Often it's even practical to give your workaround the same API as the Phobos equivalent. I know because I have worked with compiling to WASM. If that non-working feature were in the Druntime, you suddently get obscure errors, often from the linker, due to required DRuntime symbol missing. And if the compiler had the non-working feature, you'd have to patch and recompile the compiler before you can even start the real work!
Mar 07
parent reply Siemargl <inqnone gmail.com> writes:
On Sunday, 7 March 2021 at 12:11:26 UTC, Dukc wrote:
 On Sunday, 7 March 2021 at 07:10:30 UTC, Rumbu wrote:

 As for the language being unusable (or rather, less usable) 
 without Phobos, I highly prefer it against more stuff being in 
 the compiler. First off, it's not like you either use all of 
 Phobos or no Phobos at all. Even a runtimeless device driver 
 could easily use parts std.algorithm and std.range, since they 
 are templated and do not depend on being linked to anything.
Is anywhere a list of such features ? I've seen many talk about them, but no concretic.
 Second, if something does not work in Phobos, you can just 
 avoid using it and work around the problem. Often it's even 
 practical to give your workaround the same API as the Phobos 
 equivalent. I know because I have worked with compiling to 
 WASM. If that non-working feature were in the Druntime, you 
 suddently get obscure errors, often from the linker, due to 
 required DRuntime symbol missing. And if the compiler had the 
 non-working feature, you'd have to patch and recompile the 
 compiler before you can even start the real work!
I think, druntime and phobos were divided a couple of years ago. So i can use druntime, but without phobos. Hmm, translator jokes - fixes druntime as drunktime =)
Mar 07
parent Dukc <ajieskola gmail.com> writes:
On Sunday, 7 March 2021 at 20:08:20 UTC, Siemargl wrote:
 On Sunday, 7 March 2021 at 12:11:26 UTC, Dukc wrote:
 Even a runtimeless device driver could easily use parts 
 std.algorithm and std.range, since they are templated and do 
 not depend on being linked to anything.
Is anywhere a list of such features ? I've seen many talk about them, but no concretic.
No, just test yourself. In principle everything that does not need the GC, io, multitasking or the C standard library should be available in everywhere. But since that style of programming has less users and less testing, there is often the need to work around problems.
 I think, druntime and phobos were divided a couple of years 
 ago. So i can use druntime, but without phobos.
Yes, you can. And it also works vice-versa - you can use many parts of Phobos with your own almost-nonexistently small Druntime. See https://github.com/skoppe/spasm for example.
Mar 10
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On Sunday, 7 March 2021 at 07:10:30 UTC, Rumbu wrote:

 Secondly, basic language constructs must be available in the 
 core language not in a library, even if it's just druntime in 
 this case. Putting them in the library makes the language 
 unusable without that library.
FYI, there are a bunch of language features that require library support. One of the most obvious ones would be allocating memory using `new`. -- /Jacob Carlborg
Mar 17
parent reply rsy <rsy456 hotmail.com> writes:
This is against that the D GC competes with:

https://malloc.se/blog/zgc-jdk16

Maximum: 1ms pause for the new Java GC (zgc)
Mar 23
parent Paulo Pinto <pjmlp progtools.org> writes:
On Tuesday, 23 March 2021 at 23:12:22 UTC, rsy wrote:
 This is against that the D GC competes with:

 https://malloc.se/blog/zgc-jdk16

 Maximum: 1ms pause for the new Java GC (zgc)
That is just one of many options on the Java world, there is also Azul, while PTC and Aicas have real-time ones, deployed into production including military weapons systems in battleships and missile radar controls, where a GC pause in the wrong moment is a no-go.
Mar 24
prev sibling parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Saturday, 6 March 2021 at 20:21:41 UTC, Rumbu wrote:
 

 mechanisms for deterministic destruction.
https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/implementing-dispose#dispose-and-disposebool If that is what solved looks like, D fairs pretty good. every library that supports it and not part of the language. ``` scope(exit) obj.destroy(); ``` This provides the same semantics as the using declaration. https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-8.0/using Yes, D is different, but we shouldn't pretend other language solutions actually solved it.
Mar 09
prev sibling parent Guillaume Piolat <first.name spam.org> writes:
On Saturday, 6 March 2021 at 01:33:00 UTC, NonNull wrote:
 The article admits it is incomplete. But what can follow except 
 workarounds if someone needs to get past this? And most won't 
 work. This is not just a roadblock. It reveals the entropy 
 inside D. The core of D is fatally flawed. D is not a language 
 where system garbage is taken care of, both concretely and 
 abstractly. The idea that we all system-program ourselves 
 around these difficulties suggests both labor and depression.
I have alternative guidelines: 1. For class instances where destruction order is important (they transitively hold something that isn't only GC memory), use deterministic destruction. Can be with destroy(), scope, but not with a release() function. 2. If you fear those destructors to be called by the GC, avoid it with https://p0nce.github.io/d-idioms/#GC-proof-resource-class If you accidentally rely on the GC for destruction, it will warn you. That's it. It's super simple to have everything deterministic that way.
Mar 07