www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Life in the Fast Lane ( nogc blog post)

reply Mike Parker <aldacron gmail.com> writes:
I've been meaning to get this done for weeks but have had a 
severe case of writer's block. The fact that I had no other posts 
ready to go this week and no time to write anything at all 
motivated me to make time for it and get it done anyway. My wife 
didn't complain when I told her I had to abandon our regular 
bedtime Netflix time block (though she did extract a concession 
that I have no vote in the next series we watch). Thanks to 
Vladimir, Guillaume, and Steve, for their great feedback on such 
short notice. Their assistance kept the blog from going quiet 
this week.

The blog:
https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/

Reddit:
https://www.reddit.com/r/programming/comments/6hmlfq/life_in_the_fast_lane_using_d_without_the_gc/
Jun 16 2017
next sibling parent David Gileadi <gileadisNOSPM gmail.com> writes:
On 6/16/17 6:51 AM, Mike Parker wrote:
 I've been meaning to get this done for weeks but have had a severe case 
 of writer's block. The fact that I had no other posts ready to go this 
 week and no time to write anything at all motivated me to make time for 
 it and get it done anyway. My wife didn't complain when I told her I had 
 to abandon our regular bedtime Netflix time block (though she did 
 extract a concession that I have no vote in the next series we watch). 
 Thanks to Vladimir, Guillaume, and Steve, for their great feedback on 
 such short notice. Their assistance kept the blog from going quiet this 
 week.
 
 The blog:
 https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/
 
 Reddit:
 https://www.reddit.com/r/programming/comments/6hmlfq/life_in_the_fast_lane_usi
g_d_without_the_gc/ 
Great article! And thanks for sacrificing that Netflix time :)
Jun 16 2017
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 16 June 2017 at 13:51:18 UTC, Mike Parker wrote:
 I've been meaning to get this done for weeks but have had a 
 severe case of writer's block. The fact that I had no other 
 posts ready to go this week and no time to write anything at 
 all motivated me to make time for it and get it done anyway. My 
 wife didn't complain when I told her I had to abandon our 
 regular bedtime Netflix time block (though she did extract a 
 concession that I have no vote in the next series we watch). 
 Thanks to Vladimir, Guillaume, and Steve, for their great 
 feedback on such short notice. Their assistance kept the blog 
 from going quiet this week.

 The blog:
 https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/

 Reddit:
 https://www.reddit.com/r/programming/comments/6hmlfq/life_in_the_fast_lane_using_d_without_the_gc/
Nicely written. I never bothered to look into this GC fine-tuning, as I don't need that level of optimization, but I finally have some idea of how this works.
Jun 16 2017
parent Joakim <dlang joakim.fea.st> writes:
On Friday, 16 June 2017 at 18:26:15 UTC, Joakim wrote:
 On Friday, 16 June 2017 at 13:51:18 UTC, Mike Parker wrote:
 I've been meaning to get this done for weeks but have had a 
 severe case of writer's block. The fact that I had no other 
 posts ready to go this week and no time to write anything at 
 all motivated me to make time for it and get it done anyway. 
 My wife didn't complain when I told her I had to abandon our 
 regular bedtime Netflix time block (though she did extract a 
 concession that I have no vote in the next series we watch). 
 Thanks to Vladimir, Guillaume, and Steve, for their great 
 feedback on such short notice. Their assistance kept the blog 
 from going quiet this week.

 The blog:
 https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/

 Reddit:
 https://www.reddit.com/r/programming/comments/6hmlfq/life_in_the_fast_lane_using_d_without_the_gc/
Nicely written. I never bothered to look into this GC fine-tuning, as I don't need that level of optimization, but I finally have some idea of how this works.
And people have noticed, it's about to hit the top 10 most-liked proggit links of the last 7 days: https://www.reddit.com/r/programming/top/?sort=top&t=week One typo I forgot to mention earlier, where you wrote "aren't likey."
Jun 17 2017
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 16 June 2017 at 13:51:18 UTC, Mike Parker wrote:
 I've been meaning to get this done for weeks but have had a 
 severe case of writer's block. The fact that I had no other 
 posts ready to go this week and no time to write anything at 
 all motivated me to make time for it and get it done anyway. My 
 wife didn't complain when I told her I had to abandon our 
 regular bedtime Netflix time block (though she did extract a 
 concession that I have no vote in the next series we watch). 
 Thanks to Vladimir, Guillaume, and Steve, for their great 
 feedback on such short notice. Their assistance kept the blog 
 from going quiet this week.

 The blog:
 https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/

 Reddit:
 https://www.reddit.com/r/programming/comments/6hmlfq/life_in_the_fast_lane_using_d_without_the_gc/
Thanks for the nice write-up, Mike! Posts like this one really help clear many misconceptions about the language. Keep them coming! I think the next blog should address what's possible without the GC and what features of the language/library make it more productive than languages like C and C++ even in nogc code. I'm getting the impression that people on Reddit (and in general people inexperienced with language) see this: 1. Yes, it's *technically* possible to use D without the GC. 2. However, there's this long list of things that you have to avoid. 3. So while this " nogc" is technically possible you loose so much of the language that you're better of with C++ (where e.g. "cout" just works*). * For some unspecified meaning of working. And also comments like:
 Reading this article makes me realize how naturally C++ 
 (particularly C++11) does without a GC. It's the right tool for 
 the job if your program must not have a GC.
Which obviously can't be further from the truth. That's why we need a systematic answer to how to use D effectively without the GC. The right answer is three fold: A) Examples of idiomatic D code - generic functions agnostic about the memory management strategy like range algorithms; B) Having solid tools at the language-level for implementing correct and relatively easy-to-use library primitives like smart pointers and containers - safe + pure + scope. C) Maintaining a list of functions/modules/third-party libraries that are usable in nogc land. But most of all, the focus should be on productivity and ease-of-use, because otherwise people tend to get the impression: TL;DR this is too complicated for me -> I can't use D without the GC -> Using D without the GC is not feasible.
Jun 17 2017
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Saturday, 17 June 2017 at 07:03:53 UTC, Petar Kirov 
[ZombineDev] wrote:

 The right answer is three fold:
 A) Examples of idiomatic D code - generic functions agnostic 
 about the memory management strategy like range algorithms;

 B) Having solid tools at the language-level for implementing 
 correct and relatively easy-to-use library primitives like 
 smart pointers and containers -  safe + pure + scope.

 C) Maintaining a list of functions/modules/third-party 
 libraries that are usable in  nogc land.

 But most of all, the focus should be on productivity and 
 ease-of-use, because otherwise people tend to get the 
 impression: TL;DR this is too complicated for me -> I can't use 
 D without the GC -> Using D without the GC is not feasible.
Thanks! I agree that we need a range of posts on the GC. That's the whole point of the series. I have a set of posts I can write on topics within my current realm of knowledge, or that I can beef up on when the time comes, but beyond that I'm not the right guy to author some of the posts that need to be written. I haven't familiarized myself with all the corners, nor have I written any code that required me to push the boundaries or explore the possibilities. A while back, I posted an announcement here explaining my goals with the GC series and asking for contributors. Aside from Atila's automem post, I haven't gotten any takers yet. I can, if need be, eventually write many of the posts I'd like to have, but that requires making time to familiarize myself with the topics. Such posts will also compete with those I plan to write on other topics. So, in the interest of saving me a bit of time, I invite you and anyone who's willing to share their experiences and expertise with the GC. I'll happily publish any post that demonstrates allocation strategies, lifetime management, optimizations, or just generally helps to clear up misconceptions.
Jun 17 2017
parent Eugene Wissner <belka caraus.de> writes:
On Saturday, 17 June 2017 at 08:51:04 UTC, Mike Parker wrote:
 On Saturday, 17 June 2017 at 07:03:53 UTC, Petar Kirov 
 [ZombineDev] wrote:

 The right answer is three fold:
 A) Examples of idiomatic D code - generic functions agnostic 
 about the memory management strategy like range algorithms;

 B) Having solid tools at the language-level for implementing 
 correct and relatively easy-to-use library primitives like 
 smart pointers and containers -  safe + pure + scope.

 C) Maintaining a list of functions/modules/third-party 
 libraries that are usable in  nogc land.

 But most of all, the focus should be on productivity and 
 ease-of-use, because otherwise people tend to get the 
 impression: TL;DR this is too complicated for me -> I can't 
 use D without the GC -> Using D without the GC is not feasible.
Thanks! I agree that we need a range of posts on the GC. That's the whole point of the series. I have a set of posts I can write on topics within my current realm of knowledge, or that I can beef up on when the time comes, but beyond that I'm not the right guy to author some of the posts that need to be written. I haven't familiarized myself with all the corners, nor have I written any code that required me to push the boundaries or explore the possibilities. A while back, I posted an announcement here explaining my goals with the GC series and asking for contributors. Aside from Atila's automem post, I haven't gotten any takers yet. I can, if need be, eventually write many of the posts I'd like to have, but that requires making time to familiarize myself with the topics. Such posts will also compete with those I plan to write on other topics. So, in the interest of saving me a bit of time, I invite you and anyone who's willing to share their experiences and expertise with the GC. I'll happily publish any post that demonstrates allocation strategies, lifetime management, optimizations, or just generally helps to clear up misconceptions.
Hm. In my personal projects I'm using D exclusively as a high-level C, so without the GC. I have also Unique and RefCounted, that are very new and not so well tested. On the other hand I've written a good set of containers, that are completely nogc and are different from std and EMSI containers. I would be interested to write a few lines about container usage, why not built-in arrays and so on. But I should say I need some time, I have currently enough to write at the semester's end :)
Jun 17 2017
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 17 June 2017 at 07:03:53 UTC, Petar Kirov 
[ZombineDev] wrote:
 2. However, there's this long list of things that you have to 
 avoid.
There's like 10 things to avoid in the language itself: http://dlang.org/spec/garbage.html#op_involving_gc and most of them are obviously array stuff so it is easy to remember - and easy to skip with a simple array helper... which is often a useful optimization too, make one that uses stack space! You can write one in about 15 minutes. Now, nogc is a bit more restrictive since it isn't just avoiding GC calls.... it avoids the *possibility* of GC calls. Even if it would only actually run one in a million times at which point you want the program to die anyway.... nogc still bans it. Even if it would never actually run, but some leaf function somewhere wrote `int getLength() { return _length; }` instead of `int getLength() nogc pure const safe nothrow { return _length; }`... it obviously doesn't run the gc, but still fails nogc for its entire call tree. This is why I consider nogc to be an *extremely* niche feature and generally harmful. It makes things look a lot harder than it really is - people confuse ` nogc` with "no gc". Instead, I suggest just minding the list and using runtime profiling and debugging to ensure your needs are met in actual practice.
 3. So while this " nogc" is technically possible you loose so 
 much of the language that you're better of with C++ (where e.g. 
 "cout" just works*).
of course you could always `printf` lol. The reason writeln fails nogc is that it *might* throw an exception with most args if stdout is closed or something. Perfect example of an *extremely* rare case failing nogc's ridiculously strict requirements.
Jun 17 2017
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Saturday, 17 June 2017 at 13:05:50 UTC, Adam D. Ruppe wrote:
 This is why I consider  nogc to be an *extremely* niche feature 
 and generally harmful. It makes things look a lot harder than 
 it really is - people confuse ` nogc` with "no gc". Instead, I 
 suggest just minding the list and using runtime profiling and 
 debugging to ensure your needs are met in actual practice.
As a counterpoint: It's true that it's a bit niche, but when you have "no gc" then nogc really helps with peace of mind (only one allocation and you may crash).
Jun 17 2017
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
Guillaume Piolat wrote:

 On Saturday, 17 June 2017 at 13:05:50 UTC, Adam D. Ruppe wrote:
 This is why I consider  nogc to be an *extremely* niche feature and 
 generally harmful. It makes things look a lot harder than it really is - 
 people confuse ` nogc` with "no gc". Instead, I suggest just minding the 
 list and using runtime profiling and debugging to ensure your needs are 
 met in actual practice.
As a counterpoint: It's true that it's a bit niche, but when you have "no gc" then nogc really helps with peace of mind (only one allocation and you may crash).
yeah, it helps a little... in (let's be honest) rare situations where you really want to be *completely* free of any possible gc allocation. in most real-life scenarios you're ok with "mostly gc-free, except for exceptional cases" (like writeln, for example), and you have no way to express it. i end up not using nogc at all (except on simple `return a` getters), despite the fact that many of my code rarely allocates. there should be a way to say "i don't want gc on this code path, but it is ok for this one" (like throwing), without resorting to dirty tricks with casting and lambdas. something like ` raregc` ;-). but then, how we should define "rare"? that attribute will quickly become useless, as people will mark arbitrary code with it. so, ` nogc` is mostly useless IRL (you can't even `enforce` with it), and ` raregc` will become useless if introduced. and the sole reason (as i see it) of introducing ` nogc` was to please people complaining about gc allocations, no matter how often they're done, and on which code path. i myself see it as "PR design", which attracts people only to make them find that using D with ` nogc` is a PITA, due to ` nogc` being too restrictive. what ` nogc` *really* does now is dividing codebases into "i will do all kind of dirty tricks to avoid GC at all costs, even if it is not really necessary", and into "ah, screw it, i don't care". to me, it looks like ` nogc` does more bad than good now.
Jun 17 2017
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 17 June 2017 at 13:09:42 UTC, Guillaume Piolat wrote:
 As a counterpoint: It's true that it's a bit niche, but when 
 you have "no gc" then  nogc really helps with peace of mind 
 (only one allocation and you may crash).
Yes, when you actually need it it might be helpful, and then the associated niche libraries might use it too. Of course, you can still lie about the attributes...
Jun 17 2017
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Saturday, 17 June 2017 at 13:05:50 UTC, Adam D. Ruppe wrote:
 The reason writeln fails  nogc is that it *might* throw an 
 exception with most args if stdout is closed or something.

 Perfect example of an *extremely* rare case failing  nogc's 
 ridiculously strict requirements.
If that Walter's DIP about reference-counted exceptions gets trough it should ease problem like that quite a bit. And as he said, memory-safe programming is not the problem -verifying the safety is. I think it is the same with GC. Didn't Don Clugston say about something at some Dconf that it must trigger allocation "not hardly ever, NEVER"? So there's a real need even for the rigid nogc, albeit a niche one.
Jun 17 2017
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 17 June 2017 at 16:59:56 UTC, Dukc wrote:
 If that Walter's DIP about reference-counted exceptions gets 
 trough it should ease problem like that quite a bit.
Possibly. There's a lot of solutions to the exception thing though, and it doesn't actually bother me if you see nogc as being very niche because you can just use one of those solutions today. (most of them depend simply on the catching code calling some disposal function, or living with a (possibly temporary) leak. Aside from that, the *most* you have to do is just `throw make!Exception` instead of `throw new Exception` and problem solved).
 Didn't Don Clugston say about something at some Dconf that it 
 must trigger allocation "not hardly ever, NEVER"? So there's a 
 real need even for the rigid  nogc, albeit a niche one.
Yes, there are a few cases for it, but most people on reddit aren't one of them. It reminds me of the "web scale" thing. "but can this handle 1 million hits per second?" Probably not, but you also almost certainly going to need to... and if you do, the company will surely have the budget to hire a whole team of full time people to develop some specialized solution to their problem. The D language should certainly never get in the way of those specialized solutions, whether scaling, or low latency audio/bidding, or whatever else, but I'm not convinced it needs to support them all out of the box either - and the average D user (or web developer in general) certainly doesn't have to worry.
Jun 17 2017
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 06/17/2017 10:39 AM, Adam D. Ruppe wrote:
 On Saturday, 17 June 2017 at 16:59:56 UTC, Dukc wrote:
 Didn't Don Clugston say about something at some Dconf that it must
 trigger allocation "not hardly ever, NEVER"? So there's a real need
 even for the rigid  nogc, albeit a niche one.
Yes, there are a few cases for it, but most people on reddit aren't one of them.
[...]
 and the average D user (or web developer in general)
 certainly doesn't have to worry.
I would like to see these ideas in a blog post. It's liberating when assumed problems are convinced away. :) Ali
Jun 18 2017
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 19 June 2017 at 00:12:25 UTC, Ali Çehreli wrote:
 I would like to see these ideas in a blog post. It's liberating 
 when assumed problems are convinced away. :)
True, but I think the very blog post we're talking about already does that.
Jun 19 2017
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 06/19/2017 12:12 AM, Dukc wrote:
 On Monday, 19 June 2017 at 00:12:25 UTC, Ali Çehreli wrote:
 I would like to see these ideas in a blog post. It's liberating when
 assumed problems are convinced away. :)
True, but I think the very blog post we're talking about already does that.
LOL. You're right. :) I think I've been thinking more more about Adam's observations about nogc. I've been thinking myself that it's unfortunate that throwing an exception makes a function no- nogc. I know that there are efforts towards nogc exceptions but I think nogc should somehow related only to the normal operation of code. Since exceptions are for error cases, perhaps they shouldn't have an effect on whether a function is nogc. Ali
Jun 19 2017
parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Monday, 19 June 2017 at 10:13:33 UTC, Ali Çehreli wrote:
 I know that there are efforts towards  nogc exceptions but I 
 think  nogc should somehow related only to the normal operation 
 of code. Since exceptions are for error cases, perhaps they 
 shouldn't have an effect on whether a function is  nogc.
This strikes me as making a lot of sense. You can compare it to the system, trusted and safe trio, where system is the default and trusted could be verbosely translated as unsafe_code_that_is_acceptable_to_be_called_in_safe_code_as_an_exception_to_the_rule. Transferring this to GC, you could have gc (the default), nogc (the safe equivalent) and the trusted equivalent: gc_code_that_is_acceptable_to_be_called_in_nogc_code_as_an_ex eption_to_the_rule. I'll abbreviate this as gc78 until there is a better name. With this, exceptions can be defined to be gc78 to make them work in nogc code: As exceptions are exceptional, one can argue that it is OK for them to do exceptional things. Also, you can write a debugging function that uses string concatenation and writeln inside a DebugCondition, mark it as gc78 and you'll still be able to call it in nogc code. This won't cause collection in release versions. Bastiaan.
Jun 20 2017
parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 20 June 2017 at 09:56:03 UTC, Bastiaan Veelo wrote:

 Transferring this to GC, you could have  gc (the default), 
  nogc (the  safe equivalent) and the  trusted equivalent: 
  gc_code_that_is_acceptable_to_be_called_in_nogc_code_as_an_ex
eption_to_the_rule. I'll abbreviate this as  gc78 until there is a better name.
It can already be done: void gc78writeln(string what) nogc { import std.stdio; alias WritelnType = void function(string) nogc; (cast(WritelnType)&writeln!string)(what); } void main() nogc { import std.stdio; gc78writeln("hello world!"); } I wasn't skilled enough to make a generic version of this trough. And I do not know what happens if nogc function ends up calling gc.
Jun 20 2017
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 20 June 2017 at 15:24:06 UTC, Dukc wrote:

 I wasn't skilled enough to make a generic version of this 
 trough. And I do not know what happens if  nogc function ends 
 up calling gc.
This seems like it only would work if you know that there wouldn't be any allocations. I don't really know though. Below is a little more generic: template gc78f(alias f) { void gc78f(T...)(T what) nogc { import std.stdio; alias fType = void function(T) nogc; (cast(fType)&f!T)(what); } } void main() nogc { import std.stdio; string x = "hello world!"; gc78f!(writeln)(x); }
Jun 20 2017
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 20 June 2017 at 16:47:20 UTC, jmh530 wrote:
 Below is a little more generic:
Opps, left an extra import std.stdio in there.
Jun 20 2017
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 06/20/2017 09:47 AM, jmh530 wrote:
 On Tuesday, 20 June 2017 at 15:24:06 UTC, Dukc wrote:

 I wasn't skilled enough to make a generic version of this trough. And
 I do not know what happens if  nogc function ends up calling gc.
This seems like it only would work if you know that there wouldn't be any allocations. I don't really know though. Below is a little more generic: template gc78f(alias f) { void gc78f(T...)(T what) nogc { import std.stdio; alias fType = void function(T) nogc; (cast(fType)&f!T)(what); } } void main() nogc { import std.stdio; string x = "hello world!"; gc78f!(writeln)(x); }
And there is also assumeNoGC() which has been discussed before. (The scary thing is, although it happened just five months ago I don't remember taking part in it. :( ) http://forum.dlang.org/post/oat6lu$n94$1 digitalmars.com Ali
Jun 20 2017
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Fri, 16 Jun 2017 13:51:18 +0000
schrieb Mike Parker <aldacron gmail.com>:

 I've been meaning to get this done for weeks but have had a=20
 severe case of writer's block. The fact that I had no other posts=20
 ready to go this week and no time to write anything at all=20
 motivated me to make time for it and get it done anyway. My wife=20
 didn't complain when I told her I had to abandon our regular=20
 bedtime Netflix time block (though she did extract a concession=20
 that I have no vote in the next series we watch). Thanks to=20
 Vladimir, Guillaume, and Steve, for their great feedback on such=20
 short notice. Their assistance kept the blog from going quiet=20
 this week.
=20
 The blog:
 https://dlang.org/blog/2017/06/16/life-in-the-fast-lane/
=20
 Reddit:
 https://www.reddit.com/r/programming/comments/6hmlfq/life_in_the_fast_lan=
e_using_d_without_the_gc/
=20
=20
Nice blog post!
 Let=E2=80=99s imagine a hypothetical programmer named J.P. who, for reaso=
ns
 he considers valid, has decided he would like to avoid garbage
 collection completely in his D program. He has two immediate options.
I think I might know that hypothetical programmer ;-) -- Johannes
Jun 17 2017