www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - About GC: The Future of Rust : GC integration

reply Dsby <dushibaiyu yahoo.com> writes:
http://rustaceans.cologne/2016/06/06/rust-anniversary-part-2.html
in sides 3. : The Future of Rust
in the last 2 pages:
                  •GC integration


In rust, it will add the GC integration.
but in our D, many people want to kill the GC.
It like a joke.
Jun 07 2016
next sibling parent reply Eugene Wissner <belka caraus.de> writes:
On Wednesday, 8 June 2016 at 02:47:14 UTC, Dsby wrote:

 http://rustaceans.cologne/2016/06/06/rust-anniversary-part-2.html
 in sides 3. : The Future of Rust
 in the last 2 pages:
                  •GC integration


 In rust, it will add the GC integration.
 but in our D, many people want to kill the GC.
 It like a joke.


It isn't a joke. It is all about the possibility to choose 
between the GC and no GC. Rust will have it. In D some very 
important things like exceptions depend on GC.
You can use GC also with C++ if you want. It is all about 
preferences..
Jun 07 2016
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Wednesday, 8 June 2016 at 03:10:32 UTC, Eugene Wissner wrote:

 In D some very important things like exceptions depend on GC.


This is a common misconception. Exceptions do not have to use the 
GC, they just often are. All you have to do is malloc an 
exception and then throw it, and then remember to free it after 
you catch it up the call stack.

The Phobos developers made the decision to use the GC in order to 
be  safe rather than fast.
Jun 07 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 03:19:18 UTC, Jack Stouffer wrote:

 The Phobos developers made the decision to use the GC in order 
 to be  safe rather than fast.


Really, if performance matter to you, you'd better not have 
exception in the critical path to boot. I'm more and more 
convinced that even RC system should not try to decrement and 
leak (ie delegate to the GC) when unwinding.
Jun 07 2016
parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Wednesday, 8 June 2016 at 03:27:01 UTC, deadalnix wrote:

 I'm more and more convinced that even RC system should not try 
 to decrement and leak (ie delegate to the GC) when unwinding.


Wouldn't exceptions then still have to be registered to and then 
scanned by the GC? Doesn't that kind of defeat the purpose?
Jun 07 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 03:45:34 UTC, Jack Stouffer wrote:

 On Wednesday, 8 June 2016 at 03:27:01 UTC, deadalnix wrote:

 I'm more and more convinced that even RC system should not try 
 to decrement and leak (ie delegate to the GC) when unwinding.


 Wouldn't exceptions then still have to be registered to and 
 then scanned by the GC? Doesn't that kind of defeat the purpose?


GC is only a problem when it collects, which won't happen often 
if you don't leak that much. Used as a malloc/free, there is no 
reason for it to be slower than a malloc/free (a fast malloc 
already needs to be able to find the allocation metadata from 
free's argument effisciently, so you you got what you need for a 
GC already).
Jun 07 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jun 08, 2016 at 03:50:26AM +0000, deadalnix via Digitalmars-d wrote:

 On Wednesday, 8 June 2016 at 03:45:34 UTC, Jack Stouffer wrote:

 On Wednesday, 8 June 2016 at 03:27:01 UTC, deadalnix wrote:

 I'm more and more convinced that even RC system should not try to
 decrement and leak (ie delegate to the GC) when unwinding.


 
 Wouldn't exceptions then still have to be registered to and then
 scanned by the GC? Doesn't that kind of defeat the purpose?


 
 GC is only a problem when it collects, which won't happen often if you
 don't leak that much. Used as a malloc/free, there is no reason for it
 to be slower than a malloc/free (a fast malloc already needs to be
 able to find the allocation metadata from free's argument
 effisciently, so you you got what you need for a GC already).


I thought the usual implementation is to store the metadata immediately
before the pointer, so it's O(1) to look it up.


T

-- 
Which is worse: ignorance or apathy? Who knows? Who cares? -- Erich Schubert
Jun 07 2016
parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 04:15:17 UTC, H. S. Teoh wrote:

 I thought the usual implementation is to store the metadata 
 immediately before the pointer, so it's O(1) to look it up.


No, it gets you terrible fragmentation and it makes it easy to 
corrupt allocator data with just a buffer overflow/underflow, 
which is a big nono.

But the general idea remain: the metadata address can indeed be 
found in O(1) from the address of the data, usually with some 
cheap bit twiddling, for most allocations (for instance, jemalloc 
does it in O(1) fro all allocation < 256k and in log(n) for 
larger ones).
Jun 07 2016
prev sibling parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Wednesday, 8 June 2016 at 03:50:26 UTC, deadalnix wrote:

 (a fast malloc already needs to be able to find the allocation 
 metadata from free's argument effisciently,


BTW GLib (GTK+) have a GSlice API that doesn't allow resizing and 
requires the allocation size to be passed when freeing:

https://developer.gnome.org/glib/stable/glib-Memory-Slices.html#glib-Memory-Slices.description

I wonder if Andrei's allocator API supports freeing with a size 
argument.
Jun 08 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/16 1:59 PM, Nick Treleaven wrote:

 I wonder if Andrei's allocator API supports freeing with a size argument.


Size is required with all deallocations. -- Andrei
Jun 08 2016
parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Wednesday, 8 June 2016 at 12:38:25 UTC, Andrei Alexandrescu 
wrote:

 On 6/8/16 1:59 PM, Nick Treleaven wrote:

 I wonder if Andrei's allocator API supports freeing with a 
 size argument.


 Size is required with all deallocations. -- Andrei


Thanks. So I suppose it's up to each allocator whether this will 
work or not:

alloc.makeArray!T(5).ptr.Alias!(p => alloc.dispose(p));

I guess it would work for Mallocator.
Jun 09 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 11:59:49 UTC, Nick Treleaven wrote:

 On Wednesday, 8 June 2016 at 03:50:26 UTC, deadalnix wrote:

 (a fast malloc already needs to be able to find the allocation 
 metadata from free's argument effisciently,


 BTW GLib (GTK+) have a GSlice API that doesn't allow resizing 
 and requires the allocation size to be passed when freeing:

 https://developer.gnome.org/glib/stable/glib-Memory-Slices.html#glib-Memory-Slices.description

 I wonder if Andrei's allocator API supports freeing with a size 
 argument.


Yes, there are many of these initiatives these days. I was 
initially enthusiastic, but truth is, nobody has been able to 
produce anything that beat tcmalloc or jemalloc (clearly the king 
in town these days).

This extra bit of infos seems to be less useful in practice than 
you'd naively expect. Someone may come up with something 
interesting to do with it someday, but I haven't seen anything 
that groundbreaking so far.
Jun 08 2016
parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On Wednesday, 8 June 2016 at 17:23:22 UTC, deadalnix wrote:

 I wonder if Andrei's allocator API supports freeing with a 
 size argument.




...

 This extra bit of infos seems to be less useful in practice 
 than you'd naively expect. Someone may come up with something 
 interesting to do with it someday, but I haven't seen anything 
 that groundbreaking so far.


OK. What if the allocation size was passed as a compile time 
parameter on deallocation? Then CTFE could calculate e.g. the 
next power of 2.
Jun 10 2016
prev sibling next sibling parent Eugene Wissner <belka caraus.de> writes:
On Wednesday, 8 June 2016 at 03:19:18 UTC, Jack Stouffer wrote:

 On Wednesday, 8 June 2016 at 03:10:32 UTC, Eugene Wissner wrote:

 In D some very important things like exceptions depend on GC.


 This is a common misconception. Exceptions do not have to use 
 the GC, they just often are. All you have to do is malloc an 
 exception and then throw it, and then remember to free it after 
 you catch it up the call stack.

 The Phobos developers made the decision to use the GC in order 
 to be  safe rather than fast.


Thanks for the clarification. Just tested it with the Mallocator. 
It works fine!
Jun 07 2016
prev sibling parent reply Matthias Bentrup <matthias.bentrup googlemail.com> writes:
On Wednesday, 8 June 2016 at 03:19:18 UTC, Jack Stouffer wrote:

 On Wednesday, 8 June 2016 at 03:10:32 UTC, Eugene Wissner wrote:

 In D some very important things like exceptions depend on GC.


 This is a common misconception. Exceptions do not have to use 
 the GC, they just often are. All you have to do is malloc an 
 exception and then throw it, and then remember to free it after 
 you catch it up the call stack.

 The Phobos developers made the decision to use the GC in order 
 to be  safe rather than fast.


Exceptions and memory allocation are a pain to use anyway. When 
you call a function that calls a function that calls a function, 
and you get an Exception, how do you know how to properly 
deallocate it ?

And it's not just the Exception Object itself, usually you also 
have to allocate at least a string for the Exception message.
Jun 08 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/16 3:08 PM, Matthias Bentrup wrote:

 On Wednesday, 8 June 2016 at 03:19:18 UTC, Jack Stouffer wrote:

 On Wednesday, 8 June 2016 at 03:10:32 UTC, Eugene Wissner wrote:

 In D some very important things like exceptions depend on GC.


 This is a common misconception. Exceptions do not have to use the GC,
 they just often are. All you have to do is malloc an exception and
 then throw it, and then remember to free it after you catch it up the
 call stack.

 The Phobos developers made the decision to use the GC in order to be
  safe rather than fast.


 Exceptions and memory allocation are a pain to use anyway. When you call
 a function that calls a function that calls a function, and you get an
 Exception, how do you know how to properly deallocate it ?

 And it's not just the Exception Object itself, usually you also have to
 allocate at least a string for the Exception message.


Correct. We need a mechanism for deallocating exceptions automatically. 
Their error message should be a CStr. -- Andrei
Jun 08 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 8 June 2016 at 03:10:32 UTC, Eugene Wissner wrote:

 You can use GC also with C++ if you want. It is all about 
 preferences..


Yes, Inkscape uses the boehm collector, which is the same kind of 
collector D uses. Inkscape feels sluggish.

I've yet to see a large desktop app relying on GC that does not 
feel sluggish.

Even JetBrain products feel sluggish, with a top-of-the-line? 
collector, so maybe for other reasons. Or maybe not.

Applications that perform well, and use a GC, seem to only use 
the GC for the "scripting" part of the application, but not the 
core engine of the app.
Jun 07 2016
next sibling parent reply Chris <wendlec tcd.ie> writes:
On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 8 June 2016 at 03:10:32 UTC, Eugene Wissner wrote:

 You can use GC also with C++ if you want. It is all about 
 preferences..


 Yes, Inkscape uses the boehm collector, which is the same kind 
 of collector D uses. Inkscape feels sluggish.

 I've yet to see a large desktop app relying on GC that does not 
 feel sluggish.


I've used Inkscape quite a bit and I don't find it sluggish. 
Question: did it start to feel sluggish after you found out it 
used the Boehm collector or before? Sometimes programs `feel` 
fast or sluggish according to what you know about them. Yet 
benchmarking can prove your intuition wrong.

[snip]
Jun 08 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 8 June 2016 at 11:57:40 UTC, Chris wrote:

 Question: did it start to feel sluggish after you found out it 
 used the Boehm collector or before? Sometimes programs `feel` 
 fast or sluggish according to what you know about them. Yet 
 benchmarking can prove your intuition wrong.


No, I'm objective ;-). It has always felt sluggish, how much 
depends on the version and the load. Maybe recent versions are 
less sluggish under high load, I assume they have spent some 
effort on improvements.
Jun 08 2016
prev sibling next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
wrote:

 I've yet to see a large desktop app relying on GC that does not 
 feel sluggish.


i've yet to see a large desktop app that does not feel sluggish.
Jun 08 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 8 June 2016 at 12:04:34 UTC, ketmar wrote:

 On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
 wrote:

 I've yet to see a large desktop app relying on GC that does 
 not feel sluggish.


 i've yet to see a large desktop app that does not feel sluggish.


Well, Atom and Microsoft Visual Code are quite ok, despite using 
GC-backed JavaScript. But then again, the core engine is in C++ 
and the GC is concurrent and the heap probably small.

D's GC does not stand a chance without support for using 
partitioned datasets (either by context, fiber or thread) for 
large applications that do extensive caching -> 32 GB heap.

Compare this to a language like Pony that has per-actor GC and 
can have millions of actors.

The sane and scalable thing to do is associating a GC heap to the 
fiber and then use RC for borrowing (references that escape the 
fiber are reference counted, keeping the object pinned).
Jun 08 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 8 June 2016 at 12:04:34 UTC, ketmar wrote:

 On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
 wrote:

 I've yet to see a large desktop app relying on GC that does 
 not feel sluggish.


 i've yet to see a large desktop app that does not feel sluggish.


Most games.
Jun 08 2016
parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 8 June 2016 at 17:13:38 UTC, deadalnix wrote:

 On Wednesday, 8 June 2016 at 12:04:34 UTC, ketmar wrote:

 On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
 wrote:

 I've yet to see a large desktop app relying on GC that does 
 not feel sluggish.


 i've yet to see a large desktop app that does not feel 
 sluggish.


 Most games.


it is usually enough to hit "load" to make 'em hang for a long 
time. often it is enough to simply *start* a game to have that 
effect.
Jun 08 2016
parent reply Adam Wilson <flyboynw gmail.com> writes:
ketmar wrote:

 On Wednesday, 8 June 2016 at 17:13:38 UTC, deadalnix wrote:

 On Wednesday, 8 June 2016 at 12:04:34 UTC, ketmar wrote:

 On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad wrote:

 I've yet to see a large desktop app relying on GC that does not feel
 sluggish.


 i've yet to see a large desktop app that does not feel sluggish.


 Most games.


 it is usually enough to hit "load" to make 'em hang for a long time.
 often it is enough to simply *start* a game to have that effect.


But to be fair, that's not a memory management problem but a disk IO 
problem.

-- 
// Adam Wilson
// import quiet.dlang.dev;
Jun 08 2016
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thursday, 9 June 2016 at 05:52:47 UTC, Adam Wilson wrote:

 But to be fair, that's not a memory management problem but a 
 disk IO problem.


but still, example is not really works. also, i bet many people 
expirienced "lags", even in single-player games: lags aren't 
necessarily I/O problems.
Jun 08 2016
prev sibling parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 9 June 2016 at 05:52:47 UTC, Adam Wilson wrote:

 But to be fair, that's not a memory management problem but a 
 disk IO problem.


Quoting for truth. The single biggest problem large games have is 
bandwidth, both in terms of offline storage and memory/bus 
bandwidth. This will only get worse with time as the quality of 
resources increases at a faster rate than bandwidth increases.
Jun 09 2016
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
wrote:

 
 Yes, Inkscape uses the boehm collector, which is the same kind 
 of collector D uses. Inkscape feels sluggish.


Wow, I did not expect to see "Inkscape" pop up in the D forums!
Inkscape can be sluggish, but that has nothing to do with GC.

- Johan
(I was one of the main contributors of Inkscape)

[*] For example, many operations result in writing the 
human-readable SVG string representation of path data and then 
reading that back in again to complete the operation. This is of 
course not very fast for large paths.
Jun 08 2016
next sibling parent Chris <wendlec tcd.ie> writes:
On Wednesday, 8 June 2016 at 15:24:45 UTC, Johan Engelen wrote:

 On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
 wrote:

 
 Yes, Inkscape uses the boehm collector, which is the same kind 
 of collector D uses. Inkscape feels sluggish.


 Wow, I did not expect to see "Inkscape" pop up in the D forums!
 Inkscape can be sluggish, but that has nothing to do with GC.

 - Johan
 (I was one of the main contributors of Inkscape)

 [*] For example, many operations result in writing the 
 human-readable SVG string representation of path data and then 
 reading that back in again to complete the operation. This is 
 of course not very fast for large paths.


That explains why it's not sluggish when I use it. I use it for 
simple graphics (icons, logos and the like). Nothing too fancy.

The point to take home is that it's not the GC! Funny :-)

By the way, thanks Johan for Inkscape, I really appreciate it!
Jun 08 2016
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 8 June 2016 at 15:24:45 UTC, Johan Engelen wrote:

 On Wednesday, 8 June 2016 at 06:19:08 UTC, Ola Fosheim Grøstad 
 wrote:

 
 Yes, Inkscape uses the boehm collector, which is the same kind 
 of collector D uses. Inkscape feels sluggish.


 Wow, I did not expect to see "Inkscape" pop up in the D forums!
 Inkscape can be sluggish, but that has nothing to do with GC.

 - Johan
 (I was one of the main contributors of Inkscape)

 [*] For example, many operations result in writing the 
 human-readable SVG string representation of path data and then 
 reading that back in again to complete the operation. This is 
 of course not very fast for large paths.


Well, that explains why Inkscape consistently feels sluggish. I 
believe it also freeze short-term from time to time which I 
normally would attribute to boehm-style GC, but given what you 
say it could be due to a number of causes so not a useful case 
either way.
Jun 08 2016
prev sibling parent reply David <David.dave dave.com> writes:
On Wednesday, 8 June 2016 at 02:47:14 UTC, Dsby wrote:

 http://rustaceans.cologne/2016/06/06/rust-anniversary-part-2.html
 in sides 3. : The Future of Rust
 in the last 2 pages:
                  •GC integration


 In rust, it will add the GC integration.
 but in our D, many people want to kill the GC.
 It like a joke.


It's not a joke. Rust is doing it the correct way. GC makes a lot 
of sense as a library solution. I need to dig into Rust, seeing 
how my D project ain't going anywhere.
Jun 08 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/16 2:35 PM, David wrote:

 my D project ain't going anywhere.


What's the holdup? -- Andrei
Jun 08 2016