www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [OT] Why mobile web apps are slow

reply Paulo Pinto <pjmlp progtools.org> writes:
A bit off-topic, but well worth reading,

http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

--
Paulo
Jul 09 2013
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jul 9, 2013, at 11:12 AM, Paulo Pinto <pjmlp progtools.org> wrote:

 A bit off-topic, but well worth reading,
=20
 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

Thanks for the link. In my experience, mobile networking is slow in = general. When I run Speedtest on my phone vs. a laptop sitting right = next to it, the phone has a fraction of the bandwidth of the laptop. So = there's even more at issue than raw JS performance.=
Jul 09 2013
prev sibling next sibling parent "QAston" <qaston gmail.com> writes:
On Tuesday, 9 July 2013 at 18:12:24 UTC, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

 --
 Paulo

I think that the garbage collection part of the atricle is very relevant to the usage of D on mobile.
Jul 09 2013
prev sibling next sibling parent "Joakim" <joakim airpost.net> writes:
On Tuesday, 9 July 2013 at 19:27:22 UTC, QAston wrote:
 On Tuesday, 9 July 2013 at 18:12:24 UTC, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

 --
 Paulo

I think that the garbage collection part of the atricle is very relevant to the usage of D on mobile.

If Intel ever makes a comeback on mobile- Samsung is putting x86 chips in their next generation of Galaxy Tab tablets, so it's possible- it might not take much effort to port D/ldc to Android/x86 though, so maybe it will be relevant someday. Good article, with some good data.
Jul 09 2013
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
On Tuesday, 9 July 2013 at 18:12:24 UTC, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

An interesting article, a strong critique of many kinds of garbage collection. Having good enough built-in means to avoid relying on a GC seems quite important to use D where memory is limited. Bye, bearophile
Jul 09 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/10/2013 7:52 AM, qznc wrote:
 On Tuesday, 9 July 2013 at 20:39:30 UTC, Brian Rogoff wrote:
 PS: That silhouette of the SR-71 at the point allocators are mentioned sets a
 high bar for the design!

I did not like that analogy at all. Have you seen the user interface of an SR-71? http://www.likecool.com/Gear/Pic/Spy%20Plane%20SR71%20Blackbird%20Cockpit/big/Spy-Plane-SR71-Blackbird-Cockpit.jpg

I always love the utilitarian design of military cockpits. No logos, no fake wood grain paneling, no styling, no color scheme, no cupholder. All business.
Jul 10 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/10/2013 12:40 PM, Nick Sabalausky wrote:
 On Wed, 10 Jul 2013 10:47:16 -0700
 Walter Bright <newshound2 digitalmars.com> wrote:

 On 7/10/2013 7:52 AM, qznc wrote:
 On Tuesday, 9 July 2013 at 20:39:30 UTC, Brian Rogoff wrote:
 PS: That silhouette of the SR-71 at the point allocators are
 mentioned sets a high bar for the design!

I did not like that analogy at all. Have you seen the user interface of an SR-71? http://www.likecool.com/Gear/Pic/Spy%20Plane%20SR71%20Blackbird%20Cockpit/big/Spy-Plane-SR71-Blackbird-Cockpit.jpg

I always love the utilitarian design of military cockpits. No logos, no fake wood grain paneling, no styling, no color scheme, no cupholder. All business.

No cupholders is all fine and good until the sky gets clogged with ariel traffic and you have to put down your coffee. *Then* you'll regret not opting for the deluxe package!

Ok, I'll concede the cupholder! But I'm holding the line on the 8-track.
Jul 10 2013
prev sibling next sibling parent "Brian Rogoff" <brogoff gmail.com> writes:
On Tuesday, 9 July 2013 at 20:02:33 UTC, bearophile wrote:
 On Tuesday, 9 July 2013 at 18:12:24 UTC, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

An interesting article, a strong critique of many kinds of garbage collection. Having good enough built-in means to avoid relying on a GC seems quite important to use D where memory is limited.

It's not just in the limited memory case of mobile devices that GC is a problem. There are a few domains that come to mind where the ability to escape GC and rely on manual memory management is extremely important. While there was some reference made to an approach to allocators in the closing talk of DConf, I haven't seen much discussion of an approach using D's features. -- Brian PS: That silhouette of the SR-71 at the point allocators are mentioned sets a high bar for the design!
Jul 09 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Tue, 09 Jul 2013 20:12:25 +0200
Paulo Pinto <pjmlp progtools.org> wrote:

 A bit off-topic, but well worth reading,
 
 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/
 
 --
 Paulo

Very good article. (Actually, a nice site design, too.)
Jul 09 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/9/2013 11:12 AM, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

It isn't off-topic at all. It's very relevant to D. I also agree with what he says about GC.
Jul 09 2013
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 It isn't off-topic at all. It's very relevant to D.

 I also agree with what he says about GC.

There's a long way from recognizing those problems, to having good enough solutions in D. Some possible attack strategies for D are: - A less allocating Phobos to produce a bit less garbage; - A more precise GC to avoid some memory leaks; - Perhaps an annotation to disallow heap allocations in a function or a piece of code; - Some good way to allocate variable length arrays on the stack (http://d.puremagic.com/issues/show_bug.cgi?id=9832 ) (so some arrays produce no garbage); - The implementation of "scope" maybe helps a bit; - Multiple alias this opens some opportunities to use structs in more places, avoiding some heap allocations. - Currently Phobos Scoped/emplace are not very good. Is that enough? Rust language designers seem to think that's not enough. Opinions are welcome. Bye, bearophile
Jul 09 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 9 July 2013 at 22:40:31 UTC, bearophile wrote:
 - A less allocating Phobos to produce a bit less garbage;

Yes, and options to pass output ranges to more functions too, instead of always returning gc allocated things.
 - Perhaps an annotation to disallow heap allocations in a 
 function or a piece of code;

I don't think that will often get used, especially when we're already doing safe pure const static void foo(); as it is.
 - Some good way to allocate variable length arrays on the stack 
 (http://d.puremagic.com/issues/show_bug.cgi?id=9832 ) (so some 
 arrays produce no garbage);

These might be cool. Something we can do today is use static arrays as buffers. In my non-gc D experiments, I've made great use of a StackArray!(type, max_capacity) with the convenience operators overloaded. Combined with a solid "scope" implementation, this could be fast, convenient, and safe. Of course it would need to have a statically known max length big enough to store what you want to store without being too wasteful. But I don't think that's particularly hard in most cases - at least we have RangeError so it dies gracefully instead of buffer overflowing like you'd get i C. (It could also automatically grow to the gc heap if the programmer is ok with that.)
 - The implementation of "scope" maybe helps a bit;

Yes.
 Is that enough? Rust language designers seem to think that's 
 not enough. Opinions are welcome.

I think if we do scope well enough, we'll have something similar to Rust that we can flesh out more in the library.
Jul 09 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jul 10, 2013 at 01:01:01AM +0200, Adam D. Ruppe wrote:
 On Tuesday, 9 July 2013 at 22:40:31 UTC, bearophile wrote:
- A less allocating Phobos to produce a bit less garbage;

Yes, and options to pass output ranges to more functions too, instead of always returning gc allocated things.

+1. I think thus far we've only truly explored the input side of the range concept. We should do more with output ranges -- they are a lot more flexible than returning values (be that gc-allocated or not). You can bypass a lot of unnecessary buffering by doing that (think to!string and std.format, for example), which helps performance and reduces garbage for the GC. We should develop some good idioms for chaining / nesting output ranges, so that what is today a bunch of string appends, say, could be a UFCS-empowered chain of output ranges that basically writes the output directly to its destination instead of sitting around in strings or buffers, etc.. T -- Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG
Jul 09 2013
prev sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 9 July 2013 at 22:40:31 UTC, bearophile wrote:
 Walter Bright:

 It isn't off-topic at all. It's very relevant to D.

 I also agree with what he says about GC.

There's a long way from recognizing those problems, to having good enough solutions in D. Some possible attack strategies for D are: - A less allocating Phobos to produce a bit less garbage; - A more precise GC to avoid some memory leaks; - Perhaps an annotation to disallow heap allocations in a function or a piece of code; - Some good way to allocate variable length arrays on the stack (http://d.puremagic.com/issues/show_bug.cgi?id=9832 ) (so some arrays produce no garbage); - The implementation of "scope" maybe helps a bit; - Multiple alias this opens some opportunities to use structs in more places, avoiding some heap allocations. - Currently Phobos Scoped/emplace are not very good. Is that enough? Rust language designers seem to think that's not enough. Opinions are welcome. Bye, bearophile

I agree. Additionally I think it might be worthwhile to also have a look at other system languages with GC. The ones I had some contact with: - Oberon, Oberon-2, Oberon-7, Component Pascal, Active Oberon (Basically Oberon Family) - Modula-3 - Sing# (C# 2.0 with extensions for Singularity) The main problem is that they failed to enter the industry, as such for some of them (Modula-3) it is very hard to get proper information nowadays. In the Oberon family for example, GC only works when New or string manipulations are involved. The rest of memory is the usual static allocation at compile time or VLA arrays as bearophile mentions. Both Modula-3 and Active Oberon allow for the declaration of untraced pointers, which boils down to manual memory management with compiler help. In all of them it is also possible to circumvent the type system and do manual memory management via the SYSTEM package/unsafe constructs. Although it is seen as something to do with expert hat on and big red alert. :) -- Paulo
Jul 10 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 9 July 2013 at 19:44:26 UTC, Joakim wrote:
 On Tuesday, 9 July 2013 at 19:27:22 UTC, QAston wrote:
 On Tuesday, 9 July 2013 at 18:12:24 UTC, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

 --
 Paulo

I think that the garbage collection part of the atricle is very relevant to the usage of D on mobile.

If Intel ever makes a comeback on mobile- Samsung is putting x86 chips in their next generation of Galaxy Tab tablets, so it's possible- it might not take much effort to port D/ldc to Android/x86 though, so maybe it will be relevant someday. Good article, with some good data.

There are embedded devices that can be targeted with GC enabled systems programming languages. For example Oberon-7 for ARM Cortex-M3 and NXP LPC2000 microcontrollers: http://www.astrobe.com/default.htm What happens is that you can also control memory outside the GC if really needed, via the usual stack/global memory allocation and the SYSTEM package. -- Paulo
Jul 10 2013
prev sibling next sibling parent reply "thedeemon" <dlang thedeemon.com> writes:
On Tuesday, 9 July 2013 at 18:12:24 UTC, Paulo Pinto wrote:
 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

The chart of different kinds of GC is very worth looking there. It shows how much generational GCs are faster than simple scan-whole-heap ones. Without generational GC D will remain rather slow, as each collection cycle with a 1 GB heap of small objects now takes ~5 seconds. And after you've gained 1 GB of data in the heap, each 8 MB of new allocated data cost you another 5 seconds of full scan/collect. No mature GCed languages behave that bad.
Jul 10 2013
parent Paulo Pinto <pjmlp progtools.org> writes:
Am 11.07.2013 21:58, schrieb bearophile:
 thedeemon:

 HotSpot and, I suppose, other mature JVMs provide generational GCs, so
 you won't get 5 seconds delay after each 8 MB of allocated data there.

They need a certain amount of seconds for each GB of heap memory (assuming an average amount of pointers in that GB of data structures. Java data structures contain many pointers, unlike C# and D). The G1 garbage collector helps spread this computation. Bye, bearophile

I just noticed there is a presentation about G1 at InfoQ, http://www.infoq.com/presentations/java-g1?utm_source=infoq&utm_medium=videos_homepage&utm_campaign=videos_row1 I am yet to see it, so cannot attest if it is worthwhile to watch. -- Paulo
Jul 11 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
thedeemon:

 No mature GCed languages behave that bad.

I think there is one JavaVM that manages to avoid part of the problem you explain (and maybe it needs special kernel support). I think all other JavaVMs suffer it, more or less. Bye, bearophile
Jul 10 2013
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2013-07-09 18:12:25 +0000, Paulo Pinto <pjmlp progtools.org> said:

 A bit off-topic, but well worth reading,
 
 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

What I'm retaining from this is that garbage collectors are wasteful. They're viable if you have a lot of RAM to spare. They cause noticeable hiccups at unpredictable times unless you have a battery-hungry overpowered CPU that makes pauses impossible to notice. And while those pauses are not that bad for non-realtime apps, all iOS apps are considered realtime by Apple because you don't want hiccups messing smooth scrolling and animations. Also, non-deterministic deallocation makes it hard for an app to fit within a fixed memory limit. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Jul 10 2013
next sibling parent "renoX" <renozyx gmail.com> writes:
On Wednesday, 10 July 2013 at 13:33:15 UTC, Michel Fortin wrote:
 On 2013-07-09 18:12:25 +0000, Paulo Pinto <pjmlp progtools.org> 
 said:

 A bit off-topic, but well worth reading,
 
 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

What I'm retaining from this is that garbage collectors are wasteful. They're viable if you have a lot of RAM to spare.

But I think that this is why newer Android phone have a lot of RAM: a GS4 has 2GB of RAM.
 They cause noticeable hiccups at unpredictable times unless you 
 have a battery-hungry overpowered CPU that makes pauses 
 impossible to notice.

Yes and "pauseless GCs" have even higher CPU usage.. renoX
 And while those pauses are not that bad for non-realtime apps, 
 all iOS apps are considered realtime by Apple because you don't 
 want hiccups messing smooth scrolling and animations.

 Also, non-deterministic deallocation makes it hard for an app 
 to fit within a fixed memory limit.

Jul 11 2013
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 11 July 2013 at 08:53:07 UTC, renoX wrote:
 They cause noticeable hiccups at unpredictable times unless 
 you have a battery-hungry overpowered CPU that makes pauses 
 impossible to notice.

Yes and "pauseless GCs" have even higher CPU usage..

It is more about floating garbage than CPU usage. But yes, they have drawbacks.
Jul 11 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 10 July 2013 at 13:30:26 UTC, bearophile wrote:
 thedeemon:

 No mature GCed languages behave that bad.

I think there is one JavaVM that manages to avoid part of the problem you explain (and maybe it needs special kernel support). I think all other JavaVMs suffer it, more or less. Bye, bearophile

Yes, the C4 garbage collector in the Azul JVM http://www.azulsystems.com/products/zing/whatisit http://www.azulsystems.com/products/zing/c4-java-garbage-collector-wp They used to have special hardware, but now they use standard kernels. RedHat is also planning to have a go at it for the OpenJDK, http://rkennke.wordpress.com/2013/06/10/shenandoah-a-pauseless-gc-for-openjdk/ -- Paulo
Jul 10 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Paulo Pinto:

 They used to have special hardware, but now they use standard 
 kernels.

This is very good. (Maybe in the meantime someone has folded their needs inside the standard kernel). Bye, bearophile
Jul 10 2013
prev sibling next sibling parent "qznc" <qznc web.de> writes:
On Tuesday, 9 July 2013 at 20:39:30 UTC, Brian Rogoff wrote:
 PS: That silhouette of the SR-71 at the point allocators are 
 mentioned sets a high bar for the design!

I did not like that analogy at all. Have you seen the user interface of an SR-71? http://www.likecool.com/Gear/Pic/Spy%20Plane%20SR71%20Blackbird%20Cockpit/big/Spy-Plane-SR71-Blackbird-Cockpit.jpg
Jul 10 2013
prev sibling next sibling parent "Oleg Kuporosov" <Oleg.Kuporosov gmail.com> writes:
  Thanks for the link.  In my experience, mobile networking is
 slow in general.  When I run Speedtest on my phone vs. a laptop 
 sitting right next to it, the phone has a fraction of the 
 bandwidth of the laptop.  So there's even more at issue than 
 raw JS performance.

Sean is right, this is only one side of problem and very often not a major, another side is mobile network bandwidth and infrastructure load. It is usual for corporate offices/cities (sharing), weak signal conditions and movement you will have quite limited bandwidth even for LTE/3G cases.
Jul 10 2013
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Jul 9, 2013, at 11:12 AM, Paulo Pinto <pjmlp progtools.org> wrote:

 A bit off-topic, but well worth reading,
=20
 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

Oh, regarding ObjC (and I'll qualify this by saying that I'm not an ObjC = programmer). My understanding is that ObjC was originally reference = counted (ARC =3D Automatic Reference Counting). Apple then introduced a = mark & sweep GC for ObjC and then in the following release deprecated it = and switched back to ARC for reasons I don't recall. However, reference = counting *is* garbage collection, despite what that slide suggests. It = just behaves in a manner that tends to spread the load out more evenly = across the application lifetime.=
Jul 10 2013
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-07-10 19:25, Sean Kelly wrote:
 On Jul 9, 2013, at 11:12 AM, Paulo Pinto <pjmlp progtools.org> wrote:

 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

Oh, regarding ObjC (and I'll qualify this by saying that I'm not an ObjC programmer). My understanding is that ObjC was originally reference counted (ARC = Automatic Reference Counting). Apple then introduced a mark & sweep GC for ObjC and then in the following release deprecated it and switched back to ARC for reasons I don't recall. However, reference counting *is* garbage collection, despite what that slide suggests. It just behaves in a manner that tends to spread the load out more evenly across the application lifetime.

Objective-C originally used manual reference counting. Then Apple created a GC (never available on iOS). Then they implemented ARC in Clang. And now they have deprecated the GC and one should use ARC. -- /Jacob Carlborg
Jul 10 2013
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 10.07.2013 20:32, schrieb Jacob Carlborg:
 On 2013-07-10 19:25, Sean Kelly wrote:
 On Jul 9, 2013, at 11:12 AM, Paulo Pinto <pjmlp progtools.org> wrote:

 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

Oh, regarding ObjC (and I'll qualify this by saying that I'm not an ObjC programmer). My understanding is that ObjC was originally reference counted (ARC = Automatic Reference Counting). Apple then introduced a mark & sweep GC for ObjC and then in the following release deprecated it and switched back to ARC for reasons I don't recall. However, reference counting *is* garbage collection, despite what that slide suggests. It just behaves in a manner that tends to spread the load out more evenly across the application lifetime.

Objective-C originally used manual reference counting. Then Apple created a GC (never available on iOS). Then they implemented ARC in Clang. And now they have deprecated the GC and one should use ARC.

What sometimes goes missed between the lines is that one of the decisions to go ARC instead of GC, is because the Objective-C GC never worked properly and ARC offers a better fit for the current state of Objective-C world. First of all, GC was an opt-in and very few libraries supported it. Then we have the typical issues with a conservative GC in a C based language, which lead to tons of issues if one looks into developer forums. This is, of course, not good PR to explain the real technical reason why they decided to go ARC instead, was that the GC implementation was a failure. -- Paulo
Jul 10 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-07-10 20:43, Paulo Pinto wrote:

 What sometimes goes missed between the lines is that one of the
 decisions to go ARC instead of GC, is because the Objective-C GC never
 worked properly and ARC offers a better fit for the current state of
 Objective-C world.

 First of all, GC was an opt-in and very few libraries supported it.

Wasn't it possible to use the GC with all libraries but not the other way around?
 Then we have the typical issues with a conservative GC in a C based
 language, which lead to tons of issues if one looks into developer forums.

The GC only worked for the Objective-C part. -- /Jacob Carlborg
Jul 11 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-07-11 10:46, Paulo Pinto wrote:

 This is way there was such a joy when ARC was announced, not because GC
 per se is bad.

Guess I didn't know about all these flaws in the GC. -- /Jacob Carlborg
Jul 11 2013
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 14.07.2013 09:56, schrieb SomeDude:
 On Wednesday, 10 July 2013 at 17:25:31 UTC, Sean Kelly wrote:
 On Jul 9, 2013, at 11:12 AM, Paulo Pinto <pjmlp progtools.org> wrote:

 A bit off-topic, but well worth reading,

 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

Oh, regarding ObjC (and I'll qualify this by saying that I'm not an ObjC programmer). My understanding is that ObjC was originally reference counted (ARC = Automatic Reference Counting). Apple then introduced a mark & sweep GC for ObjC and then in the following release deprecated it and switched back to ARC for reasons I don't recall. However, reference counting *is* garbage collection, despite what that slide suggests. It just behaves in a manner that tends to spread the load out more evenly across the application lifetime.

C++ has gone te ARC route as well with shared_ptr. I find the scoped_ptr/shared_ptr combination quite convenient and quite safe overall.

The main problem is that C++ compilers are not shared_ptr aware. So while systems like Objective-C ARC and ParaSail do minimize increment/decrement operations, by removed superfluous pairs of operations, in C++'s case you get some performance lost because of those operations everywhere when pointers ownership changes. -- Paulo
Jul 14 2013
prev sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 16.07.2013 16:53, schrieb Jonathan Dunlap:
 So before Java was born, I was already convinced a GC was


Can you explain (simply) how D's GC works and what improvements are on the roadmap for it? As it sits, I'm a little hesitant to use it for large applications because of its overhead I've heard about on other threads. http://3d.benjamin-thaut.de/?p=20 "the manual memory managed version runs at 5 ms which is 200 FPS and thus nearly 3 times as fast as the GC collected version."

I am just a D follower, so Benjamin and other guys are better indicated to speak about it. Having said this, currently D's GC is a stop the world collector with a basic implementation, when compared with Java and .NET offerings. Last year some improvements were made, specially associating some type information via the generation of TypeInfo instances. http://dlang.org/phobos/object.html#.TypeInfo At D2013 there were some presentations about how to improve the GC. A Precise Garbage Collector for D http://www.youtube.com/watch?v=LQY1m_eT37c Concurrent Garbage Collection for D http://www.youtube.com/watch?v=1MF5bcmvJ0o Now, it all depends on the type of application you are planning to use it. In many cases, even with the current GC implementation, it is possible to achieve good performance if you code in a GC friendly way. You can also make use of library types for reference counting, http://dlang.org/phobos/std_typecons.html#.RefCounted And if you really, really need, also manual memory management by calling the C functions and letting the GC know not to track that memory. http://dlang.org/phobos/core_memory.html#.GC.BlkAttr.NO_SCAN There are some discussions going on about adding reference counting at the compiler level, similar to Rust/ParaSail. Additionally, you can count that the GC will eventually be improved, just that these things take time. -- Paulo
Jul 16 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Sean Kelly:

 However, reference counting *is* garbage collection,

Of course, it's explained well here: http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf Bye, bearophile
Jul 10 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 10 Jul 2013 10:47:16 -0700
Walter Bright <newshound2 digitalmars.com> wrote:

 On 7/10/2013 7:52 AM, qznc wrote:
 On Tuesday, 9 July 2013 at 20:39:30 UTC, Brian Rogoff wrote:
 PS: That silhouette of the SR-71 at the point allocators are
 mentioned sets a high bar for the design!

I did not like that analogy at all. Have you seen the user interface of an SR-71? http://www.likecool.com/Gear/Pic/Spy%20Plane%20SR71%20Blackbird%20Cockpit/big/Spy-Plane-SR71-Blackbird-Cockpit.jpg

I always love the utilitarian design of military cockpits. No logos, no fake wood grain paneling, no styling, no color scheme, no cupholder. All business.

No cupholders is all fine and good until the sky gets clogged with ariel traffic and you have to put down your coffee. *Then* you'll regret not opting for the deluxe package!
Jul 10 2013
prev sibling next sibling parent "Jonathan A Dunlap" <jdunlap outlook.com> writes:
My 2cents: for D to be successful for the game development 
community, it has to be possible to mostly sidestep the GC or opt 
into a minimal one like ARC. Granted, this is a bit premature 
considering that OpenGL library support is still in alpha quality.
Jul 10 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 10 July 2013 at 21:05:32 UTC, Jonathan A Dunlap 
wrote:
 My 2cents: for D to be successful for the game development 
 community, it has to be possible to mostly sidestep the GC or 
 opt into a minimal one like ARC. Granted, this is a bit 
 premature considering that OpenGL library support is still in 
 alpha quality.

I've noticed that when you reply to a thread, you reply to the most recent response, irrelevant of context. This is a bit confusing to those of us who view the group in a threaded layout. For example here Walter is talking about not having inflight audio on a discontinued? military recon plane and then there's you listed as a replying to him, talking about ARC and GCs!
Jul 10 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jul 10, 2013 at 11:32:29PM +0200, John Colvin wrote:
 On Wednesday, 10 July 2013 at 21:05:32 UTC, Jonathan A Dunlap wrote:
My 2cents: for D to be successful for the game development
community, it has to be possible to mostly sidestep the GC or opt
into a minimal one like ARC. Granted, this is a bit premature
considering that OpenGL library support is still in alpha quality.

I've noticed that when you reply to a thread, you reply to the most recent response, irrelevant of context. This is a bit confusing to those of us who view the group in a threaded layout. For example here Walter is talking about not having inflight audio on a discontinued? military recon plane and then there's you listed as a replying to him, talking about ARC and GCs!

There's a long-standing bug in the mailing list interface to the forum that rewrites message IDs when it shouldn't, thus breaking threads. This problem has been irking me for a long time now, but it seems nobody is interested to fix it. :-( T -- There are four kinds of lies: lies, damn lies, and statistics.
Jul 10 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 10 July 2013 at 22:22:41 UTC, H. S. Teoh wrote:
 On Wed, Jul 10, 2013 at 11:32:29PM +0200, John Colvin wrote:
 On Wednesday, 10 July 2013 at 21:05:32 UTC, Jonathan A Dunlap 
 wrote:
My 2cents: for D to be successful for the game development
community, it has to be possible to mostly sidestep the GC or 
opt
into a minimal one like ARC. Granted, this is a bit premature
considering that OpenGL library support is still in alpha 
quality.

I've noticed that when you reply to a thread, you reply to the most recent response, irrelevant of context. This is a bit confusing to those of us who view the group in a threaded layout. For example here Walter is talking about not having inflight audio on a discontinued? military recon plane and then there's you listed as a replying to him, talking about ARC and GCs!

There's a long-standing bug in the mailing list interface to the forum that rewrites message IDs when it shouldn't, thus breaking threads. This problem has been irking me for a long time now, but it seems nobody is interested to fix it. :-( T

There is, but I don't think this was one of those cases. That manifests as an entirely new thread in the forum interface.
Jul 10 2013
prev sibling next sibling parent "Brian Rogoff" <brogoff gmail.com> writes:
On Wednesday, 10 July 2013 at 14:01:51 UTC, Paulo Pinto wrote:
 On Wednesday, 10 July 2013 at 13:30:26 UTC, bearophile wrote:
 thedeemon:

 No mature GCed languages behave that bad.

I think there is one JavaVM that manages to avoid part of the problem you explain (and maybe it needs special kernel support). I think all other JavaVMs suffer it, more or less. Bye, bearophile

Yes, the C4 garbage collector in the Azul JVM http://www.azulsystems.com/products/zing/whatisit http://www.azulsystems.com/products/zing/c4-java-garbage-collector-wp They used to have special hardware, but now they use standard kernels. RedHat is also planning to have a go at it for the OpenJDK, http://rkennke.wordpress.com/2013/06/10/shenandoah-a-pauseless-gc-for-openjdk/ -- Paulo

Interesting. There was some discussion of adding a 'pauseless' GC to Go as well, here https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2BU and in that discussion Gil Tene, one of the authors of Azul, opines "Starting with a precise and generational stop-the-world implementation that is robust is a must, and a good launching pad towards a concurrent compacting collector (which is what a "pauseless" collector must be in server-scale environments). Each of those qualities (precise, generational) slaps serious requirements on the execution environment and on the compilers (whether they are pre-compilers or JIT compilers doesn't matter): precise collectors require full identification of all references at code safepoints, and also require a robust safepoint mechanism. Code safepoints must be frequent (usually devolve to being at every method entry and loop back edge), and support in non-compiler-generated code (e.g. library and runtime code written in C/C++) usually involves some form of reference handle support around safepoints. Generational collectors require a write barrier (a ref-store barrier to be precise) with full coverage for any heap reference store operations (in compiler-generated code and in all runtime code). It is my opinion that investing in the above capabilities early in the process (i.e. start now!) is critical. Environments that skip this step for too long and try to live with conservative GC in order to avoid putting in the required work for supporting precise collectors in the compilers and runtime and libraries find themselves heavily invested in compiler code that would need to be completely re-vamped to move forward. ..." Sounds like that precise GC talk at DConf was quite timely. Let's hope that prediction about being too heavily invested in conservative GC dependencies isn't too true! -- Brian
Jul 10 2013
prev sibling next sibling parent "Jonathan Dunlap" <jdunlap outlook.com> writes:
 There is, but I don't think this was one of those cases. That 
 manifests as an entirely new thread in the forum interface.

Your correct John, totally my bad. I'll be more vigilant in which post I reply to as I didn't realize that anyone really read into the reply's source. I'm critical about D's GC as I am super passionate about being able to one day use D for professional game development. =)
Jul 10 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 11 July 2013 at 07:34:48 UTC, Jacob Carlborg wrote:
 On 2013-07-10 20:43, Paulo Pinto wrote:

 What sometimes goes missed between the lines is that one of the
 decisions to go ARC instead of GC, is because the Objective-C 
 GC never
 worked properly and ARC offers a better fit for the current 
 state of
 Objective-C world.

 First of all, GC was an opt-in and very few libraries 
 supported it.

Wasn't it possible to use the GC with all libraries but not the other way around?

No, it depended how you compiled the code. https://developer.apple.com/legacy/library/#documentation/Cocoa/Conceptual/GarbageCollection/Articles/gcEssentials.html#//apple_ref/doc/uid/TP40002452-SW1 Using -fobjc-gc-only meant your library could only work with GC enabled applications. Or another quote from the documentation: "Not all frameworks and technologies support garbage collection; for example, iCloud is not supported in applications that use garbage collection."
 Then we have the typical issues with a conservative GC in a C 
 based
 language, which lead to tons of issues if one looks into 
 developer forums.

The GC only worked for the Objective-C part.

True, but you were forced to do manual hacks like calling objc_assign_global() for write barriers in global and static variables. There is the compiler flag -Wassign-intercept to show where write barriers are being generated as a help method if everything is being generated properly. The Core Foundation needs to be used in a slight different way when interacting with GC enabled objects. With ARC all these problems do not exist, because the compiler does what the developers would be expected to do manually anyway. This is way there was such a joy when ARC was announced, not because GC per se is bad. -- Paulo
Jul 11 2013
prev sibling next sibling parent "thedeemon" <dlang thedeemon.com> writes:
On Wednesday, 10 July 2013 at 13:30:26 UTC, bearophile wrote:
 thedeemon:

 No mature GCed languages behave that bad.

I think there is one JavaVM that manages to avoid part of the problem you explain (and maybe it needs special kernel support). I think all other JavaVMs suffer it, more or less.

HotSpot and, I suppose, other mature JVMs provide generational GCs, so you won't get 5 seconds delay after each 8 MB of allocated data there.
Jul 11 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
thedeemon:

 HotSpot and, I suppose, other mature JVMs provide generational 
 GCs, so you won't get 5 seconds delay after each 8 MB of 
 allocated data there.

They need a certain amount of seconds for each GB of heap memory (assuming an average amount of pointers in that GB of data structures. Java data structures contain many pointers, unlike C# and D). The G1 garbage collector helps spread this computation. Bye, bearophile
Jul 11 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Wednesday, 10 July 2013 at 21:05:32 UTC, Jonathan A Dunlap 
wrote:
 My 2cents: for D to be successful for the game development 
 community, it has to be possible to mostly sidestep the GC or 
 opt into a minimal one like ARC. Granted, this is a bit 
 premature considering that OpenGL library support is still in 
 alpha quality.

What do you mean OpenGL library support is still in alpha quality? There's several high quality ports of the OpenGL headers and they work like a charm. I've had no problems. Using the GC is fine as long as you aren't allocating every frame. I use GC allocation for all my big systems and anything that isn't created or destroyed during gameplay. It's very convenient for that kind of thing and you can just let the GC run at the end of a level or something.
Jul 12 2013
prev sibling next sibling parent "Kagamin" <spam here.lot> writes:
We at work have problems with performance and memory consumption 
on x86 hardware. The application uses lots of data from a big 
database, but it's country-scale government organization, so 
clients have only XP-class hardware. No unlimited RAM, no 
unlimited GHz, no unlimited cores.
Jul 13 2013
prev sibling next sibling parent "SomeDude" <lovelydear mailmetrash.com> writes:
On Wednesday, 10 July 2013 at 17:25:31 UTC, Sean Kelly wrote:
 On Jul 9, 2013, at 11:12 AM, Paulo Pinto <pjmlp progtools.org> 
 wrote:

 A bit off-topic, but well worth reading,
 
 http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/

Oh, regarding ObjC (and I'll qualify this by saying that I'm not an ObjC programmer). My understanding is that ObjC was originally reference counted (ARC = Automatic Reference Counting). Apple then introduced a mark & sweep GC for ObjC and then in the following release deprecated it and switched back to ARC for reasons I don't recall. However, reference counting *is* garbage collection, despite what that slide suggests. It just behaves in a manner that tends to spread the load out more evenly across the application lifetime.

C++ has gone te ARC route as well with shared_ptr. I find the scoped_ptr/shared_ptr combination quite convenient and quite safe overall.
Jul 14 2013
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Jul 14, 2013, at 12:56 AM, SomeDude <lovelydear mailmetrash.com> =
wrote:
=20
 C++ has gone te ARC route as well with shared_ptr. I find the =

overall. The thing that finally pushed me towards D was one day when I was = looking at my C++ code and I realized just how much effort I'd put into = defining data ownership rules. And while tools like shared_ptr may = automate the reference counting portion of the task in that case, the = pointer type still needs to be defined somewhere and honored by all = users of that API. And shared_ptr isn't even terribly efficient by = default because it has to assume sharing across threads, so you're stuck = with memory synchronization techniques being employed every time a = shared_ptr is copied. Don't get me wrong, I think shared_ptr is a = wonderful thing, but to be really competitive it would have to be truly = automatic and have its behavior informed by a type label like "shared". Sean=
Jul 15 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 16 July 2013 at 00:09:26 UTC, Sean Kelly wrote:
 On Jul 14, 2013, at 12:56 AM, SomeDude 
 <lovelydear mailmetrash.com> wrote:
 
 C++ has gone te ARC route as well with shared_ptr. I find the 
 scoped_ptr/shared_ptr combination quite convenient and quite 
 safe overall.

The thing that finally pushed me towards D was one day when I was looking at my C++ code and I realized just how much effort I'd put into defining data ownership rules. And while tools like shared_ptr may automate the reference counting portion of the task in that case, the pointer type still needs to be defined somewhere and honored by all users of that API. And shared_ptr isn't even terribly efficient by default because it has to assume sharing across threads, so you're stuck with memory synchronization techniques being employed every time a shared_ptr is copied. Don't get me wrong, I think shared_ptr is a wonderful thing, but to be really competitive it would have to be truly automatic and have its behavior informed by a type label like "shared". Sean

For me it was an experience with Native Oberon in the mid-90's. A desktop operating system coded in a systems programming language with GC (Oberon) offering a Smalltalk like experience. Sadly only people at Zurich's technical university, or followers from Wirth's work, are aware of it and its successor Blue Bottle (A2). Some years later I also had access to information about Modula-3, which provided a few ideas that made their way into C#. Then all the work I did with Smalltalk and Camllight while at the university. So before Java was born, I was already convinced a GC was possible for systems programming. The problem is that the way Java and .NET have been sold in the industry, most people without compiler development background, tend to assume GC == VM. -- Paulo
Jul 15 2013
prev sibling next sibling parent "Jonathan Dunlap" <jdunlap outlook.com> writes:
 So before Java was born, I was already convinced a GC was


Can you explain (simply) how D's GC works and what improvements are on the roadmap for it? As it sits, I'm a little hesitant to use it for large applications because of its overhead I've heard about on other threads. http://3d.benjamin-thaut.de/?p=20 "the manual memory managed version runs at 5 ms which is 200 FPS and thus nearly 3 times as fast as the GC collected version."
Jul 16 2013
prev sibling next sibling parent "Jonathan A Dunlap" <jdunlap outlook.com> writes:
 You can also make use of library types for reference counting,

 http://dlang.org/phobos/std_typecons.html#.RefCounted

 And if you really, really need, also manual memory management 
 by calling the C functions and letting the GC know not to track 
 that memory.

 http://dlang.org/phobos/core_memory.html#.GC.BlkAttr.NO_SCAN

Fascinating! I assume the Phobos RefCounted then avoids using the GC by utilizing GC.malloc (NO_SCAN) behind the scenes? Has anyone benchmarked an application using D's GC versus using RefCounted for critical paths?
Jul 16 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 16 July 2013 at 18:21:05 UTC, Jonathan A Dunlap wrote:
 You can also make use of library types for reference counting,

 http://dlang.org/phobos/std_typecons.html#.RefCounted

 And if you really, really need, also manual memory management 
 by calling the C functions and letting the GC know not to 
 track that memory.

 http://dlang.org/phobos/core_memory.html#.GC.BlkAttr.NO_SCAN

Fascinating! I assume the Phobos RefCounted then avoids using the GC by utilizing GC.malloc (NO_SCAN) behind the scenes?

Nope, just plain old malloc and free from core.stdc.stdlib
Jul 16 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 16 July 2013 at 14:53:05 UTC, Jonathan Dunlap wrote:
 So before Java was born, I was already convinced a GC was


Can you explain (simply) how D's GC works and what improvements are on the roadmap for it? As it sits, I'm a little hesitant to use it for large applications because of its overhead I've heard about on other threads.

Good news, you can actually free bunch of memory yourself in D if you need to via GC.free to reduce the GC pressure.
 http://3d.benjamin-thaut.de/?p=20
 "the manual memory managed version runs at 5 ms which is 200 
 FPS and thus nearly 3 times as fast as the GC collected 
 version."

This highlight the poor GC performances, but also the fact that way too much allocation are done when not required, which increase the GC pressure. This is something considered seriously and the amount is decreasing with each version of D.
Jul 16 2013
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 16 July 2013 at 18:21:05 UTC, Jonathan A Dunlap wrote:
 You can also make use of library types for reference counting,

 http://dlang.org/phobos/std_typecons.html#.RefCounted

 And if you really, really need, also manual memory management 
 by calling the C functions and letting the GC know not to 
 track that memory.

 http://dlang.org/phobos/core_memory.html#.GC.BlkAttr.NO_SCAN

Fascinating! I assume the Phobos RefCounted then avoids using the GC by utilizing GC.malloc (NO_SCAN) behind the scenes? Has anyone benchmarked an application using D's GC versus using RefCounted for critical paths?

I highly advice not to use the NO_SCAN unless you really know what you are doing. Not that the GC will trigger only when enough memory is allocated, so if you deallocate most memory manually, the GC won't trigger often.
Jul 16 2013