www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Smart pointers instead of GC?

reply "Sven Over" <dlang svenover.de> writes:
Hello dear D community.

This is my very first post here. I've been intrigued with D for 
quite a while, and I would love to have a project to work on in 
D. I'm working as a C++ software engineer, and should be 
moderately up-to-date with best practices of C++ programming.

I've read a good part of "The D Programming Language" and must 
say that I *love* almost everything about D. Metaprogramming, 
which quickly gets very convoluted in C++ (to say the least), is 
just beautiful in D. The approach to multi-threaded programming, 
and the adoption of concepts of functional programming, are all 
pure gold.

The one thing that I am unsure about in D is garbage collection. 
I guess this is some sort of FAQ and may have been answered 
dozens of times. I apologise in advance.

The main reason I dislike garbage collection for is that 
destructors of objects on the heap are not called at defined 
times. Techniques like RAII 
(http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization) 
have become an often used pattern in my programming, and works 
nicely for short lived objects (on the stack), but by means of 
smart pointers (i.e. reference counting) also for long lived 
objects (on the heap).

Also, garbage collection often tends to increase the memory 
footprint of your software. You may be able to avoid this with 
some expert knowledge about the garbage collector, but this of 
course invalidates one of the main arguments in favour of GC, 
which is less complexity. I'd say that if you want to write 
memory- and CPU-efficient code, then you'll need a certain amount 
of understanding of your memory management. Be it your manual 
memory management, or the inner workings of your garbage 
collector.

To cut things short, even after giving it a lot of thought I 
still feel uncomfortable about garbage collection. I'm wondering 
whether it is viable to use the same smart pointer techniques as 
in C++: is there an implementation already of such smart 
pointers? Can I switch off GC in my D programs altogether? How 
much does the standard library rely on GC? I.e. would I have to 
write a alternative standard library to do this?

There is one blog post I found interesting to read in this 
context, which also underlines my worries: 
http://3d.benjamin-thaut.de/?p=20
Dec 25 2012
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 25 December 2012 at 14:13:21 UTC, Sven Over wrote:
 To cut things short, even after giving it a lot of thought I 
 still feel uncomfortable about garbage collection. I'm 
 wondering whether it is viable to use the same smart pointer 
 techniques as in C++: is there an implementation already of 
 such smart pointers? Can I switch off GC in my D programs 
 altogether? How much does the standard library rely on GC? I.e. 
 would I have to write a alternative standard library to do this?

std.typecons.RefCounted!T core.memory.GC.disable(); Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.
Dec 25 2012
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 01.02.2014 06:29, schrieb Manu:
 On 26 December 2012 00:48, Sven Over <dlang svenover.de
 <mailto:dlang svenover.de>> wrote:

         std.typecons.RefCounted!T

         core.memory.GC.disable();


     Wow. That was easy.

     I see, D's claim of being a multi-paradigm language is not false.


 It's not a realistic suggestion. Everything you want to link uses the
 GC, and the language its self also uses the GC. Unless you write
 software in complete isolation and forego many valuable features, it's
 not a solution.


         Phobos does rely on the GC to some extent. Most algorithms and
         ranges do not though.


     Running (library) code that was written with GC in mind and turning
     GC off doesn't sound ideal.

     But maybe this allows me to familiarise myself more with D. Who
     knows, maybe I can learn to stop worrying and love garbage collection.

     Thanks for your help!


 I've been trying to learn to love the GC for as long as I've been around
 here. I really wanted to break that mental barrier, but it hasn't happened.
 In fact, I am more than ever convinced that the GC won't do. My current
 #1 wishlist item for D is the ability to use a reference counted
 collector in place of the built-in GC.
 You're not alone :)

 I write realtime and memory-constrained software (console games), and
 for me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable
 memory footprint of the application. You can't make any guarantees or
 predictions about the GC, which is fundamentally incompatible with
 realtime software.

Meanwhile Unity and similar engines are becoming widespread, with C++ being pushed all the way to the bottom on the stack. At least from what I hear in the gaming communities I hop around. What is your experience there? -- Paulo
Feb 01 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 2/1/2014 3:20 AM, Paulo Pinto wrote:
 Meanwhile Unity and similar engines are becoming widespread, with C++
 being pushed all the way to the bottom on the stack.

 At least from what I hear in the gaming communities I hop around.

 What is your experience there?

It's not uncommon for Unity-engine games to drop down to C++ to optimize bottlenecks. Somewhat of a pain since you have to deal with the CLR <-> unmanaged marshalling mess. (I do really wish Unity would expose an optional C/C++ API so you wouldn't need a pointless CLR layer sandwiched between the C++-based engine and any non-CLR game code...) Keep in mind too that while Unity is certainly powerful, the games written with it aren't usually the ultra-AAA titles (like Battlefield, Crysis or Last of Us, for example) where a major part of the business is "wow everyone with never-before-seen technical achievements or risk failing to recoup the gigantic development costs". Come to think of it, I wonder what language Frostbite 3 uses for game-level "scripting" code. From what I've heard about them, Frostbite 3 and Unreal Engine 4 both sound like they have some notable similarities with Unity3D (from the standpoint of the user-experience for game developers), although AIUI Unreal Engine 4 still uses C++ for game code (unless the engine user wants to to do lua or something on their own). I imagine Frostbite's probably the same, C++, but I haven't actually heard anything.
Feb 01 2014
parent Paulo Pinto <pjmlp progtools.org> writes:
Am 01.02.2014 13:35, schrieb develop32:
 On Saturday, 1 February 2014 at 12:29:10 UTC, Nick Sabalausky wrote:
 Come to think of it, I wonder what language Frostbite 3 uses for
 game-level "scripting" code. From what I've heard about them,
 Frostbite 3 and Unreal Engine 4 both sound like they have some notable
 similarities with Unity3D (from the standpoint of the user-experience
 for game developers), although AIUI Unreal Engine 4 still uses C++ for
 game code (unless the engine user wants to to do lua or something on
 their own). I imagine Frostbite's probably the same, C++, but I
 haven't actually heard anything.

Frostbite has an option for both Lua and C++. C++ is the preferred one. Unreal Engine went from Unrealscript to C++ and everyone applauded that.

Do you have any experience with Project Anarchy? -- Paulo
Feb 01 2014
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 01.02.2014 08:35, schrieb Manu:
 On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com
 ...

 No serious console game I'm aware of has ever been written in a language
 with a GC. Casual games, or games that don't attempt to raise the bar
 may get away with it, but that's not the industry I work in.

Not sure how much serious you consider The Witcher 2 for the XBox 360, they used a GC in their engine. http://www.makinggames.de/index.php/magazin/2155_porting_the_witcher_2_on_xbox_360 Yes, they faced some issues with it, but it wasn't thrown away, rather optimized how it was being used. -- Paulo
Feb 01 2014
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-01 09:27:18 +0000, "Adam Wilson" <flyboynw gmail.com> said:

 That's kind of my point. You're asking for massive changes throughout 
 the  entire compiler to support what is becoming more of an edge case, 
 not less  of one. For the vast majority of use cases, a GC is the right 
 call and D  has to cater to the majority if it wants to gain any 
 significant mindshare  at all. You don't grow by increasing 
 specialization...

Keep in mind that improving the GC we have now is probably going to require pretty much the same changes in the compiler, perhaps even more drastic ones than ARC. To make it short, to implement a concurrent GC the compiler must add some code for each pointer assignment and each pointer move. To implement ARC, the compiler must add some code only to pointer assignments. The point is, unless we're satisfied with the current GC implementation, those massive changes to the compiler will need to be done anyway. From the compiler's point of view ARC or a better concurrent GC have pretty similar requirements. And given current D semantics (where stack variables are movable at will) ARC is easier to implement than a concurrent GC. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 01 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/1/14, 6:59 PM, Frank Bauer wrote:
 Just to clarify: I would like to have the following pointers. I'll start
 with the GC ones to maybe reach some consensus quicker ;)

 1. GC(T)
 Pointer to a heap allocated T, garbage collected.

 2. ARC(T)
 Reference counted pointer to a heap allocated T, compiler manages
 lifetime automatically via retain/ release.

 3. Owning(T)
 Pointer to a heap allocated T, freed when pointer goes out of scope or
 is reassigned (some special interactions with Borrowed(T) on top of that).

 4. Borrowed(T)
 Pointer to a T, not necessarily heap allocated, has no impact on memory
 allocation/ deallocation of the T whatsoever, if assigned from an
 Owning(T) prevents that owning pointer from being freed or reassigned as
 long as the borrowed pointer is in scope.

 Performance, or, should I say, meddling of the compiler and runtime in
 your code, is from worst to best.

 We could argue over which pointer concepts would be granted the honor of
 receiving their own D definition syntax (new, ~,  , &, whatever).

Whoa, this won't work without an explosion in language complexity. Andrei
Feb 01 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu wrote:
 Whoa, this won't work without an explosion in language complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei
Feb 01 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/1/14, 9:34 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 05:30:02 UTC, Andrei Alexandrescu wrote:
 On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu wrote:
 Whoa, this won't work without an explosion in language complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei

^^ Lets do it while we're young ...

Sure. While you're young typecheck this: class A { auto stuff() { ...; return this; } } Andrei
Feb 01 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/1/14, 10:00 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 05:41:53 UTC, Andrei Alexandrescu wrote:
 Sure. While you're young typecheck this:

 class A {
    auto stuff() { ...; return this; }
 }


 Andrei

It would be a Borrowed!A, I would say, no matter how an A is allocated. A GC!A, an ARC!A and an Owning!A convert equally well to a Borrowed!A. As long as the Borrowed!A is in use (i.e. scope) after stuff() returns, the A object must not be freed (compiler enforced). Would that make sense?

No. What I'm driving at is that right now the type is A, as is in a variety of other situations. It would be a whole different language that would be wholly incompatible with D. Andrei
Feb 01 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 2:53 AM, Dicebot wrote:
 On Sunday, 2 February 2014 at 07:35:21 UTC, Andrei Alexandrescu wrote:
 No.

 What I'm driving at is that right now the type is A, as is in a
 variety of other situations. It would be a whole different language
 that would be wholly incompatible with D.

What do you think about my `scope` proposal?

I think that's more likely to be made to work. Andrei
Feb 02 2014
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 02.02.2014 15:11, schrieb Adam D. Ruppe:
 On Sunday, 2 February 2014 at 11:23:56 UTC, JR wrote:
 I *seem* to remember reading here that you and Walter were
 increasingly growing to favor ARC, but I can't find the post.

Personally, I think ARC is a waste of time and none of the arguments I've seen for it are convincing. Improving the Gc implementation would be a much bigger win.

It might be that ARC will be a solution given the recent work in Objective-C, ParaSail, Rust and the misterious M#. I was even surprised to discover that Cedar actually used RC with a GC just for collecting cycles. What I don't like is using Objective-C as a poster child for ARC, because the only reason it exists, is that Apple engineers weren't able to write a GC for Objective-C that didn't core dump left and right. Apple of course sold the history in a different way. -- Paulo
Feb 02 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 3:23 AM, JR wrote:
 On Sunday, 2 February 2014 at 05:30:02 UTC, Andrei Alexandrescu wrote:
 On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu wrote:
 Whoa, this won't work without an explosion in language complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei

So, going forward, what would you say is the preferred direction to strive toward? I *seem* to remember reading here that you and Walter were increasingly growing to favor ARC, but I can't find the post. (Memory bitrot on my part is more than likely.)

I think of the following foci for the first half of 2014: 1. Add nullable and provide a -nullable compiler flag to verify it. The attribute is inferred locally and for white-box functions (lambdas, templates), and required as annotation otherwise. References not annotated with nullable are statically enforced to never be null. 2. Work on Phobos to see what can be done about avoiding unnecessary allocation. Most likely we'll need to also add a nogc flag. 3. Work on adding tracing capabilities to allocators and see how to integrate them with the language and Phobos. 4. Work on the core language and druntime to see how to seamlessly accommodate alternate GC mechanisms such as reference counting. Andrei
Feb 02 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/02/2014 05:55 PM, Andrei Alexandrescu wrote:
 I think of the following foci for the first half of 2014:

 1. Add  nullable and provide a -nullable compiler flag to verify it. The
 attribute is inferred locally and for white-box functions (lambdas,
 templates), and required as annotation otherwise. References not
 annotated with  nullable are statically enforced to never be null.

- I assume that also, references annotated with nullable are statically enforced to never be dereferenced? - how to eg. create an array of nullable references? struct Nullable!T{ nullable T t; alias t this; } Nullable!T[] foo; ? I think that would cause problems with IFTI.
Feb 02 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/02/2014 06:55 PM, Timon Gehr wrote:
 struct Nullable!T

struct Nullable(T). Non-uniformity of declaration and usage...
Feb 02 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2014 9:55 AM, Timon Gehr wrote:
 - how to eg. create an array of nullable references? struct Nullable!T{
  nullable T t; alias t this; } Nullable!T[] foo; ? I think that would cause
 problems with IFTI.

nullable being a storage class, this couldn't be done with nullable just as you cannot create an array of 'ref' types.
Feb 02 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 12:07 PM, Walter Bright wrote:
 On 2/2/2014 9:55 AM, Timon Gehr wrote:
 - how to eg. create an array of nullable references? struct Nullable!T{
  nullable T t; alias t this; } Nullable!T[] foo; ? I think that would
 cause
 problems with IFTI.

nullable being a storage class, this couldn't be done with nullable just as you cannot create an array of 'ref' types.

I don't see how this restriction would work. On the face of it it turns one cannot create an array of Objects that may or may not be null, which is a very odd restriction. Such arrays occur in a lot of places. Andrei
Feb 02 2014
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 9:55 AM, Timon Gehr wrote:
 On 02/02/2014 05:55 PM, Andrei Alexandrescu wrote:
 I think of the following foci for the first half of 2014:

 1. Add  nullable and provide a -nullable compiler flag to verify it. The
 attribute is inferred locally and for white-box functions (lambdas,
 templates), and required as annotation otherwise. References not
 annotated with  nullable are statically enforced to never be null.

- I assume that also, references annotated with nullable are statically enforced to never be dereferenced?

Correct.
 - how to eg. create an array of nullable references? struct Nullable!T{
  nullable T t; alias t this; } Nullable!T[] foo; ? I think that would
 cause problems with IFTI.

I don't know. We'd need to figure out a way. Andrei
Feb 02 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2014 8:55 AM, Andrei Alexandrescu wrote:
 1. Add  nullable and provide a -nullable compiler flag to verify it. The
 attribute is inferred locally and for white-box functions (lambdas, templates),
 and required as annotation otherwise. References not annotated with  nullable
 are statically enforced to never be null.

I want to clarify that nullable would be a storage class, NOT a type constructor. This means it will apply to the variable itself, not its type. For example: nullable T* p; auto q = p; // q is a T*, not nullable In a way, it would work much like ref, which is also a storage class and not a type constructor.
Feb 02 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 11:58 AM, Walter Bright wrote:
 On 2/2/2014 8:55 AM, Andrei Alexandrescu wrote:
 1. Add  nullable and provide a -nullable compiler flag to verify it. The
 attribute is inferred locally and for white-box functions (lambdas,
 templates),
 and required as annotation otherwise. References not annotated with
  nullable
 are statically enforced to never be null.

I want to clarify that nullable would be a storage class, NOT a type constructor. This means it will apply to the variable itself, not its type. For example: nullable T* p; auto q = p; // q is a T*, not nullable In a way, it would work much like ref, which is also a storage class and not a type constructor.

That creates problems with creating arrays of nullable objects, as Timon Gehr mentioned. We need to think this through. Andrei
Feb 02 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2014 1:16 PM, Andrei Alexandrescu wrote:
 That creates problems with creating arrays of nullable objects, as Timon Gehr
 mentioned.

 We need to think this through.

Yup.
Feb 02 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2014 9:30 AM, Namespace wrote:
 Sounds good. But why  nullable instead of C# choice of "Type?" ?

Let me rephrase that as "why use a storage class rather than a type constructor?" An excellent question. One of the big problems with a type constructor is we've already got a number of them - const, immutable, shared, inout. These and their conversions have taken quite a while for people to get comfortable with, and there are still some issues in the language arising from their interactions. Adding another type constructor will double the number of cases. I worry that D would collapse under the strain. This applies as well to type constructors for "borrowed", etc. On the other hand, D took the risky leap into the unknown by making 'ref' a storage class rather than a type constructor (in C++ it's a type constructor). This has turned out to be a satisfying win. I suspect it will be for nullable, too, though we have to work through the cases to find out for sure.
Feb 02 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 12:05 PM, Walter Bright wrote:
 On 2/2/2014 9:30 AM, Namespace wrote:
 Sounds good. But why  nullable instead of C# choice of "Type?" ?

Let me rephrase that as "why use a storage class rather than a type constructor?" An excellent question. One of the big problems with a type constructor is we've already got a number of them - const, immutable, shared, inout.

This post is confused, sorry. const, immutable, shared, inout are qualifiers (in addition to being type constructors). General type constructors don't have a particular combinatorial problem. Andrei
Feb 02 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2014 1:18 PM, Andrei Alexandrescu wrote:
 On 2/2/14, 12:05 PM, Walter Bright wrote:
 One of the big problems with a type constructor is we've already got a
 number of them - const, immutable, shared, inout.

This post is confused, sorry. const, immutable, shared, inout are qualifiers (in addition to being type constructors). General type constructors don't have a particular combinatorial problem.

Yes, I used the wrong term.
Feb 02 2014
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 02/02/2014 09:05 PM, Walter Bright wrote:
 On 2/2/2014 9:30 AM, Namespace wrote:
 Sounds good. But why  nullable instead of C# choice of "Type?" ?

Let me rephrase that as "why use a storage class rather than a type constructor?" An excellent question. One of the big problems with a type constructor is we've already got a number of them - const, immutable, shared, inout. These and their conversions have taken quite a while for people to get comfortable with, and there are still some issues in the language arising from their interactions. ...

Nullable interacts with those type constructors basically in the same way as the fixed-size array type constructor [1]. Regarding implicit conversion, I think one good form to state the implicit conversion rules in, for later procedural implementation, is the following: A converts to B ────────────────── A converts to B? A converts to B ─────────────────── A? converts to B? e converts to B? ────────────────── (flow analysis proves e non-null) e converts to B Where would be potentially bad interactions?
 Adding another type constructor will double the number of cases. I worry
 that D would collapse under the strain. This applies as well to type
 constructors for "borrowed", etc.
 ...

This sounds as if there was a structural problem in DMD. (Adding the relevant implicit conversion rules would be easy given the current state of my own front end implementation effort.) How are type qualifiers represented in the AST? How is the implicit conversion checker structured?
 On the other hand, D took the risky leap into the unknown by making
 'ref' a storage class rather than a type constructor (in C++ it's a type
 constructor). This has turned out to be a satisfying win. I suspect it
 will be for  nullable, too, though we have to work through the cases to
 find out for sure.

These cases have almost nothing in common. (e.g, 'ref' does not influence implicit conversion behaviour.)
Feb 02 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/2/2014 3:40 PM, Timon Gehr wrote:
 Nullable interacts with those type constructors basically in the same way as
the
 fixed-size array type constructor [1]. Regarding implicit conversion, I think
 one good form to state the implicit conversion rules in, for later procedural
 implementation, is the following:

   A converts to B
 ──────────────────
   A converts to B?

    A converts to B
 ───────────────────
   A? converts to B?

   e converts to B?
 ────────────────── (flow analysis proves e
non-null)
   e converts to B

 Where would be potentially bad interactions?

For starters, grep through the source code for all the implicit conversions. Then think about how it fits in with match levels, function overloading, template overloading, partial ordering, ?:, covariance, contravariance, name mangling, TypeInfo, default initialization, will general functions have to be written twice (once for T, again for T?), auto function returns, and that's just off the top of my head. It's not just writing a truth table and throwing it over the wall.
Feb 02 2014
parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-03 04:06:00 +0000, Walter Bright <newshound2 digitalmars.com> said:

 For starters, grep through the source code for all the implicit 
 conversions. Then think about how it fits in with match levels, 
 function overloading, template overloading, partial ordering, ?:, 
 covariance, contravariance, name mangling, TypeInfo, default 
 initialization, will general functions have to be written twice (once 
 for T, again for T?), auto function returns, and that's just off the 
 top of my head.
 
 It's not just writing a truth table and throwing it over the wall.

But wouldn't that prevent all function arguments from being null at all? As well as function return values? That'll make it almost impossible to port existing code to this new D model, or interface with existing C functions. It'd still be possible, because you could always use a struct to pass a nullable pointer around, but then that struct will not work with partial ordering, ?:, covariance, and contravariance, making things quite annoying (just like Rebindable is for tail const/immutable classes). As for writing functions twice, I don't see it. T is implicitly convertible to T?, so a single function with a T? parameter will work with both T and T? arguments. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 03 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2014 3:45 AM, Michel Fortin wrote:
 As for writing functions twice, I don't see it. T is implicitly convertible to
 T?, so a single function with a T? parameter will work with both T and T?
 arguments.

It's the same issue as with const (which we dealt with by adding the 'inout' qualifier). It's the affect of the argument type on the return type.
Feb 03 2014
parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-03 19:53:58 +0000, Walter Bright <newshound2 digitalmars.com> said:

 On 2/3/2014 3:45 AM, Michel Fortin wrote:
 As for writing functions twice, I don't see it. T is implicitly convertible to
 T?, so a single function with a T? parameter will work with both T and T?
 arguments.

It's the same issue as with const (which we dealt with by adding the 'inout' qualifier). It's the affect of the argument type on the return type.

Const can represent either mutable or immutable data, so you have inout standing in as "same output as input". This is useful, necessary even, if your function returns some part of its input, like the value of a field. In theory it could make sense to have something similar for nullable, where you have a qualifier standing for as "same nullability as input". That said, you can't return a part of the input if the value is null, and that part's nullability won't depend on its parent (nullability is not transitive), so it's not as useful. The only useful thing that comes to mind is a function where you return the passed argument directly, such as this one: Object? foo(Object? o) { if (o) writeln(o); return o; } But, you could implement this as a template function if you absolutely need to transfer nullability, and the template will actually make things more efficient because the template instantiated with a non-nullable function argument will optimize away the unnecessary branch. And to get to the root of this, I think it'd be much more useful to have a "same type as input" stand-in, because notice how in the above function if you pass a DerivedObject, you'll get an Object returned, losing the derived part? Exact same problem, and we've been living with it for decades. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 03 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2014 7:24 PM, Michel Fortin wrote:
 But, you could implement this as a template function if you absolutely need to
 transfer nullability, and the template will actually make things more efficient
 because the template instantiated with a non-nullable function argument will
 optimize away the unnecessary branch.

Yes, you could use a template, but the idea was to avoid bloat in the executable by having multiple sets of functions that differ only by their mangled name. Furthermore, templates cannot be used to create virtual functions.
 And to get to the root of this, I think it'd be much more useful to have a
"same
 type as input" stand-in, because notice how in the above function if you pass a
 DerivedObject, you'll get an Object returned, losing the derived part? Exact
 same problem, and we've been living with it for decades.

The const issue is a recurring theme in a lot of C++ code, the derived type one just doesn't seem to come up. Also, what is being looked at to transfer is the qualifier, not the type, as the type may actually change (such as returning a pointer to a field of the argument).
Feb 03 2014
parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-04 03:45:53 +0000, Walter Bright <newshound2 digitalmars.com> said:

 On 2/3/2014 7:24 PM, Michel Fortin wrote:
 But, you could implement this as a template function if you absolutely need to
 transfer nullability, and the template will actually make things more efficient
 because the template instantiated with a non-nullable function argument will
 optimize away the unnecessary branch.

Yes, you could use a template, but the idea was to avoid bloat in the executable by having multiple sets of functions that differ only by their mangled name. Furthermore, templates cannot be used to create virtual functions.

Unlike with the mutable/const/immutable case, those templates actually won't only differ only by their mangled names, because any check for null will be quashed by the optimizer in the not-null variant. You can write two function if you really need them to be virtual, but as explained below I doubt you'll need to do that anyway.
 And to get to the root of this, I think it'd be much more useful to 
 have a "same
 type as input" stand-in, because notice how in the above function if you pass a
 DerivedObject, you'll get an Object returned, losing the derived part? Exact
 same problem, and we've been living with it for decades.

The const issue is a recurring theme in a lot of C++ code, the derived type one just doesn't seem to come up. Also, what is being looked at to transfer is the qualifier, not the type, as the type may actually change (such as returning a pointer to a field of the argument).

Yes, but const in C++ applies to all the fields of an aggregate. Making a struct variable const means all its fields are seen as const. To write accessors that works with both const and non-const you need to duplicate each of them. That's not the case for nullable. Fields do not become nullable because the pointer leading to the struct is nullable. You only have to write one accessor that takes a non-nullable pointer or ref, and let the parent context make sure it is not nullable before calling it. Even today, most accessors don't check "this" for null anyway, so we're just formalizing that contract and enforcing it statically. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 03 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2014 8:17 PM, Michel Fortin wrote:
 That's not the case for nullable.

Yes, it is. If the function accepts a pointer to S and returns a pointer to S.f, then a null pointer in means a null pointer out, and a completely different type.
Feb 03 2014
parent reply Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-04 04:26:22 +0000, Walter Bright <newshound2 digitalmars.com> said:

 On 2/3/2014 8:17 PM, Michel Fortin wrote:
 That's not the case for nullable.

Yes, it is. If the function accepts a pointer to S and returns a pointer to S.f, then a null pointer in means a null pointer out, and a completely different type.

For that to happen you'd have to have a function like this one: class MyObject { OtherObject value; } OtherObject? foo(MyObject? o) { if (o) return o.other; else return null; } The corresponding not-nullable version would be like that (notice the efficiency gain): OtherObject foo(MyObject o) { return o.other; // no need to check for null here } Honestly, I don't think writing accessors that check for null is that common, except to detect null pointers passed in error because of the lack of static checking. In general for an accessor it's better to let the parent function check for null and only implement the later version of the accessor. The parent function can do one null check before calling all the accessors it wants, and accessors don't have to check for null. You lose that efficiency if each accessors has to check for null. For the rare cases where you actually want both versions to work, you can write them twice or use a template (except in a virtual context), and in both cases you keep the efficiency of not checking for null when the argument is not nullable. In any case, I have yet to understand why nullable as a storage class would be any better. How do you solve that problem with a storage class? -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 04 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2014 4:23 AM, Michel Fortin wrote:
 For the rare cases where you actually want both versions to work,

I think you're making a vast assumption that the case is rare. When I write utility functions, I want them to work on as wide a variety of inputs as possible. Otherwise, they are not very useful.
 you can write them twice  or use a template (except in a virtual context), and
in both cases
 you keep the efficiency of not checking for null when the argument is not
nullable.

That's just what I wish to avoid. Consider adding more pointer types - the combinatorics quickly explode. Heck, just have two pointer parameters, and you already have 4 cases. I wonder how Rust deals with this.
 In any case, I have yet to understand why  nullable as a storage class would be
 any better. How do you solve that problem with a storage class?

Good question. I don't have an answer offhand.
Feb 04 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2014 2:39 PM, Dicebot wrote:
 Also non-nullable types should be implicitly cast to nullable parameters so you
 don't always need to support all cases distinctively.

The difficulty comes when needing to transfer the nullability of the argument to the nullability of the return type. Implicit conversions don't help with that. See inout.
Feb 04 2014
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2014 3:03 PM, Adam D. Ruppe wrote:
 So bottom line, you don't duplicate functions for the different types. You
 borrow references for processing (analogous to implementing algorithms with
 ranges) and own references for storing... which you need to know about, so only
 one type makes sense.

What happens with this: T identity(T t) { return t; } ?
Feb 04 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2014 4:07 PM, Frank Bauer wrote:
 Regarding different pointer types in Rust as function arguments:

 A function that takes a borrowed pointer &T can also be called with an owning
 pointer ~T, an RC pointer Rc<T>, or a GC pointer Gc<T>. They all convert neatly
 to a &T. One function to rule them ... err .. accomodate all.

Again, what happens with: T identity(T t) { return t; } ? I.e. the transference of the argument pointer type to the return type?
Feb 04 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/4/14, 5:34 PM, Frank Bauer wrote:
 On Wednesday, 5 February 2014 at 00:50:55 UTC, Walter Bright wrote:
 Again, what happens with:

     T identity(T t) { return t; }

 ? I.e. the transference of the argument pointer type to the return type?

As far as I see, in Rust the pointer type is lost on the return value as well, if your function takes a reference (borrowed pointer). But you can do: fn identity(x: &T) -> T { return *T; } fn main() { let a = ~T; let r = ~identity(a); } That is: pass by reference and return a value type.

Looks like a major weakness. Andrei
Feb 04 2014
prev sibling parent Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-04 22:30:40 +0000, Walter Bright <newshound2 digitalmars.com> said:

 On 2/4/2014 4:23 AM, Michel Fortin wrote:
 For the rare cases where you actually want both versions to work,

I think you're making a vast assumption that the case is rare. When I write utility functions, I want them to work on as wide a variety of inputs as possible. Otherwise, they are not very useful.

Well, if you think it's needed I won't contradict you. But that does not bode well for the storage class solution. If you really think it's needed, then you could just double the '?' to give it a similar meaning to inout: Object?? passthrough(Object?? o) { // do something here return o; } It looks elegant to me, even if I'm not convinced it is necessary.
 you can write them twice  or use a template (except in a virtual 
 context), and in both cases you keep the efficiency of not checking for 
 null when the argument is not nullable.

That's just what I wish to avoid. Consider adding more pointer types - the combinatorics quickly explode. Heck, just have two pointer parameters, and you already have 4 cases.

So you are already worried we'll have to add more pointer types? ;-) Ok, let's have some fun: '#' means unique now. So we can have four kinds of pointers: Object o1; // not-nullable & not-unique Object? o2; // nullable & not-unique Object# o3; // not-nullable & unique Object?# o4; // nullable & unique And like above, double the marker to make it work like inout: Object??## passthrough(Object??## o) { // have some fun here return o; } If you really think more pointer types will be added, then I suggest the implementation for this in the compiler be generic enough to support yes/no/maybe states for a couple of flags. Perhaps even user-defined flags for use by libraries (i.e. Dicebot's SQLEscapedString, or UnicodeNormalizedNFD, or why not PrimeNumber). -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 04 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/2/14, 9:30 AM, Namespace wrote:
 Sounds good. But why  nullable instead of C# choice of "Type?" ?

Because nullable is already in the language. Andrei
Feb 02 2014
prev sibling parent Shammah Chancellor <anonymous coward.com> writes:
On 2014-02-02 17:30:27 +0000, Namespace said:

 Sounds good. But why  nullable instead of C# choice of "Type?" ?

nullable is not supposed to do the same thing as "Type?" (or rather the syntatic sugar for Nullabe<Type> template). Nullable<type> is to enable Value types to be null. For example: Nullable<Double> nullable in D is supposed to allow reference types to contain null. In C# reference types are ALWAYS allowed to be null. -S
Feb 04 2014
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 02.02.2014 11:44, schrieb Dicebot:
 On Sunday, 2 February 2014 at 00:57:38 UTC, Adam Wilson wrote:
 ...

I often regret that D went so well as "yet another but faster" general-purpose language and is often viewed as such. And this is a niche overloaded with languages. Contrary to that, real-time / barebone stuff is rather narrow niche but there are only few usable languages there (and 0 good languages).

I consider Ada a good one.
Feb 02 2014
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 02.02.2014 10:03, schrieb Martin Cejp:
 On Sunday, 2 February 2014 at 05:12:05 UTC, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 04:23:35 UTC, Adam Wilson wrote:
 Maybe, but my point is that even with no allocations in phobos, you'd
 have to verify that ALL the libraries you use don't GC allocate.
 That's why so many people in the ARC camp want to make it the only GC
 system in D. Multiple GC systems increase your cognitive load
 considerably.

Good point. What we DON'T want is a library that comes in MySuperLib-GC, MySuperLib-ARC, MySuperLib-Owning and MySuperLib-Borrowed flavors. But I don't think that is necessary. A library function would typically take arguments by reference aka as a Borrowed(T). GC(T), ARC(T) and Owning(T) convert equally well to Borrowed(T). So somebody who uses only GC(T) in her code could use the library just as easily as someone who prefers Owning(T). The library would be compatible with the two, because it does not own the (reference to the) passed object. GC(T) or Owning(T) at the call site would free the object automatically (at the next collection cycle or when leaving the scope). Everybody is happy! All that is left for the caller is to choose whether she wants to use a library that uses GC *internally*. Interaction is possible in every combination: 1. I use GC(T), the library uses GC(T) internally -> works (with GC), 2. I use Owning(T), the library uses GC(T) internally -> works (with GC), 3. I use GC(T), the library uses Owning(T) internally -> works (with GC), 4. I use Owning(T), the library uses Owning(T) internally -> works (w/o GC).

This exists. It's called C++. I've pretty much given up on GC and do all my allocations in the real-time path manually. In a game, there shouldn't be that much stuff to allocate once live, anyway.

The problem with C++ is the C traps that live within and are the cause of many C++ gotchas, as described by Bjarne on the "The Design and Evolution of C++" book. -- Paulo
Feb 02 2014
prev sibling next sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Frustrated"  wrote in message news:kpejxijjckravodhixpi forum.dlang.org... 
 It's pure laziness.

Pull requests welcome.
Feb 03 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei
Feb 03 2014
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
04-Feb-2014 00:21, Adam Wilson пишет:
 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei


 Sadly, although written as hyperbole, I feel that the above is fairly
 close to the actual position of the ARC crowd.

I won't be surprised that half of current GC problems are because it's simply too stupid as an allocator. I had some numbers that I'd need to dig up were GC.malloc was ~2x slower then malloc even with garbage collection disabled. With that said I'd focus on ref-counting somehow coexisting with tracing GC. It doesn't go well at the moment - try creating dynamic arrays of std.stdio.File (that is ref-counted). -- Dmitry Olshansky
Feb 03 2014
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 2/3/2014 4:13 PM, H. S. Teoh wrote:
 I've seen real-life
 examples of ARCs gone horribly, horribly wrong, whereas had a GC been
 used in the first place things wouldn't have gone down that route.

I'm curious to hear more about this.
Feb 03 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 6:05 PM, Nick Sabalausky wrote:
 On 2/3/2014 4:13 PM, H. S. Teoh wrote:
 I've seen real-life
 examples of ARCs gone horribly, horribly wrong, whereas had a GC been
 used in the first place things wouldn't have gone down that route.

I'm curious to hear more about this.

Me too. Andrei
Feb 03 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 12:21 PM, Adam Wilson wrote:
 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

Andrei, I am sorry to report that anything other than complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users.

Why would you be sorry? "It is what it is", "customer is always right" etc. If we deem that important then we should look into supporting that scenario.
 No arguments
 can be made to otherwise, regardless of validity. As far as they are
 concerned the discussion of ARC vs. GC is closed and decided. ARC is the
 only path forward to the bright and glorious future of D. ARC most
 efficiently solves all memory management problems ever encountered.
 Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC!

 Sadly, although written as hyperbole, I feel that the above is fairly
 close to the actual position of the ARC crowd.

I'm going on a limb here, but it seems you're not convinced :o). Andrei
Feb 03 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 2:51 PM, Frank Bauer wrote:
 I'm not quite sure that I understand what you mean by GC avoidance being
 a major focus of 2014 though. In the long term, can I look forward to
 writing an absolutely, like in 100 %, like in guaranteed, GC free D app
 with all of current D's and Phobos' features if I choose to? Or does it
 mean: well for the most part it will avoid the GC, but if you're unlucky
 the GC might still kick in if you don't pay attention and when you least
 expect it?

Depends on how we (including YOU) design it. Andrei
Feb 03 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 3:13 PM, woh wrote:
   Any system that forces a single way of handling memory as the only
 viable method, be it GC( as D currently does), or ARC, is undesirable to me

    Rust seems to have found a very nice model, and even cpp with
 value/unique/rc/weak is IMO superior to what D currently offers

I think Rust expended too much language complexity on that one issue. Andrei
Feb 03 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 5:36 PM, Adam Wilson wrote:
 You still haven't dealt with the cyclic reference problem in ARC. There
 is absolutely no way ARC can handle that without programmer input,
 therefore, it is simply not possible to switch D to ARC without adding
 some language support to deal with cyclic-refs. Ergo, it is simply not
 possible to seamlessly switch D to ARC without creating all kinds of
 havoc as people now how memory leaks where they didn't before. In order
 to support ARC the D language will necessarily have to grow/change to
 accommodate it. Apple devs constantly have trouble with cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles. Andrei
Feb 03 2014
parent Michel Fortin <michel.fortin michelf.ca> writes:
On 2014-02-04 03:45:33 +0000, Manu <turkeyman gmail.com> said:

 The majority of trivial allocations don't produce cycles; closures,
 strings, temporary arrays and working sets.

Oh, no. Beware of closures. That's a frequent problem in Objective-C ARC, even for those who understand ARC well. You have to be very careful of closures creating cycles if you use them as callbacks. For instance, you have a view that has a pointer to the model and sets a callback for the model to call when something change to update the view; that's a cycle and you need to use a weak ref to the view within the closure. Pretty common pattern. -- Michel Fortin michel.fortin michelf.ca http://michelf.ca
Feb 03 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 5:36 PM, Adam Wilson wrote:
 Boehm demonstrated in one of his papers (2004) that thread-safe ARC may
 even lead to longer maximum pause times than a standard Tracing GC.

At least this paper claims RC/GC parity has been reached: http://research.microsoft.com/pubs/202163/rcix-oopsla-2013.pdf Andrei
Feb 03 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2014 3:13 PM, woh wrote:
   Any system that forces a single way of handling memory as the only viable
 method, be it GC( as D currently does)

This is incorrect. You can use malloc/free in D, as well as write & use your own allocators, and even write your own ref counted types.
Feb 03 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/4/14, 6:26 AM, Frank Bauer wrote:
 On Tuesday, 4 February 2014 at 06:47:03 UTC, Walter Bright wrote:
 On 2/3/2014 7:03 PM, Adam Wilson wrote:
 Note that ObjC has special syntax to handle weak pointers. It's not well
 understood by many.

Sounds like explicitly managed memory is hardly worse. On 2/3/2014 3:13 PM, woh wrote:
  Any system that forces a single way of handling memory as the only
 viable
 method, be it GC( as D currently does)

This is incorrect. You can use malloc/free in D, as well as write & use your own allocators, and even write your own ref counted types. On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
 It's also probably
 possible to create a drop-in replacement for the GC to do something
 else.

It certainly is possible. There's nothing magic about the current GC, it's just library code.

Andrei Alexandrescu wrote:
 2. Work on Phobos to see what can be done about avoiding unnecessary
 allocation. Most likely we'll need to also add a  nogc flag.
 ...
 4. Work on the core language and druntime to see how to seamlessly
 accommodate alternate GC mechanisms such as reference counting.
 ...
 I thought I made it clear that GC avoidance (which includes
 considering built-in reference counting) is a major focus of 2014.

 Andrei

Im totally confused: Walter, do you back what Andrei says or do we have a good cop / bad cop situation here?

There's no contradiction. Walter and I are on the same page. Andrei
Feb 04 2014
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/4/14, 1:59 AM, Don wrote:
 We're using D as a systems language on a global commercial scale. And
 no, we don't use malloc/free. We just don't use Phobos.

What do you use instead of malloc/free? Andrei
Feb 04 2014
prev sibling parent John J <john.joyus gmail.com> writes:
On 02/01/2014 07:57 PM, Adam Wilson wrote:
 On TIOBE, there are three non-GC languages in the entire
 top 20 (C, C++, Obj-C) the rest feature a non-ARC GC of some form.

Add "Pascal" and "Delphi/Object Pascal" to that.
Feb 04 2014
prev sibling next sibling parent reply "Sven Over" <dlang svenover.de> writes:
 std.typecons.RefCounted!T

 core.memory.GC.disable();

Wow. That was easy. I see, D's claim of being a multi-paradigm language is not false.
 Phobos does rely on the GC to some extent. Most algorithms and 
 ranges do not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help!
Dec 25 2012
next sibling parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 31.12.2012 13:14, schrieb Sven Over:
 A smart-pointer type for arrays can easily provide slices. It keeps a
 reference to the full array (which gets destructed when the last
 reference is dropped), but addresses a subrange.

I did exactly implement this in https://github.com/Ingrater/druntime/blob/master/src/core/refcounted.d Basically I implemented a struct that mimics a regular D array as far as possible, but it is using reference counting under the hood. Kind Regards Benjamin Thaut
Dec 31 2012
prev sibling next sibling parent Benjamin Thaut <code benjamin-thaut.de> writes:
Am 31.12.2012 13:36, schrieb Sven Over:
 Hi Benjamin! I've seen your druntime and thBase repos on GitHub. Very
 interesting stuff. As I have little (or to be honest: no) experience in
 D it's difficult for me to judge how much work needs to be done. Is the
 idea to replace Phobos completely?

The idea is not to replace phobos completely. Currenly it only contains the functionality that I did need for my projects and that could not be used from phobos. std.traits, std.typetuple, and others are still used from phobos because they don't leak (or could be made non leaking with very small modifications). thBase also contains a lot of functionality phobos does not have. E.g. containers, a better xml parser/writer, templates for automiatcally serializing and deserializing objects to/from xml files and much more. Everything that I think could be usefull for other projects in the future gets added to thBase. Knid Regards Benjamin Thaut
Dec 31 2012
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 12/31/2012 7:14 AM, Sven Over wrote:
 In my job I'm writing backend services that power a big web site.
 Perfomance is key, as the response time of the data service in most
 cases directly adds to the page load time. The bare possibility that the
 whole service pauses for, say, 100ms is making me feel very uncomfortable.

FWIW, Vibe.d <http://vibed.org/> is very careful about avoiding unnecessary GC allocations.
Jan 31 2014
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 01.02.2014 05:33, schrieb Frank Bauer:
 Yes, that is what Rust calls an "owning pointer". While we are at it, we
 might as well dream of "non-owning" or "borrowed" pointers, a.k.a.
 references in Rust. They don't have an effect on memory-deallocation
 when they go out of scope, but they prevent the owning pointer they
 "borrow" from (i.e. are assigned from) from being reassigned to a
 different object as long as there are borrowing references in scope.
 There is more to owning and borrowed pointers, but I think that is the
 essence.

  Andrei (who I believe is the go-to (!) guy for all things memeory
 allocation related right now):
 IIRC you mentioned that it was convenient to have the GC around for
 implementing the D language features. Would it just be a minor
 inconvenience to drop that dependency in the generated compiler output
 and replace it with new/delete or something equivalent to owning
 pointers, say over the next one or two years? Or would it be a major
 roadblock that would require too much redesign work? Maybe you could
 test the concept of owning and borrowed pointers internally for some
 compiler components before actually bringing them "to the surface" for
 us to play with, if it turns out useful. But only of course, if we could
 leave the rest of D as good as it is today.

 I would really like to see the GC-free requesting crowd be taken a lttle
 more seriously. Without asking them to forego D features or do manual
 C++-style memory management. Javas and .NETs garbage collectors had
 enough time to mature and are still unsatisfactory for many applications.

Just to nitpick, reference counting is also GC from CS point of view. :)
Feb 01 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/31/14, 8:33 PM, Frank Bauer wrote:
  Andrei (who I believe is the go-to (!) guy for all things memeory
 allocation related right now):
 IIRC you mentioned that it was convenient to have the GC around for
 implementing the D language features. Would it just be a minor
 inconvenience to drop that dependency in the generated compiler output
 and replace it with new/delete or something equivalent to owning
 pointers, say over the next one or two years? Or would it be a major
 roadblock that would require too much redesign work?

It would be a major effort. It's possible we have to do it.
 Maybe you could
 test the concept of owning and borrowed pointers internally for some
 compiler components before actually bringing them "to the surface" for
 us to play with, if it turns out useful. But only of course, if we could
 leave the rest of D as good as it is today.

 I would really like to see the GC-free requesting crowd be taken a lttle
 more seriously. Without asking them to forego D features or do manual
 C++-style memory management. Javas and .NETs garbage collectors had
 enough time to mature and are still unsatisfactory for many applications.

I consider memory allocation a major issue to focus on in 2014. Andrei
Feb 01 2014
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02/01/2014 06:57 PM, Frustrated wrote:
 Right, and because of the mentality that the GC is one size that
 fits all, phobos got this way. That mentality is still pervasive. ...

In the D community? Hardly.
 I call it laziness.
 ...

Well, name-calling does not get anything fixed.
Feb 01 2014
prev sibling next sibling parent "Leandro Lucarella" <leandro.lucarella sociomantic.com> writes:
On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
 Phobos does rely on the GC to some extent. Most algorithms and 
 ranges do not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal.

It isn't if you care about memory leaks. And also bare in mind that some language constructs also relies on the GC, like some dynamic arrays operations (concatenation and appending for example). Right now is pretty hard to write D code that doesn't really use the GC at all.
Dec 25 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, December 25, 2012 16:33:27 Leandro Lucarella wrote:
 On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
 Phobos does rely on the GC to some extent. Most algorithms and
 ranges do not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal.

It isn't if you care about memory leaks. And also bare in mind that some language constructs also relies on the GC, like some dynamic arrays operations (concatenation and appending for example). Right now is pretty hard to write D code that doesn't really use the GC at all.

There's also often no reason not to have the GC on and use it for certain stuff but use ref-counted objects as your main type of memory-management. Then, you avoid whatever issues the GC has in most cases but don't have to worry about what it takes to be able to not use it at all. For instance, arrays would probably be GC-allocated in general, since then you can use slices and whatnot, but maybe all of your user-defined objects on the heap could be malloced and freed with reference counts (though that sort of thing should be much easier once we have custom allocators - it's a bit more of a pain right now than it should be). - Jonathan M Davis
Dec 25 2012
prev sibling next sibling parent "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
wrote:
 On Tuesday, December 25, 2012 16:33:27 Leandro Lucarella wrote:
 On Tuesday, 25 December 2012 at 14:48:48 UTC, Sven Over wrote:
 Phobos does rely on the GC to some extent. Most algorithms 
 and
 ranges do not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal.

It isn't if you care about memory leaks. And also bare in mind that some language constructs also relies on the GC, like some dynamic arrays operations (concatenation and appending for example). Right now is pretty hard to write D code that doesn't really use the GC at all.

There's also often no reason not to have the GC on and use it for certain stuff but use ref-counted objects as your main type of memory-management. Then, you avoid whatever issues the GC has in most cases but don't have to worry about what it takes to be able to not use it at all. For instance, arrays would probably be GC-allocated in general, since then you can use slices and whatnot, but maybe all of your user-defined objects on the heap could be malloced and freed with reference counts (though that sort of thing should be much easier once we have custom allocators - it's a bit more of a pain right now than it should be). - Jonathan M Davis

but it is possible. Also it makes writing Code also easier ans more effektive in some places because: -You don't have to register manually allocated blocks with the gc -You control the exact order of destruction -You control the time of destruction -You control in which thread a object gets destructed -You will be encouraged to not write code that does do 100 allocations just to format a string (because it will leak) But obviously it has an impact on productivity. But I'm still more productive writing manual memory managed code in D then in C++. Kind Regards Benjamin Thaut
Dec 26 2012
prev sibling next sibling parent "Sven Over" <dlang svenover.de> writes:
On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
wrote:
 There's also often no reason not to have the GC on and use it 
 for certain stuff

One thing that really freaks me out is the fact that the garbage collector pauses the whole process, i.e. all threads. In my job I'm writing backend services that power a big web site. Perfomance is key, as the response time of the data service in most cases directly adds to the page load time. The bare possibility that the whole service pauses for, say, 100ms is making me feel very uncomfortable. We easily achieve the performance and reliability we need in C++, but I would love to give D a chance, as it solves many inconveniences of C++ in an elegant way. Metaprogramming and the threading model, just to name two.
 For instance, arrays would probably be GC-allocated in general, 
 since
 then you can use slices and whatnot,

A smart-pointer type for arrays can easily provide slices. It keeps a reference to the full array (which gets destructed when the last reference is dropped), but addresses a subrange. Thanks everyone for all the replies!
Dec 31 2012
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:
 On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
 wrote:
 There's also often no reason not to have the GC on and use it 
 for certain stuff

One thing that really freaks me out is the fact that the garbage collector pauses the whole process, i.e. all threads. In my job I'm writing backend services that power a big web site. Perfomance is key, as the response time of the data service in most cases directly adds to the page load time. The bare possibility that the whole service pauses for, say, 100ms is making me feel very uncomfortable. We easily achieve the performance and reliability we need in C++, but I would love to give D a chance, as it solves many inconveniences of C++ in an elegant way. Metaprogramming and the threading model, just to name two.
 For instance, arrays would probably be GC-allocated in 
 general, since
 then you can use slices and whatnot,

A smart-pointer type for arrays can easily provide slices. It keeps a reference to the full array (which gets destructed when the last reference is dropped), but addresses a subrange. Thanks everyone for all the replies!

I think the problem would be if you try to append to the slice (~): If the underlying array is not GC allocated, then it will append to a new GC allocated array (AFAIK). As long as you don't append, I'd say you are fine.
Dec 31 2012
prev sibling next sibling parent "Sven Over" <dlang svenover.de> writes:
On Wednesday, 26 December 2012 at 11:43:55 UTC, Benjamin Thaut 
wrote:
 In my experience its quite some work to remove the GC entierly, 
 but it is possible.

Hi Benjamin! I've seen your druntime and thBase repos on GitHub. Very interesting stuff. As I have little (or to be honest: no) experience in D it's difficult for me to judge how much work needs to be done. Is the idea to replace Phobos completely?
 Also it makes writing Code also easier ans more effektive in 
 some places
 because:
 -You don't have to register manually allocated blocks with the 
 gc
 -You control the exact order of destruction
 -You control the time of destruction
 -You control in which thread a object gets destructed
 -You will be encouraged to not write code that does do 100 
 allocations just to format a string (because it will leak)

I personally agree with all your points, but I must admit that their relevance depends on the programming patterns you are using. Experienced GC programmers certainly favour patterns that work well with GC. I personally prefer a style of programming that makes use of destructors and relies on them being executed at the right time.
 But obviously it has an impact on productivity.

I'm not sure about this at all. I write software that is designed to offer a certain performance, and latency is a big issue. I don't believe that I can achieve that with a "just allocate memory and don't ever care" attitude. I would need some background knowledge about how the GC works, and write code that doesn't screw up badly. Manual memory management also requires me to have an understanding of the implications behind it. But there are well-tested helpers (e.g. smart pointers) that simplify the task a lot. I do admit that, when performance (both in the sense of being fast and offering reliable performance) is not such a critical issue, then GC won't hurt and "just allocating" is fine and of course reduces development time. However, I tend to just use Python for this class problems. It comes down to a matter of choice and personal preference. I'm not saying that GC is bad and shouldn't be used. Many people are very happy with it and it does a great job for them. But my personal preference is manual memory management (then again greatly automated by use of reference counting and smart pointers).
Dec 31 2012
prev sibling next sibling parent "Sven Over" <dlang svenover.de> writes:
On Monday, 31 December 2012 at 12:36:06 UTC, monarch_dodra wrote:
 On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:

 A smart-pointer type for arrays can easily provide slices. It 
 keeps a reference to the full array (which gets destructed 
 when the last reference is dropped), but addresses a subrange.


 I think the problem would be if you try to append to the slice 
 (~): If the underlying array is not GC allocated, then it will 
 append to a new GC allocated array (AFAIK).

 As long as you don't append, I'd say you are fine.

Of course your smart-array-pointer type needs to implement the ~ operator and create a copy of the array in that case. I guess the full implementation would include something that would resemble that of std::vector in C++. I should get started writing those types. Then I could either demonstrate that it does work, or understand that it doesn't (and why).
Dec 31 2012
prev sibling next sibling parent "jerro" <a a.com> writes:
 Of course your smart-array-pointer type needs to implement the 
 ~ operator and create a copy of the array in that case. I guess 
 the full implementation would include something that would 
 resemble that of std::vector in C++.

std.container.Array already does that.
Dec 31 2012
prev sibling next sibling parent "Kiith-Sa" <kiithsacmp gmail.com> writes:
I think you're overthinking this way too much.

I'm writing a game engine, where latency is much more important
than in your case. Even 10ms would translate to visible jitter.
GC's not an issue, and disabling it permanently is very
counterproductive _unless_ you've exhausted all other options
(which you're unlikely to, as you can do everything you can do in
C++).  All you have to do is care about how you allocate and, if
GC seems to be an issue, profile to see _where_ the GC is being
called most and optimize those allocations.

Basic rules:

For classes with one or few instances (singletons, etc.), GC is
not an issue. For classes with hundreds-thousands of instances,
it might be an issue. Profile. For classes with more instances,
it probably is an issue. Profile, reuse instances, use structs,
manually allocate.  Arrays: To avoid reallocation on append, use
array.assumeSafeAppend() before the append (Assumes that the
array is not a slice of a bigger array that would get its data
overwritten).

If appending a lot, use std.array.Appender. If you need manually
allocated storage, use std.container.Array, or create a wrapper
around malloc/free. I'm using a templated wrapper that allows me
to record how much memory/instances was allocated with which
types.  Destructors: Structs are RAII like in C. And you can
easily create a malloc wrapper that will act exactly like
new/delete in C++.

Classes: call destroy(instance). Calls the dtor, but doesn't
deallocate the class (GC will do that later). Not much different
from deleting a class allocated through new in C++ (which is what
you do most of the time in C++).  If you absolutely need to free
the memory, create malloc/free wrappers for classes.  In my case,
classes are used for polymorphic stuff, which are usually single
or few-instance objects. Structs are used for most of the
many-instance objects (e.g. components of game entities), and are
usually stored in manually allocated arrays.

GC is _very_ useful for stuff like closures, which can greatly
simplify some code. Also, while I maintain that disabling GC
completely is a bad idea, it might be useful to disable it in a
small area of code _after_ profiling shows that GC is being
called too much there.

In my YAML parser library, I've found after profiling that 18% of
time was spent in GC. Most of that was in a small piece of code
that repeatedly allocated memory. Disabling GC before this code
and reenabling it after (scope(exit) to avoid leaking disabled
GC) resulted in GC taking ~2% of time, because many unnecessary
collects were consolidated into one after the GC was reenabled.
Memory usage didn't take a hit, because this was actually a
fairly small piece of code, not doing too many allocations per
call, just called very often.
Dec 31 2012
prev sibling next sibling parent "pjmlp" <pjmlp progtools.org> writes:
On Monday, 31 December 2012 at 14:05:01 UTC, Kiith-Sa wrote:
 I think you're overthinking this way too much.

 I'm writing a game engine, where latency is much more important
 than in your case. Even 10ms would translate to visible jitter.
 GC's not an issue, and disabling it permanently is very
 counterproductive _unless_ you've exhausted all other options
 (which you're unlikely to, as you can do everything you can do 
 in
 C++).  All you have to do is care about how you allocate and, if
 GC seems to be an issue, profile to see _where_ the GC is being
 called most and optimize those allocations.

 Basic rules:

 For classes with one or few instances (singletons, etc.), GC is
 not an issue. For classes with hundreds-thousands of instances,
 it might be an issue. Profile. For classes with more instances,
 it probably is an issue. Profile, reuse instances, use structs,
 manually allocate.  Arrays: To avoid reallocation on append, use
 array.assumeSafeAppend() before the append (Assumes that the
 array is not a slice of a bigger array that would get its data
 overwritten).

 If appending a lot, use std.array.Appender. If you need manually
 allocated storage, use std.container.Array, or create a wrapper
 around malloc/free. I'm using a templated wrapper that allows me
 to record how much memory/instances was allocated with which
 types.  Destructors: Structs are RAII like in C. And you can
 easily create a malloc wrapper that will act exactly like
 new/delete in C++.

 Classes: call destroy(instance). Calls the dtor, but doesn't
 deallocate the class (GC will do that later). Not much different
 from deleting a class allocated through new in C++ (which is 
 what
 you do most of the time in C++).  If you absolutely need to free
 the memory, create malloc/free wrappers for classes.  In my 
 case,
 classes are used for polymorphic stuff, which are usually single
 or few-instance objects. Structs are used for most of the
 many-instance objects (e.g. components of game entities), and 
 are
 usually stored in manually allocated arrays.

 GC is _very_ useful for stuff like closures, which can greatly
 simplify some code. Also, while I maintain that disabling GC
 completely is a bad idea, it might be useful to disable it in a
 small area of code _after_ profiling shows that GC is being
 called too much there.

 In my YAML parser library, I've found after profiling that 18% 
 of
 time was spent in GC. Most of that was in a small piece of code
 that repeatedly allocated memory. Disabling GC before this code
 and reenabling it after (scope(exit) to avoid leaking disabled
 GC) resulted in GC taking ~2% of time, because many unnecessary
 collects were consolidated into one after the GC was reenabled.
 Memory usage didn't take a hit, because this was actually a
 fairly small piece of code, not doing too many allocations per
 call, just called very often.

I think it will still take a few generations of programmers until everyone is confortable with using GC enabled languages for systems programming scenarios. Most people nowadays are not aware of what has already been done in academia, or are burned by bad experiences caused by whatever reasons. This situation will only improve when developers start aproaching the same class of problems with open minds on how to solve them in new ways. -- Paulo
Dec 31 2012
prev sibling next sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
On 25/12/2012 14:13, Sven Over wrote:
<snip>
 Also, garbage collection often tends to increase the memory footprint of
 your software.

ISTM you can't really make a like-for-like comparison, since non-GC programs vary a lot in how much they leak memory, which becomes almost a non-issue when you have GC. I say "almost" because GC systems often aren't perfect. For instance, a GC might mistake some sequence of four or eight bytes for a pointer and keep hold of the allocated memory it is pointing to. Java has been known to leak memory in certain circumstances, such as not disposing a window when you've finished with it, or creating a thread but never starting it. But there is something called packratting, which is a mistake at the code level of keeping a pointer hanging around for longer than necessary and therefore preventing whatever it's pointing to from being GC'd.
 You may be able to avoid this with some expert knowledge
 about the garbage collector, but this of course invalidates one of the
 main arguments in favour of GC, which is less complexity. I'd say that
 if you want to write memory- and CPU-efficient code, then you'll need a
 certain amount of understanding of your memory management. Be it your
 manual memory management, or the inner workings of your garbage collector.

 To cut things short, even after giving it a lot of thought I still feel
 uncomfortable about garbage collection.

That means you're uncomfortable about reference-counted smart pointers, because these are a form of garbage collection. :)
 I'm wondering whether it is
 viable to use the same smart pointer techniques as in C++: is there an
 implementation already of such smart pointers? Can I switch off GC in my
 D programs altogether? How much does the standard library rely on GC?
 I.e. would I have to write a alternative standard library to do this?

Good luck at getting your alternative library accepted as part of the D standard. :) Seriously though, it's as much built-in language functionality as the standard library that relies on GC. Any of the following will allocate memory from the GC heap: - create anything with the 'new' keyword - increase the length of a dynamic array - concatenate arrays - duplicate an array with .dup or .idup - put data into an associative array Which means sooner or later the garbage will need to be collected, unless the program is such that all memory allocation will be done at startup and then released only on exit. Of course, whether the program can reasonably be written in this way depends on the nature of the program.
 There is one blog post I found interesting to read in this context,
 which also underlines my worries: http://3d.benjamin-thaut.de/?p=20

An interesting set of observations. Looking at this one: "Calls to the druntime invariant handler are emitted in release build also and there is no way to turn them off. Even if the class does not have any invariants the invariant handler will always be called, walk the class hirarchy and generate multiple cache misses without actually doing anything." There seem to be two bugs here: that these calls are generated even in release mode, and that they are generated even if the class has no invariants. One of us needs to experiment with this a bit more and get these issues filed in Bugzilla. Stewart.
Jan 01 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, December 31, 2012 15:05:00 Kiith-Sa wrote:
 I think you're overthinking this way too much.
 
 I'm writing a game engine, where latency is much more important
 than in your case. Even 10ms would translate to visible jitter.
 GC's not an issue, and disabling it permanently is very
 counterproductive _unless_ you've exhausted all other options
 (which you're unlikely to, as you can do everything you can do in
 C++).  All you have to do is care about how you allocate and, if
 GC seems to be an issue, profile to see _where_ the GC is being
 called most and optimize those allocations.
 
 Basic rules:

If you've got this much figured out with reasonably clear guidelines, you should consider writing a blog post or article about it. - Jonathan M Davis
Jan 01 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:
 On Tuesday, 25 December 2012 at 19:23:59 UTC, Jonathan M Davis 
 wrote:
 There's also often no reason not to have the GC on and use it 
 for certain stuff

One thing that really freaks me out is the fact that the garbage collector pauses the whole process, i.e. all threads. In my job I'm writing backend services that power a big web site. Perfomance is key, as the response time of the data service in most cases directly adds to the page load time. The bare possibility that the whole service pauses for, say, 100ms is making me feel very uncomfortable.

I understand that. However, refcounted stuff tends to die in cluster as well and create pauses. The main issue here is clearly GC's implementation rather than the concept of GC in itself (which can be quite good at avoiding pauses if you are ready to make some other tradeoffs).
 We easily achieve the performance and reliability we need in 
 C++, but I would love to give D a chance, as it solves many 
 inconveniences of C++ in an elegant way. Metaprogramming and 
 the threading model, just to name two.

Here is something I tried in the past with some success : use RefCounted and GC.free . Doing so, you allocate in the GC heap, but you will limit greatly the amount of garbage that the GC have to collect by itself. Note that in some cases, GC means greater performances (usually when associated with immutability), so disabling it entirely don't seems a good idea to me.
Jan 01 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 31 December 2012 at 14:43:27 UTC, pjmlp wrote:
 I think it will still take a few generations of programmers 
 until everyone is confortable with using GC enabled languages 
 for systems programming scenarios.

 Most people nowadays are not aware of what has already been 
 done in academia, or are burned by bad experiences caused by 
 whatever reasons.

 This situation will only improve when developers start 
 aproaching the same class of problems with open minds on how to 
 solve them in new ways.

 --
 Paulo

D's GC has some implementation issue as well.
Jan 01 2013
prev sibling next sibling parent "DythroposTheImposter" <barf barf.com> writes:
  I'm interested in how the new LuaJIT GC ends up performing. But 
overall I can't say I have much hope for GC right now.

  GC/D = Generally Faster allocation. Has a cost associated with 
every living object.

  C++ = Generally Slower allocation, but while it is alive there 
is no cost.

  So as the heap grows, the GC language falls behind.

  This seems to be the case in every language I've looked at this 
uses a GC.
Jan 02 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 2 January 2013 at 11:41:33 UTC, 
DythroposTheImposter wrote:
  I'm interested in how the new LuaJIT GC ends up performing. 
 But overall I can't say I have much hope for GC right now.

  GC/D = Generally Faster allocation. Has a cost associated with 
 every living object.

True, however, GC + immutability allow new idoms that are plain impossible in a non GC world, and that are really efficients at reducing copies and allocations (and used in many high perf D libs). The comparison you present here fail to take this into account.
  C++ = Generally Slower allocation, but while it is alive there 
 is no cost.

Counting reference have a cost. Plus, when the heap grow, the object that dies together tend to grow as well, which cause pauses. Latency contrived programs, like video games, tends to avoid allocation altogether because of this. This obviously work both with GC and reference counting.
  So as the heap grows, the GC language falls behind.

That is more subtle than that. Reconsider what I wrote above, for instance that GC + immutability can be used to reduce allocations (so live heap size). You'll also have to consider that GC can be concurrent, when reference counting usually cannot (or loose some of its benefit compared to GC).
  This seems to be the case in every language I've looked at 
 this uses a GC.

Most GC language lack proper memory management, and sometime have design choice that are a nightmare for the GC (java have no value type for instance). You are comparing here way more than simply GC vs other memory management. Considering the pro and cons of each, an hybrid approach seems like the most reasonable thing to do in any high performance program. Which is possible in D because of GC.free .
Jan 02 2013
prev sibling next sibling parent "Thiez" <thiezz gmail.com> writes:
On Wednesday, 2 January 2013 at 12:32:01 UTC, deadalnix wrote:
 Most GC language lack proper memory management, and sometime 
 have design choice that are a nightmare for the GC (java have 
 no value type for instance).

Surely Java's primitive types (byte, short, int, long, float, double, boolean, and char) count as value types?
Jan 02 2013
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Tuesday, 1 January 2013 at 16:31:55 UTC, Stewart Gordon wrote:
 But there is something called packratting, which is a mistake 
 at the code level of keeping a pointer hanging around for 
 longer than necessary and therefore preventing whatever it's 
 pointing to from being GC'd.

I've heard it called "midlife crisis" :)
Jan 02 2013
prev sibling next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Wednesday, 2 January 2013 at 17:18:52 UTC, Thiez wrote:
 On Wednesday, 2 January 2013 at 12:32:01 UTC, deadalnix wrote:
 Most GC language lack proper memory management, and sometime 
 have design choice that are a nightmare for the GC (java have 
 no value type for instance).

Surely Java's primitive types (byte, short, int, long, float, double, boolean, and char) count as value types?

I think he meant non-primitives
Jan 02 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 2 January 2013 at 17:18:52 UTC, Thiez wrote:
 On Wednesday, 2 January 2013 at 12:32:01 UTC, deadalnix wrote:
 Most GC language lack proper memory management, and sometime 
 have design choice that are a nightmare for the GC (java have 
 no value type for instance).

Surely Java's primitive types (byte, short, int, long, float, double, boolean, and char) count as value types?

Yes, but they are irrelevant for the GC.
Jan 02 2013
prev sibling next sibling parent "pjmlp" <pjmlp progtools.org> writes:
On Wednesday, 2 January 2013 at 11:41:33 UTC, 
DythroposTheImposter wrote:
  I'm interested in how the new LuaJIT GC ends up performing. 
 But overall I can't say I have much hope for GC right now.

  GC/D = Generally Faster allocation. Has a cost associated with 
 every living object.

  C++ = Generally Slower allocation, but while it is alive there 
 is no cost.

  So as the heap grows, the GC language falls behind.

  This seems to be the case in every language I've looked at 
 this uses a GC.

As a former user from Native Oberon and BlueBottle OS back in the late 90's, I can attest that the GC was fast enough for powering a single user desktop operating system. Surely there are cases where it is an issue, mainly due to hardware constraints. While I am a GC supporter, I wouldn't like to fly in an airplane with a GC system, but a few missile radar systems are controled with GC based systems (Ground Master 400). So who knows?! -- Paulo
Jan 02 2013
prev sibling next sibling parent "Andrej Mitrovic" <andrej.mitrovich gmail.com> writes:
On Monday, 31 December 2012 at 14:05:01 UTC, Kiith-Sa wrote:
 All you have to do is care about how you allocate and, if
 GC seems to be an issue, profile to see _where_ the GC is being
 called most and optimize those allocations.

Bumping old thread: There is an ever-increasing amount of newcomers in #d on IRC who end up asking how to avoid the GC or at least to be able to determine where implicit allocations happen. I think we should work on creating a wiki page and gather as much advice as possible on dealing with the GC, implicit allocations, real-time constraints, etc.
Jan 31 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Wednesday, 2 January 2013 at 22:46:20 UTC, pjmlp wrote:
 On Wednesday, 2 January 2013 at 11:41:33 UTC, 
 DythroposTheImposter wrote:
 I'm interested in how the new LuaJIT GC ends up performing. 
 But overall I can't say I have much hope for GC right now.

 GC/D = Generally Faster allocation. Has a cost associated with 
 every living object.

 C++ = Generally Slower allocation, but while it is alive there 
 is no cost.

 So as the heap grows, the GC language falls behind.

 This seems to be the case in every language I've looked at 
 this uses a GC.

As a former user from Native Oberon and BlueBottle OS back in the late 90's, I can attest that the GC was fast enough for powering a single user desktop operating system. Surely there are cases where it is an issue, mainly due to hardware constraints. While I am a GC supporter, I wouldn't like to fly in an airplane with a GC system, but a few missile radar systems are controled with GC based systems (Ground Master 400). So who knows?! -- Paulo

The problem isn't stability, as you manual management can be just or even more error prone. The problem is with real time. If the GC must stop the world then it might be a stability issue with respect to the design. Audio and gaming are critical. While in some instances it is only superficial, such as video games, but absolutely a deal breaker with professional audio... even though in a sense these are not as critical as, say, a missile system, they are used much more often so are put to the test much more. There is, also, redundancy built into such critical systems as military defense... not so in a digital mixer for a PA that can't pop and click because the GC has to compact the heap. Your audience is going to be quite annoyed when they here a loud pop(due to the audio cutting off and potentially going from a high value to a low value which produces a very loud click) every 30 seconds, minute or so. Not to mention what that would do to their hearing over the course of a night. What if the GC decides to pause just at the wrong time when you only needed just a little bit more time to get the audio buffer filled? You are screwed. Manual management or ARC or whatever is at least much more predictable. Sure you can disable the GC but when you do that you just make things worse(potentially takes longer to catch up). I think with proper management these things can be avoided, but as of now, since the D core relies so much on the GC, it's kinda pointless unless you avoid all the goodies it provides. The mindset that the GC is the ultimate way to deal with memory management has led to the issues with D and the GC and why some avoid D. If it would have been designed as optional, we wouldn't be talking about these problems.
Jan 31 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
+1.

What it boils down to for me then, is:

If I want to write applications without low-latency requirements,
D is the perfect fit for me. D's templates, slices, CTFE instead
of a nasty macro or preprocessor language, compiler producing
machine code, are enough for me to switch from C# or Java.

If on the other hand, I do have low-latency requirements, I
simply don't want to jump through hoops, i.e. implement all
critical memory management C++ style by myself to prevent the GC
from kicking in. And If I even have to use GC profiling on top of
that, D's cleaner language concepts just don't cut it for me.
I'll take a look at Rust for that (admitted, not as well thought
out a language as D).
Jan 31 2014
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Friday, 31 January 2014 at 19:08:10 UTC, Andrej Mitrovic wrote:
 On Monday, 31 December 2012 at 14:05:01 UTC, Kiith-Sa wrote:
 All you have to do is care about how you allocate and, if
 GC seems to be an issue, profile to see _where_ the GC is being
 called most and optimize those allocations.

Bumping old thread: There is an ever-increasing amount of newcomers in #d on IRC who end up asking how to avoid the GC or at least to be able to determine where implicit allocations happen. I think we should work on creating a wiki page and gather as much advice as possible on dealing with the GC, implicit allocations, real-time constraints, etc.

We should concentrate on implementing scope. scope could prevent many of such things. E.g.: { scope int[] arr = new int[count]; } Since scope shall guarantee that the marked variable does not leave the scope, the compiler could safely assume that arr shouldn't handled with the GC. With that information the code could safely rewritten to: { int* tmp_arr_ptr = cast(int*) calloc(count, int.sizeof); scope(exit) free(tmp_arr_ptr); int[] arr = tmp_arr_ptr[0 .. count]; } So no need to invoke the GC. Since many objects have a small lifetime it is preferable to avoid the GC. With scope the user could mark such objects... and everyone wins. It is not perfect and does not cover all use cases but it would be a start. Sadly I doubt that scope gets implemented (soon). And since scope classes was replaced with an ugly library solution I seriously doubt that this nice dream comes true. But it would be so nice. :)
Jan 31 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Yes, that is what Rust calls an "owning pointer". While we are at 
it, we might as well dream of "non-owning" or "borrowed" 
pointers, a.k.a. references in Rust. They don't have an effect on 
memory-deallocation when they go out of scope, but they prevent 
the owning pointer they "borrow" from (i.e. are assigned from) 
from being reassigned to a different object as long as there are 
borrowing references in scope. There is more to owning and 
borrowed pointers, but I think that is the essence.

 Andrei (who I believe is the go-to (!) guy for all things 
memeory allocation related right now):
IIRC you mentioned that it was convenient to have the GC around 
for implementing the D language features. Would it just be a 
minor inconvenience to drop that dependency in the generated 
compiler output and replace it with new/delete or something 
equivalent to owning pointers, say over the next one or two 
years? Or would it be a major roadblock that would require too 
much redesign work? Maybe you could test the concept of owning 
and borrowed pointers internally for some compiler components 
before actually bringing them "to the surface" for us to play 
with, if it turns out useful. But only of course, if we could 
leave the rest of D as good as it is today.

I would really like to see the GC-free requesting crowd be taken 
a lttle more seriously. Without asking them to forego D features 
or do manual C++-style memory management. Javas and .NETs garbage 
collectors had enough time to mature and are still unsatisfactory 
for many applications.
Jan 31 2014
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
--001a11c36780a4764904f15191a7
Content-Type: text/plain; charset=UTF-8

On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:

  std.typecons.RefCounted!T
 core.memory.GC.disable();

Wow. That was easy. I see, D's claim of being a multi-paradigm language is not false.

It's not a realistic suggestion. Everything you want to link uses the GC, and the language its self also uses the GC. Unless you write software in complete isolation and forego many valuable features, it's not a solution. Phobos does rely on the GC to some extent. Most algorithms and ranges do
 not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help!

I've been trying to learn to love the GC for as long as I've been around here. I really wanted to break that mental barrier, but it hasn't happened. In fact, I am more than ever convinced that the GC won't do. My current #1 wishlist item for D is the ability to use a reference counted collector in place of the built-in GC. You're not alone :) I write realtime and memory-constrained software (console games), and for me, I think the biggest issue that can never be solved is the non-deterministic nature of the collect cycles, and the unknowable memory footprint of the application. You can't make any guarantees or predictions about the GC, which is fundamentally incompatible with realtime software. Language-level ARC would probably do quite nicely for the miscellaneous allocations. Obviously, bulk allocations are still usually best handled in a context sensitive manner; ie, regions/pools/freelists/whatever, but the convenience of the GC paradigm does offer some interesting and massively time-saving features to D. Everyone will always refer you to RefCounted, which mangles your types and pollutes your code, but aside from that, for ARC to be useful, it needs to be supported at the language-level, such that the language/optimiser is able to optimise out redundant incref/decref calls, and also that it is compatible with immutable (you can't manage a refcount if the object is immutable). --001a11c36780a4764904f15191a7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 2= 6 December 2012 00:48, Sven Over <span dir=3D"ltr">&lt;<a href=3D"mailto:dl= ang svenover.de" target=3D"_blank">dlang svenover.de</a>&gt;</span> wrote:<= br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left= :1px #ccc solid;padding-left:1ex"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> std.typecons.RefCounted!T<br> <br> core.memory.GC.disable();<br> </blockquote> <br> Wow. That was easy.<br> <br> I see, D&#39;s claim of being a multi-paradigm language is not false.</bloc= kquote><div><br></div><div>It&#39;s not a realistic suggestion. Everything = you want to link uses the GC, and the language its self also uses the GC. U= nless you write software in complete isolation and forego many valuable fea= tures, it&#39;s not a solution.</div> <div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Phobos does rely on the GC to some extent. Most algorithms and ranges do no= t though.<br> </blockquote> <br></div> Running (library) code that was written with GC in mind and turning GC off = doesn&#39;t sound ideal.<br> <br> But maybe this allows me to familiarise myself more with D. Who knows, mayb= e I can learn to stop worrying and love garbage collection.<br> <br> Thanks for your help!<br> </blockquote></div></div><div class=3D"gmail_extra"><br></div><div class=3D= "gmail_extra">I&#39;ve been trying to learn to love the GC for as long as I= &#39;ve been around here. I really wanted to break that mental barrier, but= it hasn&#39;t happened.</div> <div class=3D"gmail_extra">In fact, I am more than ever convinced that the = GC won&#39;t do. My current #1 wishlist item for D is the ability to use a = reference counted collector in place of the built-in GC.</div><div class=3D= "gmail_extra"> You&#39;re not alone :)</div><div class=3D"gmail_extra"><br></div><div clas= s=3D"gmail_extra">I write realtime and memory-constrained software (console= games), and for me, I think the biggest issue that can never be solved is = the non-deterministic nature of the collect cycles, and the unknowable memo= ry footprint of the application. You can&#39;t make any guarantees or predi= ctions about the GC, which is fundamentally incompatible with realtime soft= ware.</div> <div class=3D"gmail_extra">Language-level ARC would probably do quite nicel= y for the miscellaneous allocations. Obviously, bulk allocations are still = usually best handled in a context sensitive manner; ie, regions/pools/freel= ists/whatever, but the convenience of the GC paradigm does offer some inter= esting and massively time-saving features to D.</div> <div class=3D"gmail_extra">Everyone will always refer you to RefCounted, wh= ich mangles your types and pollutes your code, but aside from that, for ARC= to be useful, it needs to be supported at the language-level, such that th= e language/optimiser is able to optimise out redundant incref/decref calls,= and also that it is compatible with immutable (you can&#39;t manage a refc= ount if the object is immutable).</div> </div> --001a11c36780a4764904f15191a7--
Jan 31 2014
next sibling parent reply Shammah Chancellor <anonymous coward.com> writes:
This is a multi-part message in MIME format.

----------------896163706805894945
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

On 2014-02-01 07:35:44 +0000, Manu said:

 On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> wrote:
 On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:
 
 On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:
 
 std.typecons.RefCounted!T
 
 core.memory.GC.disable();
 
 
 Wow. That was easy.
 
 I see, D's claim of being a multi-paradigm language is not false.
 
 
 It's not a realistic suggestion. Everything you want to link uses the GC,
 and the language its self also uses the GC. Unless you write software in
 complete isolation and forego many valuable features, it's not a solution.
 
 
 Phobos does rely on the GC to some extent. Most algorithms and ranges do
 not though.
 
 
 Running (library) code that was written with GC in mind and turning GC off
 doesn't sound ideal.
 
 But maybe this allows me to familiarise myself more with D. Who knows,
 maybe I can learn to stop worrying and love garbage collection.
 
 Thanks for your help!
 
 
 I've been trying to learn to love the GC for as long as I've been around
 here. I really wanted to break that mental barrier, but it hasn't happened.
 In fact, I am more than ever convinced that the GC won't do. My current #1
 wishlist item for D is the ability to use a reference counted collector in
 place of the built-in GC.
 You're not alone :)
 
 I write realtime and memory-constrained software (console games), and for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable memory
 footprint of the application. You can't make any guarantees or predictions
 about the GC, which is fundamentally incompatible with realtime software.
 Language-level ARC would probably do quite nicely for the miscellaneous
 allocations. Obviously, bulk allocations are still usually best handled in
 a context sensitive manner; ie, regions/pools/freelists/whatever, but the
 convenience of the GC paradigm does offer some interesting and massively
 time-saving features to D.
 Everyone will always refer you to RefCounted, which mangles your types and
 pollutes your code, but aside from that, for ARC to be useful, it needs to
 be supported at the language-level, such that the language/optimiser is
 able to optimise out redundant incref/decref calls, and also that it is
 compatible with immutable (you can't manage a refcount if the object is
 immutable).
 
 The problem isn't GC's per se. But D's horribly naive implementation, 
 games are written on GC languages now all the time (Unity/.NET). And 
 let's be honest, games are kind of a speciality, games do things most 
 programs will never do.
 
 You might want to read the GC Handbook. GC's aren't bad, but most, like 
 the D GC, are just to simplistic for common usage today.
 
 Maybe a sufficiently advanced GC could address the performance 
 non-determinism to an acceptable level, but you're still left with the 
 memory non-determinism, and the conundrum that when your heap 
 approaches full (which is _always_ on a games console), the GC has to 
 work harder and harder, and more often to try and keep the tiny little 
 bit of overhead available.
 A GC heap by nature expects you to have lots of memory, and also lots 
 of FREE memory.
 
 No serious console game I'm aware of has ever been written in a 
 language with a GC. Casual games, or games that don't attempt to raise 
 the bar may get away with it, but that's not the industry I work in.

You can always force the GC to run between cycles in your game, and turn off automatic sweeps. This is how most games operate nowadays. It's also probably possible to create a drop-in replacement for the GC to do something else. I could see if being *VERY* useful to make the GC take a compile-time parameter to select which GC engine is used. ----------------896163706805894945 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: 8bit <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <meta http-equiv="Content-Style-Type" content="text/css"> <title></title> <meta name="Generator" content="Cocoa HTML Writer"> <meta name="CocoaVersion" content="1265"> <style type="text/css"> p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 15.0px; font: 12.0px Helvetica} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 15.0px; font: 12.0px Helvetica; min-height: 14.0px} p.p3 {margin: 0.0px 0.0px 0.0px 12.0px; font: 16.0px Times; color: #011892} p.p4 {margin: 0.0px 0.0px 0.0px 12.0px; font: 16.0px Times; color: #011892; min-height: 19.0px} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 14.0px; font: 12.0px Helvetica; min-height: 14.0px} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 14.0px; font: 12.0px Helvetica} span.s1 {direction: ltr; unicode-bidi: embed} span.s2 {text-decoration: underline ; direction: ltr; unicode-bidi: embed} span.s3 {text-decoration: underline} </style> </head> <body> <p class="p1">On 2014-02-01 07:35:44 +0000, Manu said:</p> <p class="p2"><br></p> <p class="p3">On 1 February 2014 16:26, Adam Wilson <span class="s1">&lt;<a href="mailto:flyboynw gmail.com"><span class="s2">flyboynw gmail.com</span></a>&gt;</span> wrote:</p> <p class="p3">On Fri, 31 Jan 2014 21:29:04 -0800, Manu &lt;<a href="mailto:turkeyman gmail.com"><span class="s3">turkeyman gmail.com</span></a>&gt; wrote:</p> <p class="p4"><br></p> <p class="p3">On 26 December 2012 00:48, Sven Over &lt;<a href="mailto:dlang svenover.de"><span class="s3">dlang svenover.de</span></a>&gt; wrote:</p> <p class="p4"><br></p> <p class="p3">std.typecons.RefCounted!T</p> <p class="p4"><br></p> <p class="p3">core.memory.GC.disable();</p> <p class="p4"><br></p> <p class="p4"><br></p> <p class="p3">Wow. That was easy.</p> <p class="p4"><br></p> <p class="p3">I see, D's claim of being a multi-paradigm language is not false.</p> <p class="p4"><br></p> <p class="p4"><br></p> <p class="p3">It's not a realistic suggestion. Everything you want to link uses the GC,</p> <p class="p3">and the language its self also uses the GC. Unless you write software in</p> <p class="p3">complete isolation and forego many valuable features, it's not a solution.</p> <p class="p4"><br></p> <p class="p4"><br></p> <p class="p3">Phobos does rely on the GC to some extent. Most algorithms and ranges do</p> <p class="p3">not though.</p> <p class="p4"><br></p> <p class="p4"><br></p> <p class="p3">Running (library) code that was written with GC in mind and turning GC off</p> <p class="p3">doesn't sound ideal.</p> <p class="p4"><br></p> <p class="p3">But maybe this allows me to familiarise myself more with D. Who knows,</p> <p class="p3">maybe I can learn to stop worrying and love garbage collection.</p> <p class="p4"><br></p> <p class="p3">Thanks for your help!</p> <p class="p4"><br></p> <p class="p4"><br></p> <p class="p3">I've been trying to learn to love the GC for as long as I've been around</p> <p class="p3">here. I really wanted to break that mental barrier, but it hasn't happened.</p> <p class="p3">In fact, I am more than ever convinced that the GC won't do. My current #1</p> <p class="p3">wishlist item for D is the ability to use a reference counted collector in</p> <p class="p3">place of the built-in GC.</p> <p class="p3">You're not alone :)</p> <p class="p4"><br></p> <p class="p3">I write realtime and memory-constrained software (console games), and for</p> <p class="p3">me, I think the biggest issue that can never be solved is the</p> <p class="p3">non-deterministic nature of the collect cycles, and the unknowable memory</p> <p class="p3">footprint of the application. You can't make any guarantees or predictions</p> <p class="p3">about the GC, which is fundamentally incompatible with realtime software.</p> <p class="p3">Language-level ARC would probably do quite nicely for the miscellaneous</p> <p class="p3">allocations. Obviously, bulk allocations are still usually best handled in</p> <p class="p3">a context sensitive manner; ie, regions/pools/freelists/whatever, but the</p> <p class="p3">convenience of the GC paradigm does offer some interesting and massively</p> <p class="p3">time-saving features to D.</p> <p class="p3">Everyone will always refer you to RefCounted, which mangles your types and</p> <p class="p3">pollutes your code, but aside from that, for ARC to be useful, it needs to</p> <p class="p3">be supported at the language-level, such that the language/optimiser is</p> <p class="p3">able to optimise out redundant incref/decref calls, and also that it is</p> <p class="p3">compatible with immutable (you can't manage a refcount if the object is</p> <p class="p3">immutable).</p> <p class="p4"><br></p> <p class="p3">The problem isn't GC's per se. But D's horribly naive implementation, games are written on GC languages now all the time (Unity/.NET). And let's be honest, games are kind of a speciality, games do things most programs will never do.</p> <p class="p4"><br></p> <p class="p3">You might want to read the GC Handbook. GC's aren't bad, but most, like the D GC, are just to simplistic for common usage today.</p> <p class="p4"><br></p> <p class="p3">Maybe a sufficiently advanced GC could address the performance non-determinism to an acceptable level, but you're still left with the memory non-determinism, and the conundrum that when your heap approaches full (which is _always_ on a games console), the GC has to work harder and harder, and more often to try and keep the tiny little bit of overhead available.</p> <p class="p3">A GC heap by nature expects you to have lots of memory, and also lots of FREE memory.</p> <p class="p4"><br></p> <p class="p3">No serious console game I'm aware of has ever been written in a language with a GC. Casual games, or games that don't attempt to raise the bar may get away with it, but that's not the industry I work in.</p> <p class="p5"><br></p> <p class="p6">You can always force the GC to run between cycles in your game, and turn off automatic sweeps.<span class="Apple-converted-space"> </span>This is how most games operate nowadays. <span class="Apple-converted-space"> </span>It's also probably possible to create a drop-in replacement for the GC to do something else. <span class="Apple-converted-space"> </span>I could see if being *VERY* useful to make the GC take a compile-time parameter to select which GC engine is used. <span class="Apple-converted-space"></span></p> <p class="p5"><br></p> </body> </html> ----------------896163706805894945--
Feb 03 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 4:49 PM, Adam Wilson wrote:
 That may be the case, but StackOverflow shows that ARC hasn't been
 panacea in Apple land either. Way to many people don't understand ARC
 and how to use it, and subsequently beg for help understanding
 heisenleaks and weak references. ARC places a higher cognitive load on
 the programmer than a GC does. And Android runs just fine with GC'ed
 apps, but ARC guys don't want to talk about Google's successes there.

Links please. Andrei
Feb 03 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2014 7:03 PM, Adam Wilson wrote:
 Note that ObjC has special syntax to handle weak pointers. It's not well
 understood by many.

Sounds like explicitly managed memory is hardly worse.
Feb 03 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/3/14, 5:51 PM, Manu wrote:
 I'd have trouble disagreeing more; Android is the essence of why Java
 should never be used for user-facing applications.
 Android is jerky and jittery, has random pauses and lockups all the
 time, and games on android always jitter and drop frames. Most high-end
 games on android now are written in C++ as a means to mitigate that
 problem, but then you're back writing C++. Yay!
 iOS is silky smooth by comparison to Android.

Kinda difficult to explain the market success of Android. Andrei
Feb 03 2014
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 2/3/2014 10:30 PM, Manu wrote:
 On 4 February 2014 12:59, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org
 wrote:

 Kinda difficult to explain the market success of Android.


Not really. With PalmOS long dead (RIP) and MS famously just flailing around on anything battery-powered, Android has been the only realistic option for those who want a handheld computer, but don't want to sell all their freedoms to Apple. Plus it's Google.
 2. It's free for product vendors.

No. I mean, on the surface yea it appears free, but realistically it's on the decline: Device manufacturers (partly under pressure from asswhores like Verizon, no doubt) have been moving towards a system where normal, affordable, contract-subsidized devices (not that I'm a fan of the contracts) are locked-down as much as possible [1], even using eFuses now, and thereby rendered impractical for average developers (since you can't freely test on multiple OS versions). But, GOLLY GEE!! FOR *ONLY SEVERAL HUNDREDS* OF PURELY OUT-OF-POCKET DOLLARS, these douchebag corporations like Google or Samsung will happily sell you a "developer" phone that restricts FEWER of your freedoms! Isn't that great? And best of all, Google gets to pretend being an Android developer is still free! Get it? Because the extra $$$ developers pay is for the psuedo-optional *wink wink nudge nudge* "developer phone" instead of a "developer account" like mean-ol'-Apple. [1] Not only the usual locked bootloaders, but see also Knox-secured bootloader. And no, I'm not being tin-foil-hat here, I've actually been bit by that and had a device bricked because of that forced-update Knox-secured bullshit, just because I needed to switch between 4.3 and 4.1.2 which Samsung decided I shouldn't be allowed to do (Knox eFuse). Luckily it was still under warranty and pretty much all salespeople and support reps are about as intelligent as your average house pet, so I was able to get a replacement...at least after they finally got tired of screwing up the replacement orders (hmmm, a DOA device...a wrong device from a completely different manufacturer...).
Feb 03 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/4/14, 6:45 AM, Dicebot wrote:
 On Tuesday, 4 February 2014 at 14:22:41 UTC, Paulo Pinto wrote:
 Rust still remains to prove itself on the mainstream market.

 While D already has commercial users.

It is dangerous position. D is not that far ahead to appeal to own authority.

I agree. Andrei
Feb 04 2014
prev sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 04/02/2014 04:30, Manu a écrit :
 On 4 February 2014 12:59, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org <mailto:SeeWebsiteForEmail erdani.org>>
 wrote:

     On 2/3/14, 5:51 PM, Manu wrote:

         I'd have trouble disagreeing more; Android is the essence of why
         Java
         should never be used for user-facing applications.
         Android is jerky and jittery, has random pauses and lockups all the
         time, and games on android always jitter and drop frames. Most
         high-end
         games on android now are written in C++ as a means to mitigate that
         problem, but then you're back writing C++. Yay!
         iOS is silky smooth by comparison to Android.


     Kinda difficult to explain the market success of Android.


 I think it's easy to explain.
 1. It's aggressively backed by the biggest technology company in the world.
 2. It's free for product vendors.
 3. For all product vendors at the curve of Android's success, it
 presented a realistic and well supported (by Google) competition to
 Apple, who were running away with the industry. Everybody had to compete
 with Apple, but didn't have the resources to realistically compete on
 their own. Nokia for instance were certainly in the best position to
 produce serious competition, but they fumbled multiple times. I suspect
 Google won because they're Google, and it is free.

 Even Microsoft failed. C# is, for all intents and purposes, the same as
 Java, except it's even better. If Java was a significant factor in
 Android's success, I'd argue that WindowsMobile should have been equally
 successful.
 I think it's safe to say, that's not the case.
 Personally, I suspect that Java was actually a barrier to entry in early
 Android, and even possibly the reason that it took as it did for Android
 to take root.
 It's most certainly the reason that Android had absolutely no games on
 it for so many years. They eventually released the NDK, and games
 finally appeared. There were years between Angry Birds success in
 iPhone, and any serious games appearing on Android.
 There were years where if you wanted to play games in your mobile
 device, you had to get an iDevice. It's finally levelled now that the
 indisputable success of Android is absolute (and the NDK is available).

I work almost every day on android with NDK and Qt. Tools still poor and there is a lot of issues. I found one critical last week in the ndk : an error with a python dll causing gdb launch failure, so sometimes you just haven't debugger... Developing games on such devices can be a real pain, apk can't exceed 50Mo and updating files with the obb package must be done manually. We never got those kind of difficulties with iOS even at the beginning. PS : I just buy a Nexus 5 to replace my old Motorola defy, and there is absolutely no pauses everything is perfectly smooth. I had some doubts if it was possible before buying it cause of Java,... No I just think all constructors doesn't delivers good drivers and put completely buggy additional components making the bad reputation of Android. On my defy I always got bad touch events,... It was a real pain to send an SMS :-{
Feb 04 2014
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 2/3/2014 7:49 PM, Adam Wilson wrote:
 And Android runs just fine with GC'ed
 apps, but ARC guys don't want to talk about Google's successes there.

FWIW, my Galaxy S3 randomly gets a lot of irritating second-or-two bouts of complete unresponsiveness.
Feb 03 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
 It's also probably
 possible to create a drop-in replacement for the GC to do something else.

It certainly is possible. There's nothing magic about the current GC, it's just library code.
Feb 03 2014
parent reply Shammah Chancellor <anonymous coward.com> writes:
On 2014-02-04 06:50:51 +0000, Walter Bright said:

 On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
 It's also probably
 possible to create a drop-in replacement for the GC to do something else.

It certainly is possible. There's nothing magic about the current GC, it's just library code.

Is it possible that we add some additional functionality to the GC API so that it could do ARC? I took a look at it, and it seems that right now, there'd be no way to implement ARC. -S.
Feb 04 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/4/2014 4:10 AM, Shammah Chancellor wrote:
 On 2014-02-04 06:50:51 +0000, Walter Bright said:

 On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
 It's also probably
 possible to create a drop-in replacement for the GC to do something else.

It certainly is possible. There's nothing magic about the current GC, it's just library code.

Is it possible that we add some additional functionality to the GC API so that it could do ARC? I took a look at it, and it seems that right now, there'd be no way to implement ARC.

ARC would require compiler changes, so it cannot be drop in.
Feb 04 2014
prev sibling next sibling parent "Eric Suen" <eric.suen.tech gmail.com> writes:
"Ola Fosheim Gr?stad" <ola.fosheim.grostad+dlang gmail.com>"> On Tuesday, 4
February 2014 at 00:49:04 UTC, Adam Wilson 
wrote:
 Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely successful GUI
toolkits that are not known for GC related 
 problems. I've been working with WPF since 2005, I can say the biggest
performance problem with it by far is the 
 naive rendering of rounded corners, the GC has NEVER caused a hitch.

According to Wikipedia: ?While the majority of WPF is in managed code, the composition engine which renders the WPF applications is a native component. It is named Media Integration Layer (MIL) and resides in milcore.dll. It interfaces directly with DirectX and provides basic support for 2D and 3D surfaces, timer-controlled manipulation of contents of a surface with a view to exposing animation constructs at a higher level, and compositing the individual elements of a WPF application into a final 3D "scene" that represents the UI of the application and renders it to the screen.? So, Microsoft does not think that GC is suitable for real time interactive graphics. And they are right.

As long as other code is in managed code, there is GC running at background, no matter your code write in whatever high performance language, it will be affect by GC anyway. so that "So, Microsoft does not think that GC is suitable for real time > interactive graphics. And they are right." is only your opinion. you don't know the real reason behind MS's decision. some resource need release ASAP, so you can't rely on GC. or maybe bacause of overhead when call low API in managed code.
Feb 03 2014
prev sibling next sibling parent Shammah Chancellor <anonymous coward.com> writes:
On 2014-02-03 23:00:22 +0000, woh said:
 
 On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor wrote:
 On 2014-02-01 07:35:44 +0000, Manu said:
 
 On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> wrote:
 On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:
 
 On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:
 
 std.typecons.RefCounted!T
 
 core.memory.GC.disable();
 
 
 Wow. That was easy.
 
 I see, D's claim of being a multi-paradigm language is not false.
 
 
 It's not a realistic suggestion. Everything you want to link uses the GC,
 and the language its self also uses the GC. Unless you write software in
 complete isolation and forego many valuable features, it's not a solution.
 
 
 Phobos does rely on the GC to some extent. Most algorithms and ranges do
 not though.
 
 
 Running (library) code that was written with GC in mind and turning GC off
 doesn't sound ideal.
 
 But maybe this allows me to familiarise myself more with D. Who knows,
 maybe I can learn to stop worrying and love garbage collection.
 
 Thanks for your help!
 
 
 I've been trying to learn to love the GC for as long as I've been around
 here. I really wanted to break that mental barrier, but it hasn't happened.
 In fact, I am more than ever convinced that the GC won't do. My current #1
 wishlist item for D is the ability to use a reference counted collector in
 place of the built-in GC.
 You're not alone :)
 
 I write realtime and memory-constrained software (console games), and for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable memory
 footprint of the application. You can't make any guarantees or predictions
 about the GC, which is fundamentally incompatible with realtime software.
 Language-level ARC would probably do quite nicely for the miscellaneous
 allocations. Obviously, bulk allocations are still usually best handled in
 a context sensitive manner; ie, regions/pools/freelists/whatever, but the
 convenience of the GC paradigm does offer some interesting and massively
 time-saving features to D.
 Everyone will always refer you to RefCounted, which mangles your types and
 pollutes your code, but aside from that, for ARC to be useful, it needs to
 be supported at the language-level, such that the language/optimiser is
 able to optimise out redundant incref/decref calls, and also that it is
 compatible with immutable (you can't manage a refcount if the object is
 immutable).
 
 The problem isn't GC's per se. But D's horribly naive implementation, 
 games are written on GC languages now all the time (Unity/.NET). And 
 let's be honest, games are kind of a speciality, games do things most 
 programs will never do.
 
 You might want to read the GC Handbook. GC's aren't bad, but most, like 
 the D GC, are just to simplistic for common usage today.
 
 Maybe a sufficiently advanced GC could address the performance 
 non-determinism to an acceptable level, but you're still left with the 
 memory non-determinism, and the conundrum that when your heap 
 approaches full (which is _always_ on a games console), the GC has to 
 work harder and harder, and more often to try and keep the tiny little 
 bit of overhead available.
 A GC heap by nature expects you to have lots of memory, and also lots 
 of FREE memory.
 
 No serious console game I'm aware of has ever been written in a 
 language with a GC. Casual games, or games that don't attempt to raise 
 the bar may get away with it, but that's not the industry I work in.

You can always force the GC to run between cycles in your game, and turn off automatic sweeps. This is how most games operate nowadays. It's also probably possible to create a drop-in replacement for the GC to do something else. I could see if being *VERY* useful to make the GC take a compile-time parameter to select which GC engine is used.


   ur right I never thought of that, I bet all them game devs never 
 thought of it either, they so dumb.  I bet they never tried to use a 
 GC, what fools!  Endless graphs of traced objects, oh yes oh yes!  It 
 only runs when I allocate, oh what a fool I've been, please castigate 
 me harder!
 
   Adam pls tell me more of this c# and that amazing gc it sounds so good

First of all, bottom quoting is evil. Second of all, your response is immature. Thirdly, I am not adam. And fourthly, I specifically mention that many games are currently using garbage collection. -S.
Feb 04 2014
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 04.02.2014 01:24, schrieb NoUseForAName:
 On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:
  ur right I never thought of that, I bet all them game devs never
 thought of it either, they so dumb.  I bet they never tried to use a
 GC, what fools!  Endless graphs of traced objects, oh yes oh yes!  It
 only runs when I allocate, oh what a fool I've been, please castigate
 me harder!

Also people should consider that Apple (unlike C++ game devs) did not have a tradition of contempt for GC. In fact they tried GC *before* they switched to ARC. The pro-GC camp always likes to pretend that the anti-GC one is just ignorant, rejecting GC based on prejudice not experience but Apple rejected GC based on experience. GCed Objective-C did not allow them to deliver the user experience they wanted (on mobile), because of the related latency issues. So they switched to automated ref counting. It is not in question that ref counting sacrifices throughput (compared to an advanced GC) but for interactive, user facing applications latency is much more important.

The reality of those hanging around Apple developer forums before ARC existed, was more related to crashes caused by Apple's implementation and their interaction with third party libraries. Specially the amount of special use cases one needed to take care of. They *failed* to produce a stable GC implementation for Objective-C, that is the reason, not better performance. -- Paulo
Feb 04 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/4/14, 11:28 AM, Steven Schveighoffer wrote:
 Where you have to be cognizant is avoiding cycles. Plain and simple. And
 it's not that difficult.

Do you have evidence to back that up? Andrei
Feb 04 2014
prev sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 04.02.2014 23:17, schrieb Steven Schveighoffer:
 On Tue, 04 Feb 2014 16:37:20 -0500, Ola Fosheim Grøstad
 <ola.fosheim.grostad+dlang gmail.com> wrote:

 If you blindly use ARC rather than sane RC. There is no reason to up
 the ref count if the data structure is "owned" while processing it.
 Which is a good reason to avoid ARC and use regular RC. If you know
 that the entire graph has a 1+ count, you don't need to do any ref
 counting while processing it.

This is an important point. However, the default is to be safe, and good enough for most. As long as there is a way to go into "I know what I'm doing" mode for encapsulated low-level code, the default should be to do ARC. ARC should handle simple cases, like if the reference only ever exists as a stack local. One could also say you could mark a memory region as no-scan if you know that the data it points at will never be collected for its entire existence in a GC environment. I think some mix of GC and ref counting for D would be extremely useful. In my major Objective C project, I use manual reference counting, and I do avoid extra retains for redundant pointers. I've also had a few bugs because of that :) -Steve

How big is your team? Based on my experience, this type of coding falls apart in the typical corporate team size. -- Paulo
Feb 04 2014
prev sibling next sibling parent "woh" <wojw yahoo.com> writes:
  ur right I never thought of that, I bet all them game devs never 
thought of it either, they so dumb.  I bet they never tried to 
use a GC, what fools!  Endless graphs of traced objects, oh yes 
oh yes!  It only runs when I allocate, oh what a fool I've been, 
please castigate me harder!

  Adam pls tell me more of this c# and that amazing gc it sounds 
so good




On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor 
wrote:
 On 2014-02-01 07:35:44 +0000, Manu said:

 On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> 
 wrote:
 On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> 
 wrote:
 
 On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:
 
  std.typecons.RefCounted!T
 
 core.memory.GC.disable();
 
 
 Wow. That was easy.
 
 I see, D's claim of being a multi-paradigm language is not 
 false.
 
 
 It's not a realistic suggestion. Everything you want to link 
 uses the GC,
 and the language its self also uses the GC. Unless you write 
 software in
 complete isolation and forego many valuable features, it's not 
 a solution.
 
 
  Phobos does rely on the GC to some extent. Most algorithms 
 and ranges do
 not though.
 
 
 Running (library) code that was written with GC in mind and 
 turning GC off
 doesn't sound ideal.
 
 But maybe this allows me to familiarise myself more with D. 
 Who knows,
 maybe I can learn to stop worrying and love garbage collection.
 
 Thanks for your help!
 
 
 I've been trying to learn to love the GC for as long as I've 
 been around
 here. I really wanted to break that mental barrier, but it 
 hasn't happened.
 In fact, I am more than ever convinced that the GC won't do. 
 My current #1
 wishlist item for D is the ability to use a reference counted 
 collector in
 place of the built-in GC.
 You're not alone :)
 
 I write realtime and memory-constrained software (console 
 games), and for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the 
 unknowable memory
 footprint of the application. You can't make any guarantees or 
 predictions
 about the GC, which is fundamentally incompatible with 
 realtime software.
 Language-level ARC would probably do quite nicely for the 
 miscellaneous
 allocations. Obviously, bulk allocations are still usually 
 best handled in
 a context sensitive manner; ie, 
 regions/pools/freelists/whatever, but the
 convenience of the GC paradigm does offer some interesting and 
 massively
 time-saving features to D.
 Everyone will always refer you to RefCounted, which mangles 
 your types and
 pollutes your code, but aside from that, for ARC to be useful, 
 it needs to
 be supported at the language-level, such that the 
 language/optimiser is
 able to optimise out redundant incref/decref calls, and also 
 that it is
 compatible with immutable (you can't manage a refcount if the 
 object is
 immutable).
 
 The problem isn't GC's per se. But D's horribly naive 
 implementation, games are written on GC languages now all the 
 time (Unity/.NET). And let's be honest, games are kind of a 
 speciality, games do things most programs will never do.
 
 You might want to read the GC Handbook. GC's aren't bad, but 
 most, like the D GC, are just to simplistic for common usage 
 today.
 
 Maybe a sufficiently advanced GC could address the performance 
 non-determinism to an acceptable level, but you're still left 
 with the memory non-determinism, and the conundrum that when 
 your heap approaches full (which is _always_ on a games 
 console), the GC has to work harder and harder, and more often 
 to try and keep the tiny little bit of overhead available.
 A GC heap by nature expects you to have lots of memory, and 
 also lots of FREE memory.
 
 No serious console game I'm aware of has ever been written in 
 a language with a GC. Casual games, or games that don't 
 attempt to raise the bar may get away with it, but that's not 
 the industry I work in.

You can always force the GC to run between cycles in your game, and turn off automatic sweeps. This is how most games operate nowadays. It's also probably possible to create a drop-in replacement for the GC to do something else. I could see if being *VERY* useful to make the GC take a compile-time parameter to select which GC engine is used.

Feb 03 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:
  ur right I never thought of that, I bet all them game devs 
 never thought of it either, they so dumb.  I bet they never 
 tried to use a GC, what fools!  Endless graphs of traced 
 objects, oh yes oh yes!  It only runs when I allocate, oh what 
 a fool I've been, please castigate me harder!

  Adam pls tell me more of this c# and that amazing gc it sounds 
 so good

+1
Feb 03 2014
prev sibling next sibling parent "NoUseForAName" <no spam.com> writes:
On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:
  ur right I never thought of that, I bet all them game devs 
 never thought of it either, they so dumb.  I bet they never 
 tried to use a GC, what fools!  Endless graphs of traced 
 objects, oh yes oh yes!  It only runs when I allocate, oh what 
 a fool I've been, please castigate me harder!

Also people should consider that Apple (unlike C++ game devs) did not have a tradition of contempt for GC. In fact they tried GC *before* they switched to ARC. The pro-GC camp always likes to pretend that the anti-GC one is just ignorant, rejecting GC based on prejudice not experience but Apple rejected GC based on experience. GCed Objective-C did not allow them to deliver the user experience they wanted (on mobile), because of the related latency issues. So they switched to automated ref counting. It is not in question that ref counting sacrifices throughput (compared to an advanced GC) but for interactive, user facing applications latency is much more important. You can do soft-real time with GC as long as the GC is incremental (D's is not) and you heavily rely on object reuse. That is what I am doing with LuaJIT right now and the frame rates are nice and constant indeed. However, you pay a high price for that. Object reuse means writing additional code, makes things more complex and error-prone, which is why your average app developer does not do it.. and should not have to do it. Apple had to come up with a solution which does not assume that the developers will be careful about allocations. The performance of the apps in the iOS app store are ultimately part of the user experience so ARC is the right solution because it means that your average iOS app written by Joe Coder will not have latency issues or at least less latency issues compared to any GC-based solution. I think it is an interesting decision for the D development team to make. Do you want a language which can achieve low latency *if used carefully* or one which sacrifices maximal throughput performance for less latency issues in the common case. I see no obvious answer to that. I have read D has recently been used for some server system at Facebook, ref counting usually degrades performance in that area. It is no coincidence that Java shines on the server as a high performance solution while Java is a synonym for dog slow memory hog on the desktop and mighty unpopular there because of that. The whole Java ecosystem from the VM to the libraries is optimized for enterprise server use cases, for throughput, scalability, and robustness, not for making responsive GUIs (and low latency in general) or for memory use. If D wants to be the new Java GC is the way to go, but no heap allocation happy GCed language will ever challenge C/C++ on the desktop. Which reminds me of another major company who paddled back on GC based on experience: Microsoft. Do you remember the talk back then .NET/C# were new? Microsoft totally wanted that to be the technology stack of the future "managed code" everywhere, C/C++ becoming "legacy". However, C# ended up being nothing more than Microsoft Java, shoveling enterprise CRUD in the server room. Microsoft is hosting "Going Native" conferences nowadays, declaring their present and future dedication to C++ (again) and they based the new WinRT on ref counting not GC.
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 16:24:52 -0800, NoUseForAName <no spam.com> wrote:

 On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:
  ur right I never thought of that, I bet all them game devs never  
 thought of it either, they so dumb.  I bet they never tried to use a  
 GC, what fools!  Endless graphs of traced objects, oh yes oh yes!  It  
 only runs when I allocate, oh what a fool I've been, please castigate  
 me harder!

Also people should consider that Apple (unlike C++ game devs) did not have a tradition of contempt for GC. In fact they tried GC *before* they switched to ARC. The pro-GC camp always likes to pretend that the anti-GC one is just ignorant, rejecting GC based on prejudice not experience but Apple rejected GC based on experience. GCed Objective-C did not allow them to deliver the user experience they wanted (on mobile), because of the related latency issues. So they switched to automated ref counting. It is not in question that ref counting sacrifices throughput (compared to an advanced GC) but for interactive, user facing applications latency is much more important.

That may be the case, but StackOverflow shows that ARC hasn't been panacea in Apple land either. Way to many people don't understand ARC and how to use it, and subsequently beg for help understanding heisenleaks and weak references. ARC places a higher cognitive load on the programmer than a GC does. And Android runs just fine with GC'ed apps, but ARC guys don't want to talk about Google's successes there.
 You can do soft-real time with GC as long as the GC is incremental (D's  
 is not) and you heavily rely on object reuse. That is what I am doing  
 with LuaJIT right now and the frame rates are nice and constant indeed.  
 However, you pay a high price for that. Object reuse means writing  
 additional code, makes things more complex and error-prone, which is why  
 your average app developer does not do it.. and should not have to do it.

 Apple had to come up with a solution which does not assume that the  
 developers will be careful about allocations. The performance of the  
 apps in the iOS app store are ultimately part of the user experience so  
 ARC is the right solution because it means that your average iOS app  
 written by Joe Coder will not have latency issues or at least less  
 latency issues compared to any GC-based solution.

 I think it is an interesting decision for the D development team to  
 make. Do you want a language which can achieve low latency *if used  
 carefully* or one which sacrifices maximal throughput performance for  
 less latency issues in the common case.

 I see no obvious answer to that. I have read D has recently been used  
 for some server system at Facebook, ref counting usually degrades  
 performance in that area. It is no coincidence that Java shines on the  
 server as a high performance solution while Java is a synonym for dog  
 slow memory hog on the desktop and mighty unpopular there because of  
 that. The whole Java ecosystem from the VM to the libraries is optimized  
 for enterprise server use cases, for throughput, scalability, and  
 robustness, not for making responsive GUIs (and low latency in general)  
 or for memory use.

Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely successful GUI toolkits that are not known for GC related problems. I've been working with WPF since 2005, I can say the biggest performance problem with it by far is the naive rendering of rounded corners, the GC has NEVER caused a hitch.
 If D wants to be the new Java GC is the way to go, but no heap  
 allocation happy GCed language will ever challenge C/C++ on the desktop.

So that's why nearly every desktop app (for Windows at least, but that's the overwhelming majority) that started development since .NET came out is written C#?
 Which reminds me of another major company who paddled back on GC based  
 on experience: Microsoft. Do you remember the talk back then .NET/C#  
 were new? Microsoft totally wanted that to be the technology stack of  
 the future "managed code" everywhere, C/C++ becoming "legacy". However,  
 C# ended up being nothing more than Microsoft Java, shoveling enterprise  
 CRUD in the server room. Microsoft is hosting "Going Native" conferences  
 nowadays, declaring their present and future dedication to C++ (again)  
 and they based the new WinRT on ref counting not GC.

This is primarily due to Internal Microsoft Politics rather than any desire of the Microsoft Developer Community to jettison .NET. I won't dive in to those right now, there are plenty of places on the web that tell the story better than I could. I remember sitting in the Build 2011 Keynotes thinking "They just cost themselves two years, minimum." Turns out I my estimate was low. They have tried in vain for over two years get devs to write WinRT apps using C++/CX. They've also failed miserably. I forget the exact percentage, but the overwhelming majority if WinRT apps are written in ... C#, most of the rest are VB.NET, only something like 3% of all apps in the Windows Store are C++/CX. Server apps are written almost universally in .NET languages, such that they didn't even bother making C++/CX capable of building servers. Microsoft recently had to do a mea culpa and reinvest in .NET (Build 2013). Microsoft might be trying to get us all to move to C++ with Hats, but they are failing miserably in real terms, despite what the rah-rah section of Microsoft would have you believe. At the end of the day, the overwhelming majority of the MS Dev Community it quite happy with the performance of the GC. They even gave us more performance tweaking options in the latest release. You'll not find any ARC fans in that world. And given that the MS Dev Community eclipses Apple's in real terms, that's important to note. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 00:49:04 UTC, Adam Wilson wrote:
 Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely 
 successful GUI toolkits that are not known for GC related 
 problems. I've been working with WPF since 2005, I can say the 
 biggest performance problem with it by far is the naive 
 rendering of rounded corners, the GC has NEVER caused a hitch.

According to Wikipedia: «While the majority of WPF is in managed code, the composition engine which renders the WPF applications is a native component. It is named Media Integration Layer (MIL) and resides in milcore.dll. It interfaces directly with DirectX and provides basic support for 2D and 3D surfaces, timer-controlled manipulation of contents of a surface with a view to exposing animation constructs at a higher level, and compositing the individual elements of a WPF application into a final 3D "scene" that represents the UI of the application and renders it to the screen.» So, Microsoft does not think that GC is suitable for real time interactive graphics. And they are right.
Feb 03 2014
prev sibling next sibling parent "NoUseForAName" <no spam.com> writes:
On Tuesday, 4 February 2014 at 00:49:04 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 16:24:52 -0800, NoUseForAName <no spam.com> 
 wrote:

 That may be the case, but StackOverflow shows that ARC hasn't 
 been panacea in Apple land either. Way to many people don't 
 understand ARC and how to use it, and subsequently beg for help 
 understanding heisenleaks and weak references.

Your point? ARC addressed the latency issues, I never said it was without challenges of its own.
ARC places a higher cognitive load on the programmer than a GC 
does.

Yes, it does. But the whole "not thinking about allocations" thing comes at the often unacceptable cost of unresponsive apps.
And
 Android runs just fine with GC'ed apps, but ARC guys don't want 
 to talk about Google's successes there.

Google's success there is that demanding apps are written using the NDK.
 Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely 
 successful GUI toolkits that are not known for GC related 
 problems.

Silverlight is dead and was an utter failure. WinForms and WPF have an uncertain future. Neither has ever been used much in end user applications. I would also like to say that the typical .NET or Java developer has lost all sense of what an efficient app feels like. E.g. someone who works with Eclipse all day will of course consider about everything else lightweight and snappy.
 So that's why nearly every desktop app (for Windows at least, 
 but that's the overwhelming majority) that started development 
 since .NET came out is written C#?

That is simply not true. The set of widely popular Windows desktop applications is basically .NET free. However, maybe you misunderstood me because - I admit - my phrasing was unclear. When I said "desktop" I meant end user desktop applications and games. Not enterprise/government desktop CRUD apps which are forced upon office workers who cannot reject them because of their horrible performance. I would not be surprised if most of those are indeed written in .NET (if not written in Java).
 only something like 3% of all apps in the Windows Store are 
 C++/CX.

Does anybody actually use Windows Store? Frankly I do not know anyone who does.
 Server apps are written almost universally in .NET languages

Eh.. yes. I said myself that Java and Java-likes rule that domain. Again, if D wants to compete with Java (or Microsoft's version of it) there is nothing wrong with GC.
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7bd6c620e1576304f18adf50
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 10:49, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 16:24:52 -0800, NoUseForAName <no spam.com> wrote:

  On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:
  ur right I never thought of that, I bet all them game devs never
 thought of it either, they so dumb.  I bet they never tried to use a GC,
 what fools!  Endless graphs of traced objects, oh yes oh yes!  It only runs
 when I allocate, oh what a fool I've been, please castigate me harder!

Also people should consider that Apple (unlike C++ game devs) did not have a tradition of contempt for GC. In fact they tried GC *before* they switched to ARC. The pro-GC camp always likes to pretend that the anti-GC one is just ignorant, rejecting GC based on prejudice not experience but Apple rejected GC based on experience. GCed Objective-C did not allow them to deliver the user experience they wanted (on mobile), because of the related latency issues. So they switched to automated ref counting. It is not in question that ref counting sacrifices throughput (compared to an advanced GC) but for interactive, user facing applications latency is much more important.

in Apple land either. Way to many people don't understand ARC and how to use it, and subsequently beg for help understanding heisenleaks and weak references. ARC places a higher cognitive load on the programmer than a GC does. And Android runs just fine with GC'ed apps, but ARC guys don't want to talk about Google's successes there.

I'd have trouble disagreeing more; Android is the essence of why Java should never be used for user-facing applications. Android is jerky and jittery, has random pauses and lockups all the time, and games on android always jitter and drop frames. Most high-end games on android now are written in C++ as a means to mitigate that problem, but then you're back writing C++. Yay! iOS is silky smooth by comparison to Android. I'm sure this isn't entirely attributable to the GC, or Java in general, but it can't possibly be used as an example of success. Precisely the opposite if anything. Games on Android make gamedevs who care about smooth interactivity's brains bleed. You can do soft-real time with GC as long as the GC is incremental (D's is
 not) and you heavily rely on object reuse. That is what I am doing with
 LuaJIT right now and the frame rates are nice and constant indeed. However,
 you pay a high price for that. Object reuse means writing additional code,
 makes things more complex and error-prone, which is why your average app
 developer does not do it.. and should not have to do it.

 Apple had to come up with a solution which does not assume that the
 developers will be careful about allocations. The performance of the apps
 in the iOS app store are ultimately part of the user experience so ARC is
 the right solution because it means that your average iOS app written by
 Joe Coder will not have latency issues or at least less latency issues
 compared to any GC-based solution.

 I think it is an interesting decision for the D development team to make.
 Do you want a language which can achieve low latency *if used carefully* or
 one which sacrifices maximal throughput performance for less latency issues
 in the common case.

 I see no obvious answer to that. I have read D has recently been used for
 some server system at Facebook, ref counting usually degrades performance
 in that area. It is no coincidence that Java shines on the server as a high
 performance solution while Java is a synonym for dog slow memory hog on the
 desktop and mighty unpopular there because of that. The whole Java
 ecosystem from the VM to the libraries is optimized for enterprise server
 use cases, for throughput, scalability, and robustness, not for making
 responsive GUIs (and low latency in general) or for memory use.

toolkits that are not known for GC related problems. I've been working with WPF since 2005, I can say the biggest performance problem with it by far is the naive rendering of rounded corners, the GC has NEVER caused a hitch.

On a modern many ghz PC with many cores, and many gb of ram (most of which is unallocated), a hardware virtual memory manager, and a mature RTOS. Computers come in all shapes and sizes. D is positioned as a systems language, last time I checked... or else I don't know what I'm doing here. If D wants to be the new Java GC is the way to go, but no heap allocation
 happy GCed language will ever challenge C/C++ on the desktop.

the overwhelming majority) that started development since .NET came out is written C#?

I don't think people write C# because it has a GC. People write C# because it is productive and awesome; has an amazing dev infrastructure, dev environment, well integrated GUI toolkits, debugger works awesome, docs are excellent, etc. Correlation does not imply causality. I know lots of people who love C#, even write games in it, but criticise the GC as it's biggest flaw. I'm not saying there are lots of people that love it, and for C#'s intended market, it makes perfect sense. I don't think D's market is C#'s market. If it was, I would be a happy C# developer, and I never would have given D a moments notice. --047d7bd6c620e1576304f18adf50 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 10:49, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Mon, 03 Feb 2014 16:24:52 -0800, NoUseForAName &lt;<a = href=3D"mailto:no spam.com" target=3D"_blank">no spam.com</a>&gt; wrote:<br=

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> =C2=A0ur right I never thought of that, I bet all them game devs never thou= ght of it either, they so dumb. =C2=A0I bet they never tried to use a GC, w= hat fools! =C2=A0Endless graphs of traced objects, oh yes oh yes! =C2=A0It = only runs when I allocate, oh what a fool I&#39;ve been, please castigate m= e harder!<br> </blockquote> <br> Also people should consider that Apple (unlike C++ game devs) did not have = a tradition of contempt for GC. In fact they tried GC *before* they switche= d to ARC. The pro-GC camp always likes to pretend that the anti-GC one is j= ust ignorant, rejecting GC based on prejudice not experience but Apple reje= cted GC based on experience.<br> <br> GCed Objective-C did not allow them to deliver the user experience they wan= ted (on mobile), because of the related latency issues. So they switched to= automated ref counting. It is not in question that ref counting sacrifices= throughput (compared to an advanced GC) but for interactive, user facing a= pplications latency is much more important.<br> <br> </blockquote> <br></div> That may be the case, but StackOverflow shows that ARC hasn&#39;t been pana= cea in Apple land either. Way to many people don&#39;t understand ARC and h= ow to use it, and subsequently beg for help understanding heisenleaks and w= eak references. ARC places a higher cognitive load on the programmer than a= GC does. And Android runs just fine with GC&#39;ed apps, but ARC guys don&= #39;t want to talk about Google&#39;s successes there.</blockquote> <div><br></div><div>I&#39;d have trouble disagreeing more; Android is the e= ssence of why Java should never be used for user-facing applications.</div>= <div>Android is jerky and jittery, has random pauses and lockups all the ti= me, and games on android always jitter and drop frames. Most high-end games= on android now are written in C++ as a means to mitigate that problem, but= then you&#39;re back writing C++. Yay!</div> <div>iOS is silky smooth by comparison to Android.</div><div>I&#39;m sure t= his isn&#39;t entirely attributable to the GC, or Java in general, but it c= an&#39;t possibly be used as an example of success. Precisely the opposite = if anything. Games on Android make gamedevs who care about smooth interacti= vity&#39;s brains bleed.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"><div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> You can do soft-real time with GC as long as the GC is incremental (D&#39;s= is not) and you heavily rely on object reuse. That is what I am doing with= LuaJIT right now and the frame rates are nice and constant indeed. However= , you pay a high price for that. Object reuse means writing additional code= , makes things more complex and error-prone, which is why your average app = developer does not do it.. and should not have to do it.<br> <br> Apple had to come up with a solution which does not assume that the develop= ers will be careful about allocations. The performance of the apps in the i= OS app store are ultimately part of the user experience so ARC is the right= solution because it means that your average iOS app written by Joe Coder w= ill not have latency issues or at least less latency issues compared to any= GC-based solution.<br> <br> I think it is an interesting decision for the D development team to make. D= o you want a language which can achieve low latency *if used carefully* or = one which sacrifices maximal throughput performance for less latency issues= in the common case.<br> <br> I see no obvious answer to that. I have read D has recently been used for s= ome server system at Facebook, ref counting usually degrades performance in= that area. It is no coincidence that Java shines on the server as a high p= erformance solution while Java is a synonym for dog slow memory hog on the = desktop and mighty unpopular there because of that. The whole Java ecosyste= m from the VM to the libraries is optimized for enterprise server use cases= , for throughput, scalability, and robustness, not for making responsive GU= Is (and low latency in general) or for memory use.<br> <br> </blockquote> <br></div> Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely successful GUI = toolkits that are not known for GC related problems. I&#39;ve been working = with WPF since 2005, I can say the biggest performance problem with it by f= ar is the naive rendering of rounded corners, the GC has NEVER caused a hit= ch.</blockquote> <div><br></div><div>On a modern many ghz PC with many cores, and many gb of= ram (most of which is unallocated), a hardware virtual memory manager, and= a mature RTOS.</div><div>Computers come in all shapes and sizes. D is posi= tioned as a systems language, last time I checked... or else I don&#39;t kn= ow what I&#39;m doing here.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"><div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> If D wants to be the new Java GC is the way to go, but no heap allocation h= appy GCed language will ever challenge C/C++ on the desktop.<br> <br> </blockquote> <br></div> So that&#39;s why nearly every desktop app (for Windows at least, but that&= #39;s the overwhelming majority) that started development since .NET came o= ut is written C#?</blockquote><div><br></div><div>I don&#39;t think people = write C# because it has a GC. People write C# because it is productive and = awesome; has an amazing dev infrastructure, dev environment, well integrate= d GUI toolkits, debugger works awesome, docs are excellent, etc.</div> <div>Correlation does not imply causality.</div><div>I know lots of people = who love C#, even write games in it, but criticise the GC as it&#39;s bigge= st flaw.</div><div><br></div><div>I&#39;m not saying there are lots of peop= le that love it, and for C#&#39;s intended market, it makes perfect sense. = I don&#39;t think D&#39;s market is C#&#39;s market. If it was, I would be = a happy C# developer, and I never would have given D a moments notice.</div=

--047d7bd6c620e1576304f18adf50--
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 17:48:42 -0800, NoUseForAName <no spam.com> wrote:

 On Tuesday, 4 February 2014 at 00:49:04 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 16:24:52 -0800, NoUseForAName <no spam.com> wrote:

 That may be the case, but StackOverflow shows that ARC hasn't been  
 panacea in Apple land either. Way to many people don't understand ARC  
 and how to use it, and subsequently beg for help understanding  
 heisenleaks and weak references.

Your point? ARC addressed the latency issues, I never said it was without challenges of its own.
 ARC places a higher cognitive load on the programmer than a GC does.

Yes, it does. But the whole "not thinking about allocations" thing comes at the often unacceptable cost of unresponsive apps.

So basically, all programmers who use GC's are too stupid to understand ARC and should therefore either learn it because it is categorically superior, or face the ridicule of their superior brethren? A tad unrealistic there, most use a GC for purely practical reasons, namely their workload is such that without the benefit of not having to worry about memory they'd be even further behind, and subsequently spend even more time away from family and loved-ones.
 And
 Android runs just fine with GC'ed apps, but ARC guys don't want to talk  
 about Google's successes there.

Google's success there is that demanding apps are written using the NDK.

Fair enough.
 Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely successful  
 GUI toolkits that are not known for GC related problems.

Silverlight is dead and was an utter failure. WinForms and WPF have an uncertain future. Neither has ever been used much in end user applications.

And how certain are you of this? Because that assertion doesn't match the data MS has collected.
 I would also like to say that the typical .NET or Java developer has  
 lost all sense of what an efficient app feels like. E.g. someone who  
 works with Eclipse all day will of course consider about everything else  
 lightweight and snappy.

"I don't agree with their choices or preferences so they are wrong."
 So that's why nearly every desktop app (for Windows at least, but  
 that's the overwhelming majority) that started development since .NET  
 came out is written C#?

That is simply not true. The set of widely popular Windows desktop applications is basically .NET free. However, maybe you misunderstood me because - I admit - my phrasing was unclear. When I said "desktop" I meant end user desktop applications and games. Not enterprise/government desktop CRUD apps which are forced upon office workers who cannot reject them because of their horrible performance. I would not be surprised if most of those are indeed written in .NET (if not written in Java).

So now you're tightening your definition of desktop app to something you think you can win the argument on? Not a very powerful argument. I am using the industry standard definition: A desktop app is anything that runs on a desktop OS regardless of usage. Example, Visual Studio is both a .NET/WPF app AND a desktop app.
 only something like 3% of all apps in the Windows Store are C++/CX.

Does anybody actually use Windows Store? Frankly I do not know anyone who does.

LOL, I tend to agree, but that's a strawman, and besides the point. I was merely commenting on the code makeup of the Windows Store, not it's usage.
 Server apps are written almost universally in .NET languages

Eh.. yes. I said myself that Java and Java-likes rule that domain. Again, if D wants to compete with Java (or Microsoft's version of it) there is nothing wrong with GC.

D absolutely DOES want to get compete here, otherwise we might as well evict Andrei and Facebook right now. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 18:53:39 -0800, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 4:49 PM, Adam Wilson wrote:
 That may be the case, but StackOverflow shows that ARC hasn't been
 panacea in Apple land either. Way to many people don't understand ARC
 and how to use it, and subsequently beg for help understanding
 heisenleaks and weak references. ARC places a higher cognitive load on
 the programmer than a GC does. And Android runs just fine with GC'ed
 apps, but ARC guys don't want to talk about Google's successes there.

Links please. Andrei

Easier to link a couple of Google searches: https://www.google.com/search?q=ios+stack+overflow+cyclic+reference&oq=stack+overflow+ios+cyclic+reference Note that some of those are related to circular imports as well. But the problems definitely show up in that list. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 19:00:30 -0800, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 18:53:39 -0800, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 4:49 PM, Adam Wilson wrote:
 That may be the case, but StackOverflow shows that ARC hasn't been
 panacea in Apple land either. Way to many people don't understand ARC
 and how to use it, and subsequently beg for help understanding
 heisenleaks and weak references. ARC places a higher cognitive load on
 the programmer than a GC does. And Android runs just fine with GC'ed
 apps, but ARC guys don't want to talk about Google's successes there.

Links please. Andrei

Easier to link a couple of Google searches: https://www.google.com/search?q=ios+stack+overflow+cyclic+reference&oq=stack+overflow+ios+cyclic+reference Note that some of those are related to circular imports as well. But the problems definitely show up in that list.

Note that ObjC has special syntax to handle weak pointers. It's not well understood by many. And for D it would create an explosion of leaks in existing code when there previously were none, due to the need to mark all weak-refs explicitly. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b5d3d7a25080f04f18c4416
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 12:59, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org
 wrote:

 On 2/3/14, 5:51 PM, Manu wrote:

 I'd have trouble disagreeing more; Android is the essence of why Java
 should never be used for user-facing applications.
 Android is jerky and jittery, has random pauses and lockups all the
 time, and games on android always jitter and drop frames. Most high-end
 games on android now are written in C++ as a means to mitigate that
 problem, but then you're back writing C++. Yay!
 iOS is silky smooth by comparison to Android.

Kinda difficult to explain the market success of Android.

I think it's easy to explain. 1. It's aggressively backed by the biggest technology company in the world. 2. It's free for product vendors. 3. For all product vendors at the curve of Android's success, it presented a realistic and well supported (by Google) competition to Apple, who were running away with the industry. Everybody had to compete with Apple, but didn't have the resources to realistically compete on their own. Nokia for instance were certainly in the best position to produce serious competition, but they fumbled multiple times. I suspect Google won because they're Google, and it is free. Even Microsoft failed. C# is, for all intents and purposes, the same as Java, except it's even better. If Java was a significant factor in Android's success, I'd argue that WindowsMobile should have been equally successful. I think it's safe to say, that's not the case. Personally, I suspect that Java was actually a barrier to entry in early Android, and even possibly the reason that it took as it did for Android to take root. It's most certainly the reason that Android had absolutely no games on it for so many years. They eventually released the NDK, and games finally appeared. There were years between Angry Birds success in iPhone, and any serious games appearing on Android. There were years where if you wanted to play games in your mobile device, you had to get an iDevice. It's finally levelled now that the indisputable success of Android is absolute (and the NDK is available). --047d7b5d3d7a25080f04f18c4416 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 12:59, Andrei Alexandrescu <span dir=3D"ltr">&lt;<a href=3D"= mailto:SeeWebsiteForEmail erdani.org" target=3D"_blank">SeeWebsiteForEmail = erdani.org</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"><div class=3D"im">On 2/3/14, 5:51 PM, Manu wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> I&#39;d have trouble disagreeing more; Android is the essence of why Java<b= r> should never be used for user-facing applications.<br> Android is jerky and jittery, has random pauses and lockups all the<br> time, and games on android always jitter and drop frames. Most high-end<br> games on android now are written in C++ as a means to mitigate that<br> problem, but then you&#39;re back writing C++. Yay!<br> iOS is silky smooth by comparison to Android.<br> </blockquote> <br></div> Kinda difficult to explain the market success of Android.</blockquote><div>= <br></div><div>I think it&#39;s easy to explain.</div><div>1. It&#39;s aggr= essively backed by the biggest technology company in the world.</div><div> 2. It&#39;s free for product vendors.</div><div>3. For all product vendors = at the curve of Android&#39;s success, it presented a realistic and well su= pported (by Google) competition to Apple, who were running away with the in= dustry. Everybody had to compete with Apple, but didn&#39;t have the resour= ces to realistically compete on their own. Nokia for instance were certainl= y in the best position to produce serious competition, but they fumbled mul= tiple times. I suspect Google won because they&#39;re Google, and it is fre= e.</div> <div><br></div><div>Even Microsoft failed. C# is, for all intents and purpo= ses, the same as Java, except it&#39;s even better. If Java was a significa= nt factor in Android&#39;s success, I&#39;d argue that WindowsMobile should= have been equally successful.<br> </div><div>I think it&#39;s safe to say, that&#39;s not the case.</div><div=
Personally, I suspect that Java was actually a barrier to entry in early A=

take root.</div> <div>It&#39;s most certainly the reason that Android had absolutely no game= s on it for so many years. They eventually released the NDK, and games fina= lly appeared. There were years between Angry Birds success in iPhone, and a= ny serious games appearing on Android.</div> <div>There were years where if you wanted to play games in your mobile devi= ce, you had to get an iDevice. It&#39;s finally levelled now that the indis= putable success of Android is absolute (and the NDK is available).</div> </div></div></div> --047d7b5d3d7a25080f04f18c4416--
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 22:39:02 -0800, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 2/3/2014 7:03 PM, Adam Wilson wrote:
 Note that ObjC has special syntax to handle weak pointers. It's not well
 understood by many.

Sounds like explicitly managed memory is hardly worse.

Well special syntax is needed to make ARC work because of the requirement to explicitly mark a pointer as a weak-reference. It would also mean that all existing D code would subtly break (leak memory) in any case where a weak-reference is required and was previously handled transparently by the GC. I won't say that manually managed memory is worse, because it does automatically clean up most of the trash, but ARC still allows to shot yourself in the foot easily enough. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 22:51:39 -0800, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 22:39:02 -0800, Walter Bright  
 <newshound2 digitalmars.com> wrote:

 On 2/3/2014 7:03 PM, Adam Wilson wrote:
 Note that ObjC has special syntax to handle weak pointers. It's not  
 well
 understood by many.

Sounds like explicitly managed memory is hardly worse.

Well special syntax is needed to make ARC work because of the requirement to explicitly mark a pointer as a weak-reference. It would also mean that all existing D code would subtly break (leak memory) in any case where a weak-reference is required and was previously handled transparently by the GC. I won't say that manually managed memory is worse, because it does automatically clean up most of the trash, but ARC still allows to shot yourself in the foot easily enough.

Oops. I won't say that ARC is worse than Manual Memory Management. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 23:05:35 -0800, Eric Suen <eric.suen.tech gmail.com>  
wrote:

 "Ola Fosheim Gr?stad" <ola.fosheim.grostad+dlang gmail.com>"> On  
 Tuesday, 4 February 2014 at 00:49:04 UTC, Adam Wilson
 wrote:
 Ahem. Wrong. See: WinForms, WPF, Silverlight. All extremely successful  
 GUI toolkits that are not known for GC related
 problems. I've been working with WPF since 2005, I can say the biggest  
 performance problem with it by far is the
 naive rendering of rounded corners, the GC has NEVER caused a hitch.

According to Wikipedia: ?While the majority of WPF is in managed code, the composition engine which renders the WPF applications is a native component. It is named Media Integration Layer (MIL) and resides in milcore.dll. It interfaces directly with DirectX and provides basic support for 2D and 3D surfaces, timer-controlled manipulation of contents of a surface with a view to exposing animation constructs at a higher level, and compositing the individual elements of a WPF application into a final 3D "scene" that represents the UI of the application and renders it to the screen.? So, Microsoft does not think that GC is suitable for real time interactive graphics. And they are right.

As long as other code is in managed code, there is GC running at background, no matter your code write in whatever high performance language, it will be affect by GC anyway. so that "So, Microsoft does not think that GC is suitable for real time > interactive graphics. And they are right." is only your opinion. you don't know the real reason behind MS's decision. some resource need release ASAP, so you can't rely on GC. or maybe bacause of overhead when call low API in managed code.

Actually the reason is that DirectX is a specialized Native COM-Like API that is NOT compatible with normal COM and therefore not compatible with .NET COM Interop. Milcore is a low-level immediate API. Most of the legwork in WPF is done in managed code. And yes, the GC would effect the performance regardless of whether or not the rendering API is native because so much of WPF is managed code. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 03:30:58 UTC, Manu wrote:
 On 4 February 2014 12:59, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org
 wrote:

 On 2/3/14, 5:51 PM, Manu wrote:

 I'd have trouble disagreeing more; Android is the essence of 
 why Java
 should never be used for user-facing applications.
 Android is jerky and jittery, has random pauses and lockups 
 all the
 time, and games on android always jitter and drop frames. 
 Most high-end
 games on android now are written in C++ as a means to 
 mitigate that
 problem, but then you're back writing C++. Yay!
 iOS is silky smooth by comparison to Android.

Kinda difficult to explain the market success of Android.

I think it's easy to explain. 1. It's aggressively backed by the biggest technology company in the world. 2. It's free for product vendors. 3. For all product vendors at the curve of Android's success, it presented a realistic and well supported (by Google) competition to Apple, who were running away with the industry. Everybody had to compete with Apple, but didn't have the resources to realistically compete on their own. Nokia for instance were certainly in the best position to produce serious competition, but they fumbled multiple times. I suspect Google won because they're Google, and it is free. Even Microsoft failed. C# is, for all intents and purposes, the same as Java, except it's even better. If Java was a significant factor in Android's success, I'd argue that WindowsMobile should have been equally successful. I think it's safe to say, that's not the case. Personally, I suspect that Java was actually a barrier to entry in early Android, and even possibly the reason that it took as it did for Android to take root. It's most certainly the reason that Android had absolutely no games on it for so many years. They eventually released the NDK, and games finally appeared. There were years between Angry Birds success in iPhone, and any serious games appearing on Android. There were years where if you wanted to play games in your mobile device, you had to get an iDevice. It's finally levelled now that the indisputable success of Android is absolute (and the NDK is available).

You forget to mention that the NDK has a very restrained set of APIs. If you want to do interact with the OS besides audio and OpenGL, it is JNI Ad nauseam because Google sees the NDK as a minor inconvenience and all APIs are Java based. Even Java native methods need to be called via their Java class and are not accessible to NDK code. -- Paulo
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 07:06:07 UTC, Eric Suen wrote:
 As long as other code is in managed code, there is GC running at
  background, no matter your code write in whatever high 
 performance
 language, it will be affect by GC anyway.  so that "So, 
 Microsoft does
 not think that GC is suitable for real time > interactive 
 graphics.
 And they are right." is only your opinion. you don't know the 
 real reason
 behind MS's decision. some resource need release ASAP, so you 
 can't
 rely on GC. or maybe bacause of overhead when call low API in 
 managed
 code.

Anyone who has tried to use Direct 3D from managed code knows why. They had the foundation to do everything from manage code, but chose not to. You can get smooth animations in GC javascript on browsers too, but only if relying on the C++ CSS transition engine which does the frame-by-frame interpolation for you. But I expect javascript to get better at this if/when using true isolates. If the memory pool is small enough GC can work out ok, but for very low latency/overhead you use the stack/pools, not even malloc is good enough then.
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 07:15:17 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 23:05:35 -0800, Eric Suen 
 <eric.suen.tech gmail.com> wrote:

 "Ola Fosheim Gr?stad" <ola.fosheim.grostad+dlang gmail.com>">


 So, Microsoft does not think that GC is suitable for real 
 time interactive graphics. And they are right.



 And they are right." is only your opinion. you don't know the 
 real reason
 behind MS's decision. some resource need release ASAP, so you


 Actually the reason is that DirectX is a specialized Native 
 COM-Like API that is NOT compatible with normal COM and 
 therefore not compatible with .NET COM Interop. Milcore is a

Quoting http://msdn.microsoft.com/en-us/library/ms750441(v=vs.110).aspx : «Milcore is written in unmanaged code in order to enable tight integration with DirectX. All display in WPF is done through the DirectX engine, allowing for efficient hardware and software rendering. WPF also required fine control over memory and execution. The composition engine in milcore is extremely performance sensitive, and required giving up many advantages of the CLR to gain performance.» Please note that Microsoft knows their stuff: «WPF also required fine control over memory and execution.» and «required giving up many advantages of the CLR to gain performance». WPF maintains a retained mode shadow-tree of the composition elements of the scene graph to get more responsive applications. i.e. to avoid having client code blocking rendering: «There is a very important architectural detail to notice here – the entire tree of visuals and drawing instructions is cached. In graphics terms, WPF uses a retained rendering system. This enables the system to repaint at high refresh rates without the composition system blocking on callbacks to user code. This helps prevent the appearance of an unresponsive application.» But… I don't think it was a good idea to go for full back-to-front rendering.
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 4 February 2014 at 02:59:11 UTC, Andrei Alexandrescu 
wrote:
 On 2/3/14, 5:51 PM, Manu wrote:
 I'd have trouble disagreeing more; Android is the essence of 
 why Java
 should never be used for user-facing applications.
 Android is jerky and jittery, has random pauses and lockups 
 all the
 time, and games on android always jitter and drop frames. Most 
 high-end
 games on android now are written in C++ as a means to mitigate 
 that
 problem, but then you're back writing C++. Yay!
 iOS is silky smooth by comparison to Android.

Kinda difficult to explain the market success of Android. Andrei

Market success is probably 10% about technology and 90% about PR, filling the niche, finding right time etc.
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
There is a lot of discussion ongoing about ARC vs GC but in 
practice forcing either of those is unacceptable. Language that 
is strongly coupled with hard-coded memory model will inevitably 
fail in some domain.

For me perfect solution would have been to use an allocator 
concept as language basis instead and let you chose any 
conformant allocator for built-in language features. With both GC 
and ARC available in Phobos / druntime.

Bottom-Up design is simply superior to Top-Down when it comes to 
tools.
Feb 04 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
 There is a lot of discussion ongoing about ARC vs GC but in 
 practice forcing either of those is unacceptable. Language that 
 is strongly coupled with hard-coded memory model will 
 inevitably fail in some domain.

 For me perfect solution would have been to use an allocator 
 concept as language basis instead and let you chose any 
 conformant allocator for built-in language features. With both 
 GC and ARC available in Phobos / druntime.

 Bottom-Up design is simply superior to Top-Down when it comes 
 to tools.

And who could disagree with that? Yet we have many ;/ I guess it's all about fear. There are people that use the GC and think they will never need more. They think "changing it" will cause them to have to learn something new, which is difficult for them. Hence they rather keep their world the same than allow someone else the satisfaction of making things better for them. What they don't realize is that their world won't change... so maybe it's a bit of ignorance too?
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
 For me perfect solution would have been to use an allocator 
 concept as language basis instead and let you chose any 
 conformant allocator for built-in language features. With both 
 GC and ARC available in Phobos / druntime.

That is exactly what Rust did succsessfully. http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/ got pulled. They deprecated the pointer, which is a managed poiter equivalent to what D's 'new' produces, to a library solution. They now have basically only ~T and &T left in the language. ~T being an owned pointer that frees at the end of its scope and &T, a borrowed pointer (or reference, but *not* RC) with no direct effect on memory management. There is Rc<T> and Gc<T> in the library for those who wish to use them. To quote from the above link:


available with ~ and some operations are available with . Actually, we were confused on this point for a long time as well—it wasn’t clear whether ~ or would become dominant. We debated for a long time which to present first, ~ or . However, as the language and community evolved, and coding standards became more settled, a clear winner emerged: the owning pointer ~. In practice, the rule has been that programmers should use ~ to allocate in all circumstances except when they have no way of knowing precisely when the object in question should be freed. << This is what happens if you give users choice. One of Rust's priorities is to have a runtime footprint as small as possible. I would give priority to not break existing D code. So keep GC'd 'new' in the language (the runtime will never get as small as Rust's; don't care) and see if it is possible to add owned and ARC pointers from there.
Feb 04 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 14:19:36 UTC, Frank Bauer wrote:
 On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
 For me perfect solution would have been to use an allocator 
 concept as language basis instead and let you chose any 
 conformant allocator for built-in language features. With both 
 GC and ARC available in Phobos / druntime.

That is exactly what Rust did succsessfully. http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/ got pulled. They deprecated the pointer, which is a managed poiter equivalent to what D's 'new' produces, to a library solution. They now have basically only ~T and &T left in the language. ~T being an owned pointer that frees at the end of its scope and &T, a borrowed pointer (or reference, but *not* RC) with no direct effect on memory management. There is Rc<T> and Gc<T> in the library for those who wish to use them. To quote from the above link:


available with ~ and some operations are available with . Actually, we were confused on this point for a long time as well—it wasn’t clear whether ~ or would become dominant. We debated for a long time which to present first, ~ or . However, as the language and community evolved, and coding standards became more settled, a clear winner emerged: the owning pointer ~. In practice, the rule has been that programmers should use ~ to allocate in all circumstances except when they have no way of knowing precisely when the object in question should be freed. << This is what happens if you give users choice. One of Rust's priorities is to have a runtime footprint as small as possible. I would give priority to not break existing D code. So keep GC'd 'new' in the language (the runtime will never get as small as Rust's; don't care) and see if it is possible to add owned and ARC pointers from there.

Rust still remains to prove itself on the mainstream market. While D already has commercial users.
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 4 February 2014 at 14:22:41 UTC, Paulo Pinto wrote:
 Rust still remains to prove itself on the mainstream market.

 While D already has commercial users.

It is dangerous position. D is not that far ahead to appeal to own authority. "..it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!"
Feb 04 2014
prev sibling next sibling parent "woh" <wojw yahoo.com> writes:
unique is best, should be default, down with GC!!


Tuesday, 4 February 2014 at 14:19:36 UTC, Frank Bauer wrote:
 On Tuesday, 4 February 2014 at 13:18:51 UTC, Dicebot wrote:
 For me perfect solution would have been to use an allocator 
 concept as language basis instead and let you chose any 
 conformant allocator for built-in language features. With both 
 GC and ARC available in Phobos / druntime.

That is exactly what Rust did succsessfully. http://pcwalton.github.io/blog/2013/06/02/removing-garbage-collection-from-the-rust-language/ got pulled. They deprecated the pointer, which is a managed poiter equivalent to what D's 'new' produces, to a library solution. They now have basically only ~T and &T left in the language. ~T being an owned pointer that frees at the end of its scope and &T, a borrowed pointer (or reference, but *not* RC) with no direct effect on memory management. There is Rc<T> and Gc<T> in the library for those who wish to use them. To quote from the above link:


available with ~ and some operations are available with . Actually, we were confused on this point for a long time as well—it wasn’t clear whether ~ or would become dominant. We debated for a long time which to present first, ~ or . However, as the language and community evolved, and coding standards became more settled, a clear winner emerged: the owning pointer ~. In practice, the rule has been that programmers should use ~ to allocate in all circumstances except when they have no way of knowing precisely when the object in question should be freed. << This is what happens if you give users choice. One of Rust's priorities is to have a runtime footprint as small as possible. I would give priority to not break existing D code. So keep GC'd 'new' in the language (the runtime will never get as small as Rust's; don't care) and see if it is possible to add owned and ARC pointers from there.

Feb 04 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 03 Feb 2014 19:49:04 -0500, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 16:24:52 -0800, NoUseForAName <no spam.com> wrote:

 On Monday, 3 February 2014 at 23:00:23 UTC, woh wrote:
  ur right I never thought of that, I bet all them game devs never  
 thought of it either, they so dumb.  I bet they never tried to use a  
 GC, what fools!  Endless graphs of traced objects, oh yes oh yes!  It  
 only runs when I allocate, oh what a fool I've been, please castigate  
 me harder!

Also people should consider that Apple (unlike C++ game devs) did not have a tradition of contempt for GC. In fact they tried GC *before* they switched to ARC. The pro-GC camp always likes to pretend that the anti-GC one is just ignorant, rejecting GC based on prejudice not experience but Apple rejected GC based on experience. GCed Objective-C did not allow them to deliver the user experience they wanted (on mobile), because of the related latency issues. So they switched to automated ref counting. It is not in question that ref counting sacrifices throughput (compared to an advanced GC) but for interactive, user facing applications latency is much more important.

That may be the case, but StackOverflow shows that ARC hasn't been panacea in Apple land either. Way to many people don't understand ARC and how to use it, and subsequently beg for help understanding heisenleaks and weak references. ARC places a higher cognitive load on the programmer than a GC does. And Android runs just fine with GC'ed apps, but ARC guys don't want to talk about Google's successes there.

Where you have to be cognizant is avoiding cycles. Plain and simple. And it's not that difficult. The compiler takes care of the rest. It's somewhat magical I suppose :) Managing your autorelease pools can make a difference, but is not necessarily critical. I think when working on embedded systems, it is quite important to understand the limitations and strengths of the language/hardware you are using, much more so than on a full PC/Server. Where ARC really beats the GC is on memory usage. It's very very close to malloc/free usage. I would love to see D attain an ARC system, especially if it can link seamlessly with Objective-C. What I don't know is if one can replace a GC with an ARC memory manager without having to port any high-level code. I don't think that is the case. Which leads me to think -- is it even possible to write druntime/phobos in an agnostic way? If it isn't, what subset can be done that way? -Steve
Feb 04 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 04 Feb 2014 14:31:13 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 2/4/14, 11:28 AM, Steven Schveighoffer wrote:
 Where you have to be cognizant is avoiding cycles. Plain and simple. And
 it's not that difficult.

Do you have evidence to back that up?

Does personal experience count? I can say that the main project I have worked on in Objective-C uses manual reference counting, and when I have done newer, but shorter/smaller, projects in ARC, it is much much easier to deal with. Maybe my point of reference is tainted :) If you use xcode to add member variables, it usually chooses the correct attributes for them (weak or not). But the code the compiler generates for ARC is pretty good. It has quirky rules for naming methods and how you have to return objects (either autorelease or retained), and will not let you do the wrong thing. You also can't do ANY manual memory management in ARC code, it fails to compile. Trickier aspects are dealing with C structs, which are NOT reference counted, so you have to deal with those manually using malloc/free (not really different from D). I will say that it really helps to have an understanding of how ARC is actually implemented in the compiler (that is, the process it goes through to add/elide reference increments and decrements). -Steve
Feb 04 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 4 February 2014 at 19:28:08 UTC, Steven Schveighoffer 
wrote:
 Where you have to be cognizant is avoiding cycles. Plain and 
 simple. And it's not that difficult. The compiler takes care of 
 the rest. It's somewhat magical I suppose :) Managing your 
 autorelease pools can make a difference, but is not necessarily 
 critical.

My experience is that, just like null reference, it is highly dependent on the type of code you are facing. Typically, you'll have trouble with complex graph of objects.
Feb 04 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 04 Feb 2014 15:10:48 -0500, deadalnix <deadalnix gmail.com> wrote:

 On Tuesday, 4 February 2014 at 19:28:08 UTC, Steven Schveighoffer wrote:
 Where you have to be cognizant is avoiding cycles. Plain and simple.  
 And it's not that difficult. The compiler takes care of the rest. It's  
 somewhat magical I suppose :) Managing your autorelease pools can make  
 a difference, but is not necessarily critical.

My experience is that, just like null reference, it is highly dependent on the type of code you are facing. Typically, you'll have trouble with complex graph of objects.

This is very true. I have only dealt with acyclic (except for parent pointers) graphs or linked-lists. For specialized cases, you will have to NULL out the pointers to/from a node that has to be deleted (essentially a manual delete). But it works VERY well for linked-lists/queues. My app is I/O intensive, so I have a lot of that. -Steve
Feb 04 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 4 February 2014 at 20:16:20 UTC, Steven Schveighoffer 
wrote:
 On Tue, 04 Feb 2014 15:10:48 -0500, deadalnix 
 <deadalnix gmail.com> wrote:

 On Tuesday, 4 February 2014 at 19:28:08 UTC, Steven 
 Schveighoffer wrote:
 Where you have to be cognizant is avoiding cycles. Plain and 
 simple. And it's not that difficult. The compiler takes care 
 of the rest. It's somewhat magical I suppose :) Managing your 
 autorelease pools can make a difference, but is not 
 necessarily critical.

My experience is that, just like null reference, it is highly dependent on the type of code you are facing. Typically, you'll have trouble with complex graph of objects.

This is very true. I have only dealt with acyclic (except for parent pointers) graphs or linked-lists. For specialized cases, you will have to NULL out the pointers to/from a node that has to be deleted (essentially a manual delete). But it works VERY well for linked-lists/queues. My app is I/O intensive, so I have a lot of that. -Steve

Yes. Also, another issue comes up when you share data across threads. Core can share a cache line in read mode, but can't in write mode. That mean that updating the reference count will cause contention on the cache line (core will have to fight for the cache line ownership). That is why immutability + GC is so efficient in a highly concurrent system, and ref counting would ruin that.
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 21:02:37 UTC, deadalnix wrote:
 Core can share a cache line in read mode, but can't in write 
 mode.

Why not? Level 3 is shared, but it has a latency of 30-40 cycles or so.
 That mean that updating the reference count will cause 
 contention on the cache line (core will have to fight for the 
 cache line ownership).

If they access it simultaneously and it stays in the local caches.
 That is why immutability + GC is so efficient in a highly 
 concurrent system, and ref counting would ruin that.

If you blindly use ARC rather than sane RC. There is no reason to up the ref count if the data structure is "owned" while processing it. Which is a good reason to avoid ARC and use regular RC. If you know that the entire graph has a 1+ count, you don't need to do any ref counting while processing it.
Feb 04 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 04 Feb 2014 16:37:20 -0500, Ola Fosheim Gr=C3=B8stad  =

<ola.fosheim.grostad+dlang gmail.com> wrote:

 If you blindly use ARC rather than sane RC. There is no reason to up t=

 ref count if the data structure is "owned" while processing it. Which =

 a good reason to avoid ARC and use regular RC. If you know that the  =

 entire graph has a 1+ count, you don't need to do any ref counting whi=

 processing it.

This is an important point. However, the default is to be safe, and good= = enough for most. As long as there is a way to go into "I know what I'm = doing" mode for encapsulated low-level code, the default should be to do= = ARC. ARC should handle simple cases, like if the reference only ever = exists as a stack local. One could also say you could mark a memory region as no-scan if you know= = that the data it points at will never be collected for its entire = existence in a GC environment. I think some mix of GC and ref counting for D would be extremely useful.= In my major Objective C project, I use manual reference counting, and I = do = avoid extra retains for redundant pointers. I've also had a few bugs = because of that :) -Steve
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 22:17:40 UTC, Steven Schveighoffer 
wrote:
 This is an important point. However, the default is to be safe, 
 and good enough for most.

Yes, absolutely. I am all for progressive refinement, starting out with something conceptual and very close to pseudocode, with all the convenience of high level programming. Then replace the parts that does not perform. In Objective-C you don't really care that much about performance though, Cocoa provides very versatile libraries doing a lot of the work for you, but you have to go one step down to get speed without libraries. Inefficiencies are less noticeable in a heavy library/premade-components environment like Cococa than in a more barebones hit-the-iron-C-like approach.
 One could also say you could mark a memory region as no-scan if 
 you know that the data it points at will never be collected for 
 its entire existence in a GC environment.

Yes, I agree. You could have the same type in different pools and mark one of the pools as no-scan. You could have pool-local GC/segmented GC or even type-limited GC with whole program analysis (only collecting a specific class/subclasses).
 I think some mix of GC and ref counting for D would be 
 extremely useful.

Yes, I don't want to throw out GC, but I think it would be more valuable and more realistic to have C++ reference semantics than advanced compiler inferred ARC, though I don't mind ARC, I think it probably is harder to get right for a more pure C systems level language than for Objective-C with Cocoa. So, I think ARC is right for Cocoa, but not so sure about how well it works in other environments.
Feb 04 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 04 Feb 2014 17:27:46 -0500, Paulo Pinto <pjmlp progtools.org>  =

wrote:

 Am 04.02.2014 23:17, schrieb Steven Schveighoffer:
 On Tue, 04 Feb 2014 16:37:20 -0500, Ola Fosheim Gr=C3=B8stad
 <ola.fosheim.grostad+dlang gmail.com> wrote:

 If you blindly use ARC rather than sane RC. There is no reason to up=



 the ref count if the data structure is "owned" while processing it.
 Which is a good reason to avoid ARC and use regular RC. If you know
 that the entire graph has a 1+ count, you don't need to do any ref
 counting while processing it.

This is an important point. However, the default is to be safe, and g=


 enough for most. As long as there is a way to go into "I know what I'=


 doing" mode for encapsulated low-level code, the default should be to=


 ARC. ARC should handle simple cases, like if the reference only ever
 exists as a stack local.

 One could also say you could mark a memory region as no-scan if you k=


 that the data it points at will never be collected for its entire
 existence in a GC environment.

 I think some mix of GC and ref counting for D would be extremely usef=


 In my major Objective C project, I use manual reference counting, and=


 do avoid extra retains for redundant pointers. I've also had a few bu=


 because of that :)

How big is your team?

2
 Based on my experience, this type of coding falls apart in the typical=

 corporate team size.

It's not clear what "this type of coding" refers to, but I'll assume you= = targeted the last paragraph. If it's encapsulated (and it is), it should be relatively untouched. In = = fact, I haven't touched the linked list part of it for a long time. Note that the MRC was not really our choice, the project existed before = = ARC did. -Steve
Feb 04 2014
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Wednesday, 5 February 2014 at 00:12:11 UTC, Shammah Chancellor 
wrote:
 On 2014-02-02 17:30:27 +0000, Namespace said:

 Sounds good. But why  nullable instead of C# choice of "Type?" 
 ?

nullable is not supposed to do the same thing as "Type?" (or rather the syntatic sugar for Nullabe<Type> template). Nullable<type> is to enable Value types to be null. For example: Nullable<Double> nullable in D is supposed to allow reference types to contain null. In C# reference types are ALWAYS allowed to be null. -S

class Foo { } Foo? f; It's the same.
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 21:02:37 UTC, deadalnix wrote:
 Core can share a cache line in read mode, but can't in write 
 mode. That mean that updating the reference count will cause 
 contention on the cache line (core will have to fight for the 
 cache line ownership). That is why immutability + GC is so 
 efficient in a highly concurrent system, and ref counting would 
 ruin that.

Just curious, maybe I got it a bit wrong in my previous reply. It is indeed possible to set the memory type on some CPUs such that you don't get a penalty using some special MTRR (memory type range registers). If the memory type is write-back (regular memory) you get a 43 cycle latency if the cache line might be in multiple local caches and is not dirty (read), 60+ cycle latency if it is dirty (write), and 20-30 cycles latency on local miss, but L3 hit. (if I got that right) But does D actually make sure that immutable types sit in non-write-back memory marked by MTRR?
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 5 February 2014 at 00:53:06 UTC, Ola Fosheim 
Grøstad wrote:
 But does D actually make sure that immutable types sit in 
 non-write-back memory marked by MTRR?

Guess I have to end this monologue by concluding that changing the PAT is the only reasonable way I have found so far for doing this. Which is too expensive for short-lived data. So this strategy of turning immutable into non-WB memory type in order to avoid cache coherency latencies are only going to work in a OS-free environment AFAIK. (Like BareMetal OS)
Feb 04 2014
prev sibling next sibling parent "renoX" <renozyx gmail.com> writes:
On Tuesday, 4 February 2014 at 02:59:11 UTC, Andrei Alexandrescu 
wrote:
 On 2/3/14, 5:51 PM, Manu wrote:
 I'd have trouble disagreeing more; Android is the essence of 
 why Java
 should never be used for user-facing applications.
 Android is jerky and jittery, has random pauses and lockups 
 all the
 time, and games on android always jitter and drop frames. Most 
 high-end
 games on android now are written in C++ as a means to mitigate 
 that
 problem, but then you're back writing C++. Yay!
 iOS is silky smooth by comparison to Android.

Kinda difficult to explain the market success of Android. Andrei

Are you agreeing or disagreeing with his point? I don't know about other Android phones, but on my Galaxy S3 what he says is true and just the other day I had to reboot my phone as it was very, very slow, the reboot made it "fast" again.. I don't like Apple (nor Microsoft), so I'm stuck with Android, but this doesn't mean that I consider that Android is good.. For me it's like Windows in the early days, not very good but usable. renoX
Feb 05 2014
prev sibling parent "Shammah Chancellor" <anonymous coward.com> writes:
On Wednesday, 5 February 2014 at 00:16:19 UTC, Namespace wrote:
 On Wednesday, 5 February 2014 at 00:12:11 UTC, Shammah 
 Chancellor wrote:
 On 2014-02-02 17:30:27 +0000, Namespace said:

 Sounds good. But why  nullable instead of C# choice of 
 "Type?" ?

nullable is not supposed to do the same thing as "Type?" (or rather the syntatic sugar for Nullabe<Type> template). Nullable<type> is to enable Value types to be null. For example: Nullable<Double> nullable in D is supposed to allow reference types to contain null. In C# reference types are ALWAYS allowed to be null. -S

class Foo { } Foo? f; It's the same.

No it's not. In C#, Foo f can always be null. Foo? doesn't mean anything. http://msdn.microsoft.com/en-us/library/system.nullable(v=vs.110).aspx "A type is said to be nullable if it can be assigned a value or can be assigned null, which means the type has no value whatsoever. By default, all reference types, such as String, are nullable, but all value types, such as Int32, are not. In C# and Visual Basic, you mark a value type as nullable by using the ? notation after the value type. For example, int? in C# or Integer? in Visual Basic declares an integer value type that can be assigned null. The Nullable class provides complementary support for the Nullable<T> structure. The Nullable class supports obtaining the underlying type of a nullable type, and comparison and equality operations on pairs of nullable types whose underlying value type does not support generic comparison and equality operations."
Feb 05 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:

 On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:

  std.typecons.RefCounted!T
 core.memory.GC.disable();

Wow. That was easy. I see, D's claim of being a multi-paradigm language is not false.

It's not a realistic suggestion. Everything you want to link uses the GC, and the language its self also uses the GC. Unless you write software in complete isolation and forego many valuable features, it's not a solution. Phobos does rely on the GC to some extent. Most algorithms and ranges do
 not though.

Running (library) code that was written with GC in mind and turning GC off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help!

I've been trying to learn to love the GC for as long as I've been around here. I really wanted to break that mental barrier, but it hasn't happened. In fact, I am more than ever convinced that the GC won't do. My current #1 wishlist item for D is the ability to use a reference counted collector in place of the built-in GC. You're not alone :) I write realtime and memory-constrained software (console games), and for me, I think the biggest issue that can never be solved is the non-deterministic nature of the collect cycles, and the unknowable memory footprint of the application. You can't make any guarantees or predictions about the GC, which is fundamentally incompatible with realtime software. Language-level ARC would probably do quite nicely for the miscellaneous allocations. Obviously, bulk allocations are still usually best handled in a context sensitive manner; ie, regions/pools/freelists/whatever, but the convenience of the GC paradigm does offer some interesting and massively time-saving features to D. Everyone will always refer you to RefCounted, which mangles your types and pollutes your code, but aside from that, for ARC to be useful, it needs to be supported at the language-level, such that the language/optimiser is able to optimise out redundant incref/decref calls, and also that it is compatible with immutable (you can't manage a refcount if the object is immutable).

The problem isn't GC's per se. But D's horribly naive implementation, games are written on GC languages now all the time (Unity/.NET). And let's be honest, games are kind of a speciality, games do things most programs will never do. You might want to read the GC Handbook. GC's aren't bad, but most, like the D GC, are just to simplistic for common usage today. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Jan 31 2014
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
--089e013cba6297ec6704f1535673
Content-Type: text/plain; charset=UTF-8

On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> wrote:

 On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:

  On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:
   std.typecons.RefCounted!T
 core.memory.GC.disable();

I see, D's claim of being a multi-paradigm language is not false.

It's not a realistic suggestion. Everything you want to link uses the GC, and the language its self also uses the GC. Unless you write software in complete isolation and forego many valuable features, it's not a solution. Phobos does rely on the GC to some extent. Most algorithms and ranges do
 not though.

off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help!

here. I really wanted to break that mental barrier, but it hasn't happened. In fact, I am more than ever convinced that the GC won't do. My current #1 wishlist item for D is the ability to use a reference counted collector in place of the built-in GC. You're not alone :) I write realtime and memory-constrained software (console games), and for me, I think the biggest issue that can never be solved is the non-deterministic nature of the collect cycles, and the unknowable memory footprint of the application. You can't make any guarantees or predictions about the GC, which is fundamentally incompatible with realtime software. Language-level ARC would probably do quite nicely for the miscellaneous allocations. Obviously, bulk allocations are still usually best handled in a context sensitive manner; ie, regions/pools/freelists/whatever, but the convenience of the GC paradigm does offer some interesting and massively time-saving features to D. Everyone will always refer you to RefCounted, which mangles your types and pollutes your code, but aside from that, for ARC to be useful, it needs to be supported at the language-level, such that the language/optimiser is able to optimise out redundant incref/decref calls, and also that it is compatible with immutable (you can't manage a refcount if the object is immutable).

The problem isn't GC's per se. But D's horribly naive implementation, games are written on GC languages now all the time (Unity/.NET). And let's be honest, games are kind of a speciality, games do things most programs will never do. You might want to read the GC Handbook. GC's aren't bad, but most, like the D GC, are just to simplistic for common usage today.

Maybe a sufficiently advanced GC could address the performance non-determinism to an acceptable level, but you're still left with the memory non-determinism, and the conundrum that when your heap approaches full (which is _always_ on a games console), the GC has to work harder and harder, and more often to try and keep the tiny little bit of overhead available. A GC heap by nature expects you to have lots of memory, and also lots of FREE memory. No serious console game I'm aware of has ever been written in a language with a GC. Casual games, or games that don't attempt to raise the bar may get away with it, but that's not the industry I work in. --089e013cba6297ec6704f1535673 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= February 2014 16:26, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Fri, 31 Jan 2014 21:29:04 -0800, Manu &lt;<a href=3D"m= ailto:turkeyman gmail.com" target=3D"_blank">turkeyman gmail.com</a>&gt; wr= ote:<br> <br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On 26 December 2012 00:48, Sven Over &lt;<a href=3D"mailto:dlang svenover.d= e" target=3D"_blank">dlang svenover.de</a>&gt; wrote:<br> <br> </div><div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> =C2=A0std.typecons.RefCounted!T<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> core.memory.GC.disable();<br> <br> </blockquote> <br> Wow. That was easy.<br> <br> I see, D&#39;s claim of being a multi-paradigm language is not false.<br> </blockquote> <br> <br></div><div class=3D"im"> It&#39;s not a realistic suggestion. Everything you want to link uses the G= C,<br> and the language its self also uses the GC. Unless you write software in<br=

on.<br> <br> <br></div><div class=3D"im"> =C2=A0Phobos does rely on the GC to some extent. Most algorithms and ranges= do<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><blockquote class=3D"gmail_quote" style=3D"m= argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> not though.<br> <br> </blockquote> <br> Running (library) code that was written with GC in mind and turning GC off<= br> doesn&#39;t sound ideal.<br> <br> But maybe this allows me to familiarise myself more with D. Who knows,<br> maybe I can learn to stop worrying and love garbage collection.<br> <br> Thanks for your help!<br> <br> </blockquote> <br></div><div class=3D"im"> I&#39;ve been trying to learn to love the GC for as long as I&#39;ve been a= round<br> here. I really wanted to break that mental barrier, but it hasn&#39;t happe= ned.<br> In fact, I am more than ever convinced that the GC won&#39;t do. My current= #1<br> wishlist item for D is the ability to use a reference counted collector in<= br> place of the built-in GC.<br> You&#39;re not alone :)<br> <br> I write realtime and memory-constrained software (console games), and for<b= r> me, I think the biggest issue that can never be solved is the<br> non-deterministic nature of the collect cycles, and the unknowable memory<b= r> footprint of the application. You can&#39;t make any guarantees or predicti= ons<br> about the GC, which is fundamentally incompatible with realtime software.<b= r> Language-level ARC would probably do quite nicely for the miscellaneous<br> allocations. Obviously, bulk allocations are still usually best handled in<= br> a context sensitive manner; ie, regions/pools/freelists/<u></u>whatever, bu= t the<br> convenience of the GC paradigm does offer some interesting and massively<br=

Everyone will always refer you to RefCounted, which mangles your types and<= br> pollutes your code, but aside from that, for ARC to be useful, it needs to<= br> be supported at the language-level, such that the language/optimiser is<br> able to optimise out redundant incref/decref calls, and also that it is<br> compatible with immutable (you can&#39;t manage a refcount if the object is= <br> immutable).<br> </div></blockquote> <br> The problem isn&#39;t GC&#39;s per se. But D&#39;s horribly naive implement= ation, games are written on GC languages now all the time (Unity/.NET). And= let&#39;s be honest, games are kind of a speciality, games do things most = programs will never do.<br> <br> You might want to read the GC Handbook. GC&#39;s aren&#39;t bad, but most, = like the D GC, are just to simplistic for common usage today.</blockquote><= div><br></div><div>Maybe a sufficiently advanced GC could address the perfo= rmance non-determinism to an acceptable level, but you&#39;re still left wi= th the memory non-determinism, and the conundrum that when your heap approa= ches full (which is _always_ on a games console), the GC has to work harder= and harder, and more often to try and keep the tiny little bit of overhead= available.</div> <div>A GC heap by nature expects you to have lots of memory, and also lots = of FREE memory.</div><div><br></div><div>No serious console game I&#39;m aw= are of has ever been written in a language with a GC. Casual games, or game= s that don&#39;t attempt to raise the bar may get away with it, but that&#3= 9;s not the industry I work in.</div> </div></div></div> --089e013cba6297ec6704f1535673--
Jan 31 2014
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Feb 03, 2014 at 12:52:53PM -0800, Adam Wilson wrote:
 On Mon, 03 Feb 2014 12:40:20 -0800, Dmitry Olshansky
 <dmitry.olsh gmail.com> wrote:

I won't be surprised that half of current GC problems are because
it's simply too stupid as an allocator. I had some numbers that
I'd need to dig up were GC.malloc was ~2x slower then malloc even
with garbage collection disabled.

With that said I'd focus on ref-counting somehow coexisting with
tracing GC. It doesn't go well at the moment - try creating
dynamic arrays of std.stdio.File (that is ref-counted).

I will not defend the current GC, it is about as stupid as you can make one and still have a functioning memory management system. It implements exactly none of the optimizations recommended for Mark-Sweep GC's in the GC Handbook. That said, I firmly believe that wholesale replacement of the GC is throwing the baby out with the bathwater. Effectively, the poor D GC implementation has become an excuse to launch a crusade against all GC's everywhere, never mind that Java and the .NET GC's are consistent examples of just how good GC's can actually be.

I vote for the wholesale replacement of the GC... with a better GC. :-P Preferably one that is on par with modern Java GC's. I agree that the GC phobia of some people is a little overboard. It's what they need for their specific use case, but that doesn't really apply in other use cases for D. (And I'm not convinced that ARC is the be-all and end-all of memory management issues -- I've seen real-life examples of ARCs gone horribly, horribly wrong, whereas had a GC been used in the first place things wouldn't have gone down that route.) Nevertheless, supporting both can't be a bad thing; after all, D is all about choice (it's a breath of fresh air to *not* have to shoehorn every square peg into the OO round hole, like you have to in Java), so having the choice to *not* use the GC is a net improvement IMO, even if most D users wouldn't care about the difference. T -- Маленькие детки - маленькие бедки.
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--f46d044519b9c1078904f18bfe5d
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 12:05, Nick Sabalausky <
SeeWebsiteToContactMe semitwist.com> wrote:

 On 2/3/2014 4:13 PM, H. S. Teoh wrote:

 I've seen real-life
 examples of ARCs gone horribly, horribly wrong, whereas had a GC been
 used in the first place things wouldn't have gone down that route.


Me too. --f46d044519b9c1078904f18bfe5d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 12:05, Nick Sabalausky <span dir=3D"ltr">&lt;<a href=3D"mail= to:SeeWebsiteToContactMe semitwist.com" target=3D"_blank">SeeWebsiteToConta= ctMe semitwist.com</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On 2/3/2014 4:13 PM, H. S.= Teoh wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;ve seen real-life<br> examples of ARCs gone horribly, horribly wrong, whereas had a GC been<br> used in the first place things wouldn&#39;t have gone down that route.<br> <br> </blockquote> <br></div> I&#39;m curious to hear more about this.<br></blockquote><div><br></div><di= v>Me too.</div></div></div></div> --f46d044519b9c1078904f18bfe5d--
Feb 03 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 02:05:07 UTC, Nick Sabalausky 
wrote:
 On 2/3/2014 4:13 PM, H. S. Teoh wrote:
 I've seen real-life
 examples of ARCs gone horribly, horribly wrong, whereas had a 
 GC been
 used in the first place things wouldn't have gone down that 
 route.

I'm curious to hear more about this.

An example is when you have a huge graph and the root reaches it count == 0. The time taken into a cascading deletes of the whole structure is similar to a stop-the-world GC pause. -- Paulo
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 23:31:25 -0800, Paulo Pinto <pjmlp progtools.org>  
wrote:

 On Tuesday, 4 February 2014 at 02:05:07 UTC, Nick Sabalausky wrote:
 On 2/3/2014 4:13 PM, H. S. Teoh wrote:
 I've seen real-life
 examples of ARCs gone horribly, horribly wrong, whereas had a GC been
 used in the first place things wouldn't have gone down that route.

I'm curious to hear more about this.

An example is when you have a huge graph and the root reaches it count == 0. The time taken into a cascading deletes of the whole structure is similar to a stop-the-world GC pause. -- Paulo

That's a generic example, but ARC proponents steadfastly maintain that it doesn't happen often enough in practice to be relevant. Mr. Teoh's examples of it would be most helpful. And there is more than one way ARC can go horribly wrong, like cyclic references inside a huge graph. Wouldn't THAT be fun! :-) I could design a huge graph that never gets freed in ARC in my sleep, I've built them before without trying hard in C#, most of the time quite by accident, but not always... -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling parent Manu <turkeyman gmail.com> writes:
--089e015383e452988e04f190e162
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 17:31, Paulo Pinto <pjmlp progtools.org> wrote:

 On Tuesday, 4 February 2014 at 02:05:07 UTC, Nick Sabalausky wrote:

 On 2/3/2014 4:13 PM, H. S. Teoh wrote:

 I've seen real-life
 examples of ARCs gone horribly, horribly wrong, whereas had a GC been
 used in the first place things wouldn't have gone down that route.


An example is when you have a huge graph and the root reaches it count == 0. The time taken into a cascading deletes of the whole structure is similar to a stop-the-world GC pause.

Only if it's not deferred, and even then, if you're freeing a huge structure like that, it's at a particular time where you planned to do so. No realtime app I know of goes and free's a huge runtime graph mid-frame at some random moment. That's just silly. It's still easy to defer deletion under ARC if that's what you want to do... it's all about _choice_ :) --089e015383e452988e04f190e162 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 17:31, Paulo Pinto <span dir=3D"ltr">&lt;<a href=3D"mailto:p= jmlp progtools.org" target=3D"_blank">pjmlp progtools.org</a>&gt;</span> wr= ote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border= -left:1px #ccc solid;padding-left:1ex"> On Tuesday, 4 February 2014 at 02:05:07 UTC, Nick Sabalausky wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 2/3/2014 4:13 PM, H. S. Teoh wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;ve seen real-life<br> examples of ARCs gone horribly, horribly wrong, whereas had a GC been<br> used in the first place things wouldn&#39;t have gone down that route.<br> <br> </blockquote> <br> I&#39;m curious to hear more about this.<br> </blockquote> <br> An example is when you have a huge graph and the root reaches it count =3D= =3D 0.<br> <br> The time taken into a cascading deletes of the whole structure is similar t= o a stop-the-world GC pause.<br></blockquote><div><br></div><div>Only if it= &#39;s not deferred, and even then, if you&#39;re freeing a huge structure = like that, it&#39;s at a particular time where you planned to do so.</div> <div>No realtime app I know of goes and free&#39;s a huge runtime graph mid= -frame at some random moment. That&#39;s just silly.</div><div><br></div><d= iv>It&#39;s still easy to defer deletion under ARC if that&#39;s what you w= ant to do... it&#39;s all about _choice_ :)</div> </div></div></div> --089e015383e452988e04f190e162--
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Saturday, 1 February 2014 at 04:33:23 UTC, Frank Bauer wrote:
 Yes, that is what Rust calls an "owning pointer". While we are 
 at it, we might as well dream of "non-owning" or "borrowed" 
 pointers, a.k.a. references in Rust. They don't have an effect 
 on memory-deallocation when they go out of scope, but they 
 prevent the owning pointer they "borrow" from (i.e. are 
 assigned from) from being reassigned to a different object as 
 long as there are borrowing references in scope. There is more 
 to owning and borrowed pointers, but I think that is the 
 essence.

I think D can use simplified model with only `scope`. Using it as storage class will effectively make data owned (not necessarily pointer), using it as qualifier will allow to "borrow" scope reference/pointer, for example, to pass as function argument.
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Fri, 31 Jan 2014 23:35:44 -0800, Manu <turkeyman gmail.com> wrote:

 On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> wrote:

 On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:

  On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:
   std.typecons.RefCounted!T
 core.memory.GC.disable();

I see, D's claim of being a multi-paradigm language is not false.

It's not a realistic suggestion. Everything you want to link uses the GC, and the language its self also uses the GC. Unless you write software in complete isolation and forego many valuable features, it's not a solution. Phobos does rely on the GC to some extent. Most algorithms and ranges do
 not though.

off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help!

around here. I really wanted to break that mental barrier, but it hasn't happened. In fact, I am more than ever convinced that the GC won't do. My current #1 wishlist item for D is the ability to use a reference counted collector in place of the built-in GC. You're not alone :) I write realtime and memory-constrained software (console games), and for me, I think the biggest issue that can never be solved is the non-deterministic nature of the collect cycles, and the unknowable memory footprint of the application. You can't make any guarantees or predictions about the GC, which is fundamentally incompatible with realtime software. Language-level ARC would probably do quite nicely for the miscellaneous allocations. Obviously, bulk allocations are still usually best handled in a context sensitive manner; ie, regions/pools/freelists/whatever, but the convenience of the GC paradigm does offer some interesting and massively time-saving features to D. Everyone will always refer you to RefCounted, which mangles your types and pollutes your code, but aside from that, for ARC to be useful, it needs to be supported at the language-level, such that the language/optimiser is able to optimise out redundant incref/decref calls, and also that it is compatible with immutable (you can't manage a refcount if the object is immutable).

The problem isn't GC's per se. But D's horribly naive implementation, games are written on GC languages now all the time (Unity/.NET). And let's be honest, games are kind of a speciality, games do things most programs will never do. You might want to read the GC Handbook. GC's aren't bad, but most, like the D GC, are just to simplistic for common usage today.

Maybe a sufficiently advanced GC could address the performance non-determinism to an acceptable level, but you're still left with the memory non-determinism, and the conundrum that when your heap approaches full (which is _always_ on a games console), the GC has to work harder and harder, and more often to try and keep the tiny little bit of overhead available. A GC heap by nature expects you to have lots of memory, and also lots of FREE memory. No serious console game I'm aware of has ever been written in a language with a GC. Casual games, or games that don't attempt to raise the bar may get away with it, but that's not the industry I work in.

That's kind of my point. You're asking for massive changes throughout the entire compiler to support what is becoming more of an edge case, not less of one. For the vast majority of use cases, a GC is the right call and D has to cater to the majority if it wants to gain any significant mindshare at all. You don't grow by increasing specialization... -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Wednesday, 2 January 2013 at 12:32:01 UTC, deadalnix wrote:
 On Wednesday, 2 January 2013 at 11:41:33 UTC, 
 DythroposTheImposter wrote:
 I'm interested in how the new LuaJIT GC ends up performing. 
 But overall I can't say I have much hope for GC right now.

 GC/D = Generally Faster allocation. Has a cost associated with 
 every living object.

True, however, GC + immutability allow new idoms that are plain impossible in a non GC world, and that are really efficients at reducing copies and allocations (and used in many high perf D libs).

And the problem is that D uses GC as a sledgehammer. A GC shouldn't be used for everything. If it is used for stuff it works well at and not for anything and everything, then it becomes useful. Because the GC is used for everything in D it gets clogged up with stuff like simple object allocation(which, of course, there is a way to get around, but not easily) which effects it's performance. For example, suppose the GC was used only for closures and nothing else. In this case, since closures are not used that much compared to other things, the GC would have minimal impact on performance. It could probably be ran once a month and not have memory issues. But since the GC does everything, you have to run it much more often, which means the amortization is much higher regardless if we even use closures. Ultimately the best option is control. If we were not "forced" to use the GC for everything but could enable it for different things, then we could tweak it for best performance. And why does Phobos/runtime require the GC in so many cases when it could just use an internal buffer? So much effort has been put in to make the GC work that it simply has neglected all those that can't use it as it is. Basically doing the exact opposite by making the GC more ubiquitous in D. Makes people that have no problem with the GC happy but those that do have problems just get more and more frustrated. What has happened is the GC has made the library writers lazy. It doesn't belong in 99% of Phobos.
Feb 01 2014
prev sibling next sibling parent "JR" <zorael gmail.com> writes:
On Saturday, 1 February 2014 at 05:36:44 UTC, Manu wrote:
 I write realtime and memory-constrained software (console 
 games), and for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the 
 unknowable memory
 footprint of the application. You can't make any guarantees or 
 predictions
 about the GC, which is fundamentally incompatible with realtime 
 software.

turns out even worse.) (Maybe this would be better posted in D.learn; if so I'll crosspost.) In your opinion, of how much value would deadlining be? As in, "okay handyman, you may sweep the floor now BUT ONLY FOR 6 MILLISECONDS; whatever's left after that you'll have to take care of next time, your pride as a professional Sweeper be damned"? It obviously doesn't address memory footprint, but you would get the illusion of determinism in cases similar to where race-to-idle approaches work. Inarguably, this wouldn't apply if the goal is to render as many frames per second as possible, such as for non-console shooters where tearing is not a concern but latency is very much so. I'm very much a layman in this field, but I'm trying to soak up as much knowledge as possible, and most of it from threads like these. To my uneducated eyes, an ARC collector does seem like the near-ideal solution -- assuming, as always, the code is written with the GC in mind. But am I right in gathering that it solves two-thirds of the problem? You don't need to scan the managed heap, but when memory is actually freed is still non-deterministic and may incur pauses, though not necessarily a program-wide stop. Aye? At the same time, Lucarella's dconf slides were very, very attractive. I gather that allocations themselves may become slower with a concurrent collector, but collection times in essence become non-issues. Technically parallelism doesn't equate to free CPU time; but that it more or less *is* assuming there is a cores/thread to spare. Wrong? Lastly, am I right in understanding precise collectors as identical to the stop-the-world collector we currently have, but with a smarter allocation scheme resulting in a smaller managed heap to scan? With the additional bonus of less false pointers. If so, this would seem like a good improvement to the current implementation, with the next increment in the same direction being a generational gc. I would *dearly* love to have concurrency in whatever we end up with, though. For a multi-core personal computer threads are free lunches, or close enough so. Concurrentgate and all that jazz.
Feb 01 2014
prev sibling next sibling parent "develop32" <develop32 gmail.com> writes:
On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:
 In your opinion, of how much value would deadlining be? As in, 
 "okay handyman, you may sweep the floor now BUT ONLY FOR 6 
 MILLISECONDS;

Unrelated to the whole post note: 6ms is too high, I would allow max 1ms a frame for a game I'm developing and even that is somewhat painful. If I turn on GC on the game I'm making, it takes 80ms every frame when memory usage is 800mb (shown on task manager). This was actually surprising, previously it was 6ms, good that I made my engine GC independent.
Feb 01 2014
prev sibling next sibling parent "develop32" <develop32 gmail.com> writes:
On Saturday, 1 February 2014 at 12:29:10 UTC, Nick Sabalausky 
wrote:
 Come to think of it, I wonder what language Frostbite 3 uses 
 for game-level "scripting" code. From what I've heard about 
 them, Frostbite 3 and Unreal Engine 4 both sound like they have 
 some notable similarities with Unity3D (from the standpoint of 
 the user-experience for game developers), although AIUI Unreal 
 Engine 4 still uses C++ for game code (unless the engine user 
 wants to to do lua or something on their own). I imagine 
 Frostbite's probably the same, C++, but I haven't actually 
 heard anything.

Frostbite has an option for both Lua and C++. C++ is the preferred one. Unreal Engine went from Unrealscript to C++ and everyone applauded that.
Feb 01 2014
prev sibling next sibling parent "JR" <zorael gmail.com> writes:
On Saturday, 1 February 2014 at 12:20:33 UTC, develop32 wrote:
 If I turn on GC on the game I'm making, it takes 80ms every 
 frame when memory usage is 800mb (shown on task manager). This 
 was actually surprising, previously it was 6ms, good that I 
 made my engine GC independent.

You wouldn't happen to have a blog post someplace with reflections on the steps you took to avoid the GC, by any chance? Hint hint! I'm genuinely curious. Or is it merely a matter of manual allocation?
Feb 01 2014
prev sibling next sibling parent "develop32" <develop32 gmail.com> writes:
On Saturday, 1 February 2014 at 13:16:02 UTC, JR wrote:
 On Saturday, 1 February 2014 at 12:20:33 UTC, develop32 wrote:
 If I turn on GC on the game I'm making, it takes 80ms every 
 frame when memory usage is 800mb (shown on task manager). This 
 was actually surprising, previously it was 6ms, good that I 
 made my engine GC independent.

You wouldn't happen to have a blog post someplace with reflections on the steps you took to avoid the GC, by any chance? Hint hint! I'm genuinely curious. Or is it merely a matter of manual allocation?

I don't have a blog... It is mostly a matter of memory reuse (LOTS of it) and manual freeing when needed.
Feb 01 2014
prev sibling next sibling parent "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Saturday, 1 February 2014 at 12:20:33 UTC, develop32 wrote:
 On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:
 In your opinion, of how much value would deadlining be? As in, 
 "okay handyman, you may sweep the floor now BUT ONLY FOR 6 
 MILLISECONDS;

Unrelated to the whole post note: 6ms is too high, I would allow max 1ms a frame for a game I'm developing and even that is somewhat painful. If I turn on GC on the game I'm making, it takes 80ms every frame when memory usage is 800mb (shown on task manager). This was actually surprising, previously it was 6ms, good that I made my engine GC independent.

Wow! That's interesting! What kind of game are you working on?
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <x y.com> writes:
On Saturday, 1 February 2014 at 09:27:18 UTC, Adam Wilson wrote:
 For the vast majority of use cases, a GC is the right call and 
 D has to cater to the majority if it wants to gain any 
 significant mindshare at all. You don't grow by increasing 
 specialization...

So catering to the C++ crowd who shun Java and C# not so much for JITing their code but precisely for their forced use of a GC is a "minority" issue then? Again, please take those of us more serious who don't like a GC to interfere with their business and if it's only for "programming hygiene" reasons. As you consider handling low latency requirements undue "specialisation" anyways. On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:
 To my uneducated eyes, an ARC collector does seem like the 
 near-ideal solution

ARC would be a plus for heavily interdependent code. But it doesn't beat unique_ptr semantics in local use that free their memory immediately as they go out of scope or are reassigned.
 I would *dearly* love to have concurrency in whatever we end up 
 with, though. For a multi-core personal computer threads are 
 free lunches, or close enough so.

No they are not. I want to make good use of all the cores I have available and I don't want to share them a bit with the GC. It's myyy ... precious. As you may have guessed, my criticism is harsh because I love the language :)
Feb 01 2014
prev sibling next sibling parent "develop32" <develop32 gmail.com> writes:
On Saturday, 1 February 2014 at 14:24:17 UTC, Francesco Cattoglio 
wrote:
 On Saturday, 1 February 2014 at 12:20:33 UTC, develop32 wrote:
 On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:
 In your opinion, of how much value would deadlining be? As 
 in, "okay handyman, you may sweep the floor now BUT ONLY FOR 
 6 MILLISECONDS;

Unrelated to the whole post note: 6ms is too high, I would allow max 1ms a frame for a game I'm developing and even that is somewhat painful. If I turn on GC on the game I'm making, it takes 80ms every frame when memory usage is 800mb (shown on task manager). This was actually surprising, previously it was 6ms, good that I made my engine GC independent.

Wow! That's interesting! What kind of game are you working on?

Don't want to reveal too much now, its early in development, plan to have a proper trailer few months later. Its an economical space sim, set in an asteroid field of 3000km radius, filled with 100k asteroids, hundred stations and few hundred AI ships. All of the characters/stations are simulated simultaneously. http://imgur.com/QAKzJWb
Feb 01 2014
prev sibling next sibling parent "develop32" <develop32 gmail.com> writes:
On Saturday, 1 February 2014 at 16:37:23 UTC, Paulo Pinto wrote:

 Do you have any experience with Project Anarchy?

 --
 Paulo

No I have not. I haven't got a smartphone yet, so I guess that explains it.
Feb 01 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Saturday, 1 February 2014 at 11:40:37 UTC, Frustrated wrote:
 And why does Phobos/runtime require the GC in so many cases when
 it could just use an internal buffer? So much effort has been 
 put
 in to make the GC work that it simply has neglected all those
 that can't use it as it is.

This is the crux of the problem. It's not so much that D uses garbage collection as that Phobos is built in a way that prevents the reuse of existing buffers. It becomes much harder to sell the language to a GC averse group if it turns out they can't use the standard library as well. Though I guess Tango is an option here as a replacement.
Feb 01 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Saturday, 1 February 2014 at 17:30:54 UTC, Sean Kelly wrote:
 On Saturday, 1 February 2014 at 11:40:37 UTC, Frustrated wrote:
 And why does Phobos/runtime require the GC in so many cases 
 when
 it could just use an internal buffer? So much effort has been 
 put
 in to make the GC work that it simply has neglected all those
 that can't use it as it is.

This is the crux of the problem. It's not so much that D uses garbage collection as that Phobos is built in a way that prevents the reuse of existing buffers. It becomes much harder to sell the language to a GC averse group if it turns out they can't use the standard library as well. Though I guess Tango is an option here as a replacement.

Right, and because of the mentality that the GC is one size that fits all, phobos got this way. That mentality is still pervasive. I call it laziness. I think that at some point phobos will need to be rewritten... maybe more .net like(there sense of hierarchy and presentation is excellent). Maybe D needs a proper allocator sub system built in to get to that point?
Feb 01 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0122912c31b35304f1555d8d
Content-Type: text/plain; charset=UTF-8

On 1 February 2014 18:20, Paulo Pinto <pjmlp progtools.org> wrote:

 Am 01.02.2014 06:29, schrieb Manu:

 On 26 December 2012 00:48, Sven Over <dlang svenover.de
 <mailto:dlang svenover.de>> wrote:

         std.typecons.RefCounted!T

         core.memory.GC.disable();


     Wow. That was easy.

     I see, D's claim of being a multi-paradigm language is not false.


 It's not a realistic suggestion. Everything you want to link uses the
 GC, and the language its self also uses the GC. Unless you write
 software in complete isolation and forego many valuable features, it's
 not a solution.


         Phobos does rely on the GC to some extent. Most algorithms and
         ranges do not though.


     Running (library) code that was written with GC in mind and turning
     GC off doesn't sound ideal.

     But maybe this allows me to familiarise myself more with D. Who
     knows, maybe I can learn to stop worrying and love garbage collection.

     Thanks for your help!


 I've been trying to learn to love the GC for as long as I've been around
 here. I really wanted to break that mental barrier, but it hasn't
 happened.
 In fact, I am more than ever convinced that the GC won't do. My current
 #1 wishlist item for D is the ability to use a reference counted
 collector in place of the built-in GC.
 You're not alone :)

 I write realtime and memory-constrained software (console games), and
 for me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable
 memory footprint of the application. You can't make any guarantees or
 predictions about the GC, which is fundamentally incompatible with
 realtime software.

Meanwhile Unity and similar engines are becoming widespread, with C++ being pushed all the way to the bottom on the stack. At least from what I hear in the gaming communities I hop around. What is your experience there?

Unity is indeed popular, for casual/indy games. AAA/'big games' show no signs of moving away from C++. The 'next gen' has enough memory for GC (still can't afford the time though), but handhelds and small devices are a bigger market these days. It's true that there are less 'big games' on handhelds, which are the future of resource-limited devices, but I think that rift is closing quickly. --089e0122912c31b35304f1555d8d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= February 2014 18:20, Paulo Pinto <span dir=3D"ltr">&lt;<a href=3D"mailto:p= jmlp progtools.org" target=3D"_blank">pjmlp progtools.org</a>&gt;</span> wr= ote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border= -left:1px #ccc solid;padding-left:1ex"> Am 01.02.2014 06:29, schrieb Manu:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im"> On 26 December 2012 00:48, Sven Over &lt;<a href=3D"mailto:dlang svenover.d= e" target=3D"_blank">dlang svenover.de</a><br></div><div class=3D"im"> &lt;mailto:<a href=3D"mailto:dlang svenover.de" target=3D"_blank">dlang sve= nover.de</a>&gt;&gt; wrote:<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 std.typecons.RefCounted!T<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 core.memory.GC.disable();<br> <br> <br> =C2=A0 =C2=A0 Wow. That was easy.<br> <br> =C2=A0 =C2=A0 I see, D&#39;s claim of being a multi-paradigm language is no= t false.<br> <br> <br></div><div class=3D"im"> It&#39;s not a realistic suggestion. Everything you want to link uses the<b= r> GC, and the language its self also uses the GC. Unless you write<br> software in complete isolation and forego many valuable features, it&#39;s<= br> not a solution.<br> <br> <br></div><div class=3D"im"> =C2=A0 =C2=A0 =C2=A0 =C2=A0 Phobos does rely on the GC to some extent. Most= algorithms and<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 ranges do not though.<br> <br> <br> =C2=A0 =C2=A0 Running (library) code that was written with GC in mind and t= urning<br> =C2=A0 =C2=A0 GC off doesn&#39;t sound ideal.<br> <br> =C2=A0 =C2=A0 But maybe this allows me to familiarise myself more with D. W= ho<br> =C2=A0 =C2=A0 knows, maybe I can learn to stop worrying and love garbage co= llection.<br> <br> =C2=A0 =C2=A0 Thanks for your help!<br> <br> <br></div><div class=3D"im"> I&#39;ve been trying to learn to love the GC for as long as I&#39;ve been a= round<br> here. I really wanted to break that mental barrier, but it hasn&#39;t happe= ned.<br> In fact, I am more than ever convinced that the GC won&#39;t do. My current= <br> #1 wishlist item for D is the ability to use a reference counted<br> collector in place of the built-in GC.<br> You&#39;re not alone :)<br> <br> I write realtime and memory-constrained software (console games), and<br> for me, I think the biggest issue that can never be solved is the<br> non-deterministic nature of the collect cycles, and the unknowable<br> memory footprint of the application. You can&#39;t make any guarantees or<b= r> predictions about the GC, which is fundamentally incompatible with<br> realtime software.<br> </div></blockquote> <br> <br> Meanwhile Unity and similar engines are becoming widespread, with C++ being= pushed all the way to the bottom on the stack.<br> <br> At least from what I hear in the gaming communities I hop around.<br> <br> What is your experience there?<br></blockquote><div><br></div><div>Unity is= indeed popular, for casual/indy games.</div><div>AAA/&#39;big games&#39; s= how no signs of moving away from C++. The &#39;next gen&#39; has enough mem= ory for GC (still can&#39;t afford the time though), but handhelds and smal= l devices are a bigger market these days.</div> <div>It&#39;s true that there are less &#39;big games&#39; on handhelds, wh= ich are the future of resource-limited devices, but I think that rift is cl= osing quickly.</div></div></div></div> --089e0122912c31b35304f1555d8d--
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 09:57:41 -0800, Frustrated <c1514843 drdrb.com> wrote:

 On Saturday, 1 February 2014 at 17:30:54 UTC, Sean Kelly wrote:
 On Saturday, 1 February 2014 at 11:40:37 UTC, Frustrated wrote:
 And why does Phobos/runtime require the GC in so many cases when
 it could just use an internal buffer? So much effort has been put
 in to make the GC work that it simply has neglected all those
 that can't use it as it is.

This is the crux of the problem. It's not so much that D uses garbage collection as that Phobos is built in a way that prevents the reuse of existing buffers. It becomes much harder to sell the language to a GC averse group if it turns out they can't use the standard library as well. Though I guess Tango is an option here as a replacement.

Right, and because of the mentality that the GC is one size that fits all, phobos got this way. That mentality is still pervasive. I call it laziness. I think that at some point phobos will need to be rewritten... maybe more .net like(there sense of hierarchy and presentation is excellent). Maybe D needs a proper allocator sub system built in to get to that point?

You do realize that in .NET the BCL is not GC-optional right. It GC-allocates like a fiend. What they have is a monumentally better GC than D. .NET is actually proof that GC's can work quite well in all but the most extreme edge cases. Effectively what you are saying is ".NET does it well with a GC so we need to rewrite Phobos to not use a GC" ... Um wait, what? I am not saying to Phobos needs to allocate as much as it does, just that argument itself fails to met a basic standard of logical coherency. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 04:04:54 -0800, JR <zorael gmail.com> wrote:

 On Saturday, 1 February 2014 at 05:36:44 UTC, Manu wrote:
 I write realtime and memory-constrained software (console games), and  
 for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable  
 memory
 footprint of the application. You can't make any guarantees or  
 predictions
 about the GC, which is fundamentally incompatible with realtime  
 software.

out even worse.) (Maybe this would be better posted in D.learn; if so I'll crosspost.) In your opinion, of how much value would deadlining be? As in, "okay handyman, you may sweep the floor now BUT ONLY FOR 6 MILLISECONDS; whatever's left after that you'll have to take care of next time, your pride as a professional Sweeper be damned"? It obviously doesn't address memory footprint, but you would get the illusion of determinism in cases similar to where race-to-idle approaches work. Inarguably, this wouldn't apply if the goal is to render as many frames per second as possible, such as for non-console shooters where tearing is not a concern but latency is very much so. I'm very much a layman in this field, but I'm trying to soak up as much knowledge as possible, and most of it from threads like these. To my uneducated eyes, an ARC collector does seem like the near-ideal solution -- assuming, as always, the code is written with the GC in mind. But am I right in gathering that it solves two-thirds of the problem? You don't need to scan the managed heap, but when memory is actually freed is still non-deterministic and may incur pauses, though not necessarily a program-wide stop. Aye?

It would only not be a program wide stop if you had multiple threads running, otherwise yes, ARC can still Stop-The-World for a non-deterministic period of time, because you the programmer have no idea how long that collection cycle will last. Also note that this is just a shuffling of where the collection happens. In D's GC a collection can happen any time you attempt to allocate, whereas in ARC you eagerly collect when you delete, because if you don't eagerly collect you'll have a memory leak. Also you can't make ARC concurrent.
 At the same time, Lucarella's dconf slides were very, very attractive. I  
 gather that allocations themselves may become slower with a concurrent  
 collector, but collection times in essence become non-issues.  
 Technically parallelism doesn't equate to free CPU time; but that it  
 more or less *is* assuming there is a cores/thread to spare. Wrong?

Essentially yes, concurrency does get rid of MOST of the STW aspects of GC's. However, most modern GC's are generational and typically their are one or two generations that are not collected concurrently. In .NET both Generations 0 and 1 are not collected concurrently because they can be collected very quickly, more quickly than cost of enabling concurrent collection support on allocation. For example, I use WPF for almost every project I do at work. WPF is a retained-mode GUI API based on DirectX 9, and it has a 60FPS render speed requirement. The only time I have seen the rendering bog down due to the GC is when there are a LOT of animations starting and stopping. Otherwise it's almost always because of WPF's horrifically naive rendering code.
 Lastly, am I right in understanding precise collectors as identical to  
 the stop-the-world collector we currently have, but with a smarter  
 allocation scheme resulting in a smaller managed heap to scan? With the  
 additional bonus of less false pointers. If so, this would seem like a  
 good improvement to the current implementation, with the next increment  
 in the same direction being a generational gc.

Correct, precision won't change the STW nature of the GC, just make it so there is much less to scan/collect in the first place, and believe it or not, the difference can be huge. See Rainer Schutze's for more information on a precise collector in D: http://dconf.org/2013/talks/schuetze.html
 I would *dearly* love to have concurrency in whatever we end up with,  
 though. For a multi-core personal computer threads are free lunches, or  
 close enough so. Concurrentgate and all that jazz.

You and me both, this is the way all GC's are headed, I can't think of a major GC-language that doesn't have a Concurrent-Generational-Incremental GC. :-) -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c2c484af076704f15592df
Content-Type: text/plain; charset=UTF-8

On 1 February 2014 19:27, Adam Wilson <flyboynw gmail.com> wrote:

 On Fri, 31 Jan 2014 23:35:44 -0800, Manu <turkeyman gmail.com> wrote:

  On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> wrote:
  On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:
  On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:

   std.typecons.RefCounted!T

 core.memory.GC.disable();


  Wow. That was easy.

I see, D's claim of being a multi-paradigm language is not false.

It's not a realistic suggestion. Everything you want to link uses the GC, and the language its self also uses the GC. Unless you write software in complete isolation and forego many valuable features, it's not a solution. Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.
  Running (library) code that was written with GC in mind and turning

off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help! I've been trying to learn to love the GC for as long as I've been

here. I really wanted to break that mental barrier, but it hasn't happened. In fact, I am more than ever convinced that the GC won't do. My current #1 wishlist item for D is the ability to use a reference counted collector in place of the built-in GC. You're not alone :) I write realtime and memory-constrained software (console games), and for me, I think the biggest issue that can never be solved is the non-deterministic nature of the collect cycles, and the unknowable memory footprint of the application. You can't make any guarantees or predictions about the GC, which is fundamentally incompatible with realtime software. Language-level ARC would probably do quite nicely for the miscellaneous allocations. Obviously, bulk allocations are still usually best handled in a context sensitive manner; ie, regions/pools/freelists/whatever, but the convenience of the GC paradigm does offer some interesting and massively time-saving features to D. Everyone will always refer you to RefCounted, which mangles your types and pollutes your code, but aside from that, for ARC to be useful, it needs to be supported at the language-level, such that the language/optimiser is able to optimise out redundant incref/decref calls, and also that it is compatible with immutable (you can't manage a refcount if the object is immutable).

games are written on GC languages now all the time (Unity/.NET). And let's be honest, games are kind of a speciality, games do things most programs will never do. You might want to read the GC Handbook. GC's aren't bad, but most, like the D GC, are just to simplistic for common usage today.

Maybe a sufficiently advanced GC could address the performance non-determinism to an acceptable level, but you're still left with the memory non-determinism, and the conundrum that when your heap approaches full (which is _always_ on a games console), the GC has to work harder and harder, and more often to try and keep the tiny little bit of overhead available. A GC heap by nature expects you to have lots of memory, and also lots of FREE memory. No serious console game I'm aware of has ever been written in a language with a GC. Casual games, or games that don't attempt to raise the bar may get away with it, but that's not the industry I work in.

That's kind of my point. You're asking for massive changes throughout the entire compiler to support what is becoming more of an edge case, not less of one. For the vast majority of use cases, a GC is the right call and D has to cater to the majority if it wants to gain any significant mindshare at all. You don't grow by increasing specialization...

Why is ARC any worse than GC? Why is it even a compromise at the high level? Major players have been moving away from GC to ARC in recent years. It's still a perfectly valid method of garbage collection, and it has the advantage that it's intrinsically real-time compatible. I don't think realtime software is becoming an edge case by any means, maybe 'extreme' realtime is, but that doesn't change the fact that the GC still causes problems for all realtime software. I personally believe latency and stuttering is one of the biggest usability hindrances in modern computing, and it will become a specific design focus in software of the future. People are becoming less and less tolerant of latency in all forms; just consider the success of iPhone compared to competition, almost entirely attributable to the silky smooth UI experience. It may also be a telling move that Apple switched to ARC around the same time, but I don't know the details. I also firmly believe that if D - a native compiled language familiar to virtually all low-level programmers - doesn't have any ambition to service the embedded space in the future, what will? And why not? The GC is the only thing inhibiting D from being a successful match in that context. ARC is far more appropriate, and will see it enter a lot more fields. What's the loss? I think it's also telling that newcomers constantly raise it as a massive concern, or even a deal-breaker. Would they feel the same about ARC? I seriously doubt it. I wonder if a poll is in order... Conversely, would any of the new-comers who are pro-GC feel any less happy if it were ARC instead? $100 says they probably wouldn't even know, and almost certainly wouldn't care. --001a11c2c484af076704f15592df Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= February 2014 19:27, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Fri, 31 Jan 2014 23:35:44 -0800, Manu &lt;<a href=3D"m= ailto:turkeyman gmail.com" target=3D"_blank">turkeyman gmail.com</a>&gt; wr= ote:<br> <br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"><div class=3D"im"> On 1 February 2014 16:26, Adam Wilson &lt;<a href=3D"mailto:flyboynw gmail.= com" target=3D"_blank">flyboynw gmail.com</a>&gt; wrote:<br> <br> </div><div><div class=3D"h5"><blockquote class=3D"gmail_quote" style=3D"mar= gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On Fri, 31 Jan 2014 21:29:04 -0800, Manu &lt;<a href=3D"mailto:turkeyman gm= ail.com" target=3D"_blank">turkeyman gmail.com</a>&gt; wrote:<br> <br> =C2=A0On 26 December 2012 00:48, Sven Over &lt;<a href=3D"mailto:dlang sven= over.de" target=3D"_blank">dlang svenover.de</a>&gt; wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> =C2=A0 std.typecons.RefCounted!T<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> core.memory.GC.disable();<br> <br> <br> </blockquote> Wow. That was easy.<br> <br> I see, D&#39;s claim of being a multi-paradigm language is not false.<br> <br> </blockquote> <br> <br> It&#39;s not a realistic suggestion. Everything you want to link uses the G= C,<br> and the language its self also uses the GC. Unless you write software in<br=

on.<br> <br> <br> =C2=A0Phobos does rely on the GC to some extent. Most algorithms and ranges= do<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> not though.<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> <br> </blockquote> Running (library) code that was written with GC in mind and turning GC<br> off<br> doesn&#39;t sound ideal.<br> <br> But maybe this allows me to familiarise myself more with D. Who knows,<br> maybe I can learn to stop worrying and love garbage collection.<br> <br> Thanks for your help!<br> <br> <br> </blockquote> I&#39;ve been trying to learn to love the GC for as long as I&#39;ve been a= round<br> here. I really wanted to break that mental barrier, but it hasn&#39;t<br> happened.<br> In fact, I am more than ever convinced that the GC won&#39;t do. My current= #1<br> wishlist item for D is the ability to use a reference counted collector in<= br> place of the built-in GC.<br> You&#39;re not alone :)<br> <br> I write realtime and memory-constrained software (console games), and for<b= r> me, I think the biggest issue that can never be solved is the<br> non-deterministic nature of the collect cycles, and the unknowable memory<b= r> footprint of the application. You can&#39;t make any guarantees or predicti= ons<br> about the GC, which is fundamentally incompatible with realtime software.<b= r> Language-level ARC would probably do quite nicely for the miscellaneous<br> allocations. Obviously, bulk allocations are still usually best handled in<= br> a context sensitive manner; ie, regions/pools/freelists/<u></u>whatever, bu= t the<br> convenience of the GC paradigm does offer some interesting and massively<br=

Everyone will always refer you to RefCounted, which mangles your types and<= br> pollutes your code, but aside from that, for ARC to be useful, it needs to<= br> be supported at the language-level, such that the language/optimiser is<br> able to optimise out redundant incref/decref calls, and also that it is<br> compatible with immutable (you can&#39;t manage a refcount if the object is= <br> immutable).<br> <br> </blockquote> <br> The problem isn&#39;t GC&#39;s per se. But D&#39;s horribly naive implement= ation,<br> games are written on GC languages now all the time (Unity/.NET). And let&#3= 9;s<br> be honest, games are kind of a speciality, games do things most programs<br=

<br> You might want to read the GC Handbook. GC&#39;s aren&#39;t bad, but most, = like<br> the D GC, are just to simplistic for common usage today.<br> </blockquote> <br> <br></div></div><div class=3D"im"> Maybe a sufficiently advanced GC could address the performance<br> non-determinism to an acceptable level, but you&#39;re still left with the<= br> memory non-determinism, and the conundrum that when your heap approaches<br=

br> harder, and more often to try and keep the tiny little bit of overhead<br> available.<br> A GC heap by nature expects you to have lots of memory, and also lots of<br=

<br> No serious console game I&#39;m aware of has ever been written in a languag= e<br> with a GC. Casual games, or games that don&#39;t attempt to raise the bar m= ay<br> get away with it, but that&#39;s not the industry I work in.<br> </div></blockquote> <br> That&#39;s kind of my point. You&#39;re asking for massive changes througho= ut the entire compiler to support what is becoming more of an edge case, no= t less of one. For the vast majority of use cases, a GC is the right call a= nd D has to cater to the majority if it wants to gain any significant minds= hare at all. You don&#39;t grow by increasing specialization...</blockquote=

se at the high level?</div><div>Major players have been moving away from GC= to ARC in recent years. It&#39;s still a perfectly valid method of garbage= collection, and it has the advantage that it&#39;s intrinsically real-time= compatible.</div> <div><br></div><div>I don&#39;t think realtime software is becoming an edge= case by any means, maybe &#39;extreme&#39; realtime is, but that doesn&#39= ;t change the fact that the GC still causes problems for all realtime softw= are.</div> <div><br></div><div>I personally believe latency and stuttering is one of t= he biggest usability hindrances in modern computing, and it will become a s= pecific design focus in software of the future. People are becoming less an= d less tolerant of latency in all forms; just consider the success of iPhon= e compared to competition, almost entirely attributable to the silky smooth= UI experience. It may also be a telling move that Apple switched to ARC ar= ound the same time, but I don&#39;t know the details.</div> <div><br></div><div>I also firmly believe that if D - a native compiled lan= guage familiar to virtually all low-level programmers - doesn&#39;t have an= y ambition to service the embedded space in the future, what will? And why = not?</div> <div>The GC is the only thing inhibiting D from being a successful match in= that context. ARC is far more appropriate, and will see it enter a lot mor= e fields.</div><div>What&#39;s the loss?</div><div><br></div><div>I think i= t&#39;s also telling that newcomers constantly raise it as a massive concer= n, or even a deal-breaker. Would they feel the same about ARC? I seriously = doubt it. I wonder if a poll is in order...</div> <div>Conversely, would any of the new-comers who are pro-GC feel any less h= appy if it were ARC instead? $100 says they probably wouldn&#39;t even know= , and almost certainly wouldn&#39;t care.</div></div></div></div> --001a11c2c484af076704f15592df--
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 06:33:15 -0800, Frank Bauer <x y.com> wrote:

 On Saturday, 1 February 2014 at 09:27:18 UTC, Adam Wilson wrote:
 For the vast majority of use cases, a GC is the right call and D has to  
 cater to the majority if it wants to gain any significant mindshare at  
 all. You don't grow by increasing specialization...

So catering to the C++ crowd who shun Java and C# not so much for JITing their code but precisely for their forced use of a GC is a "minority" issue then? Again, please take those of us more serious who don't like a GC to interfere with their business and if it's only for "programming hygiene" reasons. As you consider handling low latency requirements undue "specialisation" anyways.

Real-time or low-latency programming is absolutely specialization, and for more reasons than a GC. It might be what you do all day, but it's not what the majority of programmers do all day. As far as catering to C++. TIOBE says that C++ made up 7.5% of programmer interest in January. Whereas Java+C# made up 22.4% of programmer interest. That's almost three-times the interest. Like it or not, C++ IS the minority and D's mission is not solely based on converting C++ users, many of whom can't convert anyways. The reason I picked those three languages is because they are D's primary audience.
 On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:
 To my uneducated eyes, an ARC collector does seem like the near-ideal  
 solution

ARC would be a plus for heavily interdependent code. But it doesn't beat unique_ptr semantics in local use that free their memory immediately as they go out of scope or are reassigned.

So you want C++ back then? Got it. But the goal of D is NOT to be a slightly better C++, it's a be a Safe, Simple, systems language with well defined semantics that just happens to trace it's lineage back to C++.
 I would *dearly* love to have concurrency in whatever we end up with,  
 though. For a multi-core personal computer threads are free lunches, or  
 close enough so.

No they are not. I want to make good use of all the cores I have available and I don't want to share them a bit with the GC. It's myyy ... precious.

Ahem. You only have the illusion of control over the CPU, the OS scheduler set's the rules, all you can do is provide hints. Giving some up to the GC will not materially affect the performance of the your code since you never had a choice anyways.
 As you may have guessed, my criticism is harsh because I love the  
 language :)

-- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Saturday, 1 February 2014 at 19:35:08 UTC, Manu wrote:
 I also firmly believe that if D - a native compiled language 
 familiar to
 virtually all low-level programmers - doesn't have any ambition 
 to service
 the embedded space in the future, what will?

I feel like Rust have already won this. It has more suitable basic and has more developers -> evolves faster in general. Can be too late to even call it priority. // only 50% trolling
Feb 01 2014
prev sibling next sibling parent reply "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 02:15:42 -0800, Manu <turkeyman gmail.com> wrote:

 On 1 February 2014 19:27, Adam Wilson <flyboynw gmail.com> wrote:

 On Fri, 31 Jan 2014 23:35:44 -0800, Manu <turkeyman gmail.com> wrote:

  On 1 February 2014 16:26, Adam Wilson <flyboynw gmail.com> wrote:
  On Fri, 31 Jan 2014 21:29:04 -0800, Manu <turkeyman gmail.com> wrote:
  On 26 December 2012 00:48, Sven Over <dlang svenover.de> wrote:

   std.typecons.RefCounted!T

 core.memory.GC.disable();


  Wow. That was easy.

I see, D's claim of being a multi-paradigm language is not false.

It's not a realistic suggestion. Everything you want to link uses the GC, and the language its self also uses the GC. Unless you write software in complete isolation and forego many valuable features, it's not a solution. Phobos does rely on the GC to some extent. Most algorithms and ranges do not though.
  Running (library) code that was written with GC in mind and  
 turning

off doesn't sound ideal. But maybe this allows me to familiarise myself more with D. Who knows, maybe I can learn to stop worrying and love garbage collection. Thanks for your help! I've been trying to learn to love the GC for as long as I've been

here. I really wanted to break that mental barrier, but it hasn't happened. In fact, I am more than ever convinced that the GC won't do. My current #1 wishlist item for D is the ability to use a reference counted collector in place of the built-in GC. You're not alone :) I write realtime and memory-constrained software (console games), and for me, I think the biggest issue that can never be solved is the non-deterministic nature of the collect cycles, and the unknowable memory footprint of the application. You can't make any guarantees or predictions about the GC, which is fundamentally incompatible with realtime software. Language-level ARC would probably do quite nicely for the miscellaneous allocations. Obviously, bulk allocations are still usually best handled in a context sensitive manner; ie, regions/pools/freelists/whatever, but the convenience of the GC paradigm does offer some interesting and massively time-saving features to D. Everyone will always refer you to RefCounted, which mangles your types and pollutes your code, but aside from that, for ARC to be useful, it needs to be supported at the language-level, such that the language/optimiser is able to optimise out redundant incref/decref calls, and also that it is compatible with immutable (you can't manage a refcount if the object is immutable).

games are written on GC languages now all the time (Unity/.NET). And let's be honest, games are kind of a speciality, games do things most programs will never do. You might want to read the GC Handbook. GC's aren't bad, but most, like the D GC, are just to simplistic for common usage today.

Maybe a sufficiently advanced GC could address the performance non-determinism to an acceptable level, but you're still left with the memory non-determinism, and the conundrum that when your heap approaches full (which is _always_ on a games console), the GC has to work harder and harder, and more often to try and keep the tiny little bit of overhead available. A GC heap by nature expects you to have lots of memory, and also lots of FREE memory. No serious console game I'm aware of has ever been written in a language with a GC. Casual games, or games that don't attempt to raise the bar may get away with it, but that's not the industry I work in.

That's kind of my point. You're asking for massive changes throughout the entire compiler to support what is becoming more of an edge case, not less of one. For the vast majority of use cases, a GC is the right call and D has to cater to the majority if it wants to gain any significant mindshare at all. You don't grow by increasing specialization...

Why is ARC any worse than GC? Why is it even a compromise at the high level? Major players have been moving away from GC to ARC in recent years. It's still a perfectly valid method of garbage collection, and it has the advantage that it's intrinsically real-time compatible.

Define Major Players? Because I only know about Apple, but they've been doing ARC for almost a decade, and IIRC, like GC's, it's not universally loved there either. Microsoft is desperately trying to get people to move back to C++ but so far the community has spoken with a resounding "You can pry C#/.NET from our cold, dead, hands." Java has shown no particular interest in moving away from GC's probably because their GC is best in class. Even games are starting to bring in GC's (The Witcher 2 for example, and almost all of the mobile/casual games market, which is actually monetarily bigger than the triple-A games market.)
 I don't think realtime software is becoming an edge case by any means,
 maybe 'extreme' realtime is, but that doesn't change the fact that the GC
 still causes problems for all realtime software.

Yes, but my point is that there is very little real-time software written as a percentage of all software written, which, by definition, makes it an edge-case. Even vehicle control software is no longer completely real-time. [I just happen to know that because that's the industry I work in. Certain aspects are, with the rest scheduled out.] And more to the point, D has made no claim about it's suitability for RT software and I have seen little demand for it outside a very small very vocal minority that is convinced that it has the dynamic resource management panacea if everyone would just do as they say.
 I personally believe latency and stuttering is one of the biggest  
 usability
 hindrances in modern computing, and it will become a specific design  
 focus
 in software of the future. People are becoming less and less tolerant of
 latency in all forms; just consider the success of iPhone compared to
 competition, almost entirely attributable to the silky smooth UI
 experience. It may also be a telling move that Apple switched to ARC  
 around
 the same time, but I don't know the details.

I use .NET every day, seriously not one day goes by when I haven't touched some C# code. I can happily report that you are *ahem* wrong. Even Visual Studio 2013 doesn't stutter often, and only when I am pulling in some massive third-party module that may or may not be well written. Ironically, it is VisualD, it of D code fame that most slows down VS for me. I write software in C# every day I can happily report that I have yet to have a problem with stuttering in my code that wasn't of my own devise. (If you forget to asynchronously call that web-service it WILL stutter) And that's a point. I can write software in C# that works will without having to worry about circular references or if my data prematurely falls out of scope or any other of the details that are associated with ARC. And for my not-effort, I pay an effective cost of 0. Win-win. You're demanding that to suit your needs, we make a massive philosophical and language change to D that will incur HIGHER cognitive load on programmers for something that will not measurably improve the general use case? Ahem, that's good for D how?
 I also firmly believe that if D - a native compiled language familiar to
 virtually all low-level programmers - doesn't have any ambition to  
 service
 the embedded space in the future, what will? And why not?
 The GC is the only thing inhibiting D from being a successful match in  
 that
 context. ARC is far more appropriate, and will see it enter a lot more
 fields.
 What's the loss?

Cognitive load. How many details does the programmer have to worry about per line of code. Ease-of-use. A GC is easier to use in practice. You can say well they should learn to use ARC because it's better (for certain definitions of better) but they won't. They'll just move on. I'd say that's a pretty big loss. And based on my conversations with Walter, I don't think that D was ever intended to make a play for the embedded space. If D can be made to work there great, but Walter, as near as I can tell, has no interest in tying the language in knots to make it happen. So that's a non-issue. And let's be honest, the requirements of that space are fairly extreme.
 I think it's also telling that newcomers constantly raise it as a massive
 concern, or even a deal-breaker. Would they feel the same about ARC? I
 seriously doubt it. I wonder if a poll is in order...
 Conversely, would any of the new-comers who are pro-GC feel any less  
 happy
 if it were ARC instead? $100 says they probably wouldn't even know, and
 almost certainly wouldn't care.

I DON'T see a majority of newcomers raising an issue with the GC, I only see it from newcomers with some pretty extreme latency requirements, primarily for the real-time crowd. The majority of newcomers aren't interested in RT work. I think you're falling prey to confirmation bias here. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Feb 04, 2014 at 01:11:23PM +1000, Manu wrote:
 On 4 February 2014 12:05, Nick Sabalausky <
 SeeWebsiteToContactMe semitwist.com> wrote:
 
 On 2/3/2014 4:13 PM, H. S. Teoh wrote:

 I've seen real-life examples of ARCs gone horribly, horribly wrong,
 whereas had a GC been used in the first place things wouldn't have
 gone down that route.


Me too.

Well, first I have to apologize for my misleading message, I meant *reference counting systems*, not ARC specifically, since the case I had in mind is a reference-counting system implemented in C, so obviously it's not *A*RC. :-( But anyway, what happened is that in the past, we used to do just straight malloc/free, but over time discovered that it was an unending source of embarrassing pointer bugs and memory leaks. So at some point, the team lead decided to implement a reference-counting system instead. Which worked *better*, I suppose, but it was hardly the panacea that people might think. It was still prone to pointer bugs and memory leaks caused by not updating the reference counts properly -- something that an ARC wouldn't have, I guess. But then people started doing things with dtors that they shouldn't do... like this one especially egregious abuse that I had in mind, where there was a container pointing to a bunch of subobjects, and whenever one of the subobjects got deleted, the rest must be deleted as well. This was implemented by having the dtor of the subobject destruct the container, which, according to refcount thinking, should "automatically" clean up everything. Except that it didn't, because this implementation was wrong on so many levels... the first being that the subobject can't have its dtor invoked unless the container didn't contribute to the reference count, and of course, once the dtor is invoked, it tries to destruct the container by decreasing *its* reference count, which makes one wonder who's keeping the reference to the container, and what happens with the container's reference to the current subobject. This was the case that really needed a GC, because there are just too many cross-references going on. Had we used a GC to begin with, we'd never have gone down this slide of increasingly wrong code. And this was just the one instance we discovered; who knows how many other such abuses are still lurking in the code somewhere. Now you may argue that an ARC wouldn't permit this sort of abuses, but the fact of the matter is, the same people who can't write bug-free malloc/free code are the same people who can't write bug-free reference counting code, and since we have very little say in who we work with, we're going to have to continue dealing with stupid bugs of this sort. Even with an ARC, we're going to get people writing stupid code that create memory leaks with cycles and what-not. To fully address this issue at its root, what we really need is a GC, since it eliminates an entire class of memory bugs. Except that the decision makers are hardcore C coders who have irrational GC phobia (and I used to be one of them), so this will probably never happen in *this* job. (Maybe I should consider working for Sociomantic or something...) T -- Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are gone to milk the bull. -- Sam. Johnson
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Granted, low latency apps are the minority. C++ apps are a 
minority compared to C# and Java combined. Beyond any doubt there 
is merit to a GC in all apps that don't have to squeeze the last 
bit of performance out of the machine, yes, the vast majority, I 
know.

But maybe we should ask ourselves if it makes sense to get the 
low latency developers on board, especially if D's syntax remains 
basically unchanged in the process. Heck, we could even keep 
'new' instead of Rusts '~' for owning pointers.

On Saturday, 1 February 2014 at 19:40:20 UTC, Adam Wilson wrote:
 On Sat, 01 Feb 2014 06:33:15 -0800, Frank Bauer <x y.com> wrote:
 ARC would be a plus for heavily interdependent code. But it 
 doesn't beat unique_ptr semantics in local use that free their 
 memory immediately as they go out of scope or are reassigned.


Ahem ... no you don't. I tried to make my point as clear as possible. Unique_ptr *semantics*, e.g. Rusts 'let x = ~X' *without* GC is as easy to use as 'auto x = new X' *with* GC. In both cases the runtime takes care of freeing the object without user intervention. So newbies should not be turned off, if that is your concern. In general, I don't understand your arguments against dumping the GC if it does not make much difference in your code anyways? Getting rid of mandatory GC would not take away one iota of your D expressiveness. It means more work on the part of D's implementers, to be sure. I'm close to Stuttgart and Ingolstadt where Mercedes Benz and Audi produce their flagship cars for a tiny minority of customers. Yet their reputation with the average buyer relies heavily on how well their top of the line products fare. Do we want D to be the flagship?
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 16:07:20 -0800, Frank Bauer <y z.com> wrote:

 Granted, low latency apps are the minority. C++ apps are a minority  
 compared to C# and Java combined. Beyond any doubt there is merit to a  
 GC in all apps that don't have to squeeze the last bit of performance  
 out of the machine, yes, the vast majority, I know.

 But maybe we should ask ourselves if it makes sense to get the low  
 latency developers on board, especially if D's syntax remains basically  
 unchanged in the process. Heck, we could even keep 'new' instead of  
 Rusts '~' for owning pointers.

 On Saturday, 1 February 2014 at 19:40:20 UTC, Adam Wilson wrote:
 On Sat, 01 Feb 2014 06:33:15 -0800, Frank Bauer <x y.com> wrote:
 ARC would be a plus for heavily interdependent code. But it doesn't  
 beat unique_ptr semantics in local use that free their memory  
 immediately as they go out of scope or are reassigned.


Ahem ... no you don't. I tried to make my point as clear as possible. Unique_ptr *semantics*, e.g. Rusts 'let x = ~X' *without* GC is as easy to use as 'auto x = new X' *with* GC. In both cases the runtime takes care of freeing the object without user intervention. So newbies should not be turned off, if that is your concern. In general, I don't understand your arguments against dumping the GC if it does not make much difference in your code anyways? Getting rid of mandatory GC would not take away one iota of your D expressiveness. It means more work on the part of D's implementers, to be sure. I'm close to Stuttgart and Ingolstadt where Mercedes Benz and Audi produce their flagship cars for a tiny minority of customers. Yet their reputation with the average buyer relies heavily on how well their top of the line products fare. Do we want D to be the flagship?

The problem isn't syntax, it's semantics. It's not 'new' versus '~' (which by the way is not easy to type on a standard US keyboard, it's a shift+key that's in an an uncomfortable position). It's the number of rules and special cases around what happens when I do something. You're right, you are not asking for a big syntactic change, and I know that. What you are asking for is a massive semantics change, and that means new rules and most likely, more of them. Let's look at some numbers. TIOBE, Go is #36, Rust does not even make the Top 100. On TIOBE, there are three non-GC languages in the entire top 20 (C, C++, Obj-C) the rest feature a non-ARC GC of some form. I am not concerned with how the languages manage memory under-the-hood, only how they present their memory management semantics to the user. It's interesting that you bring up Mercedes and Audi. Yes, their reputation is defined by their high-end products. But how many people actually buy an R8 or an SLK Black? The vast majority or everyday don't buy a completely bonkers super-car, they by cars that get the job done. Why do Toyota and Volkswagen produce the most cars in the world? Not because either company (yes, I am aware of VW's 13 name-plates, I'm talking about the VW name-plate here) is particularly renowned for producing supercars, but because their cars reliably get the job done. A Corolla is not the fastest, it's not the prettiest, and it's certainly not the most expensive. More Corolla's have been sold world-wide (40 million) than any other single car model in history. You want a super-car, with all the fancy buttons and knobs and options, and ARC is very analogous to a super-car in this respect. In fact many of the complaints I see about Obj-C's ARC have to do with the semantics of it. ARC is by definition semantically more complex than a GC. But most people don't want those options, they just want to put the key in and have the car reliably do what they ask within reasonable boundaries and that is what a GC does best. The semantics of a good GC are nearly transparent. But that's where the analogy breaks down. You can stratify car models. You can't deliver multiple exclusive strata in a programming language. A language is what it is, there are no multiple levels. You say, let's bring on the low-latency developers to prove how fast D is and how it has all these great options. The problem is most programmers don't need those options on a day to day basis, but having to carry around the semantics for them everywhere is wearying. I say, let's fix the GC to so that the masses can do what they need to do without any fuss. Most car drivers aren't Jeremy Clarkson. And most programmers aren't real-time embedded developers. A friend of mine has a saying "Sell to the masses, eat with the classes. Sell to the classes, eat with the masses." It's a catchy way of saying, you don't make money or gain popularity by catering to the most demanding of users or the most wealthy of clients. You gain popularity by catering to the NEEDS of the everyday user, or the middle-class client. The reason I am being to vehement, is that I unlike most people here, I see D as needing to focus on making the experience better for the every day programmer, NOT the experience of the most demanding users. And as C++ has proven, the two are distinct, writing a language that caters to the high-end often precludes the semantics that appeal to the everyday precisely because those semantics are too slow. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b5d58b4b8167104f162bca8
Content-Type: text/plain; charset=UTF-8

On 1 February 2014 22:04, JR <zorael gmail.com> wrote:

 On Saturday, 1 February 2014 at 05:36:44 UTC, Manu wrote:

 I write realtime and memory-constrained software (console games), and for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable memory
 footprint of the application. You can't make any guarantees or predictions
 about the GC, which is fundamentally incompatible with realtime software.

even worse.) (Maybe this would be better posted in D.learn; if so I'll crosspost.) In your opinion, of how much value would deadlining be? As in, "okay handyman, you may sweep the floor now BUT ONLY FOR 6 MILLISECONDS; whatever's left after that you'll have to take care of next time, your pride as a professional Sweeper be damned"?

This has been my only suggestion for years. Although 6ms is way too much (almost half a frame), 500us is more realistic (a little over 3% of a frame, still quite a lot of time). It obviously doesn't address memory footprint, but you would get the
 illusion of determinism in cases similar to where race-to-idle approaches
 work. Inarguably, this wouldn't apply if the goal is to render as many
 frames per second as possible, such as for non-console shooters where
 tearing is not a concern but latency is very much so.

If it were running a small amount every frame, maybe the memory footprint would stay within a predictable level... don't know. I'd be interested in experimenting with this, but as far as I know, nobody knows how to do it. I'm very much a layman in this field, but I'm trying to soak up as much
 knowledge as possible, and most of it from threads like these. To my
 uneducated eyes, an ARC collector does seem like the near-ideal solution --
 assuming, as always, the code is written with the GC in mind. But am I
 right in gathering that it solves two-thirds of the problem? You don't need
 to scan the managed heap, but when memory is actually freed is still
 non-deterministic and may incur pauses, though not necessarily a
 program-wide stop. Aye?

I'm not sure what you mean. If you mean pauses because it's simply doing work to free things, then that's deterministically issued workload just like anything else and can (should) be budgeted. It doesn't disappear in a GC context, it just happens in bursts at unpredictable times. Like you say, it doesn't stop the world, no need for that. Additionally, it leaves room for case-by-case flexibility in destruction approach. If something takes some time to destroy, and you're okay to have it hang around for a while, you can easily plonk it in a dead list waiting for some idle time to work through it; thus truly allows the opportunity to use idle time to clean stuff up since you don't need to scan the heap every time, a list of things waiting to be destroyed is already known. You could also add dead objects to a list which can be processed by another thread (without stopping all threads). There is far more opportunity to implement the destruction pattern that suits your application, and for those not interested in implementing their own destruction patterns, it would be easy to offer a library function setDestructionPattern(patternType); which would configure a particular policy application-wide, and they'd never have to think about it again. At the same time, Lucarella's dconf slides were very, very attractive. I
 gather that allocations themselves may become slower with a concurrent
 collector, but collection times in essence become non-issues. Technically
 parallelism doesn't equate to free CPU time; but that it more or less *is*
 assuming there is a cores/thread to spare. Wrong?

IIRC, it assumes a particular operating system, and required significant resource overhead. I recall being impressed, but concluding that it wasn't a solution that could be merged into D across the board. I'd need to go back and revisit the slides... Lastly, am I right in understanding precise collectors as identical to the
 stop-the-world collector we currently have, but with a smarter allocation
 scheme resulting in a smaller managed heap to scan? With the additional
 bonus of less false pointers. If so, this would seem like a good
 improvement to the current implementation, with the next increment in the
 same direction being a generational gc.

I think the reduction in heap to scan is due to only following what are known to be pointers, but I don't think that reduces the scan volume, the memory footprint of your app is the same (more because there's new metadata), but it wouldn't chase false pointers anymore. I recall most agreed that it was a good improvement (although it benchmarked something like 5% slower), I would appreciate it simply for the improved precision... but it doesn't solve the problem in any way. The practical application for me is that it wouldn't leak (much) memory. I would *dearly* love to have concurrency in whatever we end up with,
 though. For a multi-core personal computer threads are free lunches, or
 close enough so. Concurrentgate and all that jazz.

The kicker for me with the whole GC thing, is that as long as I've been talking about it around here (among a whole bunch of experts), I've been saying that an acceptable GC would require to 1. not stop the world, 2. support incremental collection/time-slicing, so I can budget it a maximum amount of time per frame. I'm yet to hear anyone suggest how they can even IMAGINE writing a collector like that. As far as I can tell, the consensus is, that it's impossible. Certainly, nobody has ever given any suggestions how it could possibly be done in the future, or steps in that direction, which says a lot to me. I've abandoned the idea as unrealistic fantasy. --047d7b5d58b4b8167104f162bca8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= February 2014 22:04, JR <span dir=3D"ltr">&lt;<a href=3D"mailto:zorael gma= il.com" target=3D"_blank">zorael gmail.com</a>&gt;</span> wrote:<br><blockq= uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"> <div class=3D"im">On Saturday, 1 February 2014 at 05:36:44 UTC, Manu wrote:= <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I write realtime and memory-constrained software (console games), and for<b= r> me, I think the biggest issue that can never be solved is the<br> non-deterministic nature of the collect cycles, and the unknowable memory<b= r> footprint of the application. You can&#39;t make any guarantees or predicti= ons<br> about the GC, which is fundamentally incompatible with realtime software.<b= r> </blockquote></div> (tried to manually fix ugly linebreaks here, so apologies if it turns out e= ven worse.)<br> <br> (Maybe this would be better posted in D.learn; if so I&#39;ll crosspost.)<b= r> <br> In your opinion, of how much value would deadlining be? As in, &quot;okay h= andyman, you may sweep the floor now BUT ONLY FOR 6 MILLISECONDS; whatever&= #39;s left after that you&#39;ll have to take care of next time, your pride= as a professional Sweeper be damned&quot;?<br> </blockquote><div><br></div><div>This has been my only suggestion for years= . Although 6ms is way too much (almost half a frame), 500us is more realist= ic (a little over 3% of a frame, still quite a lot of time).</div><div> <br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bord= er-left:1px #ccc solid;padding-left:1ex"> It obviously doesn&#39;t address memory footprint, but you would get the il= lusion of determinism in cases similar to where race-to-idle approaches wor= k. Inarguably, this wouldn&#39;t apply if the goal is to render as many fra= mes per second as possible, such as for non-console shooters where tearing = is not a concern but latency is very much so.<br> </blockquote><div><br></div><div>If it were running a small amount every fr= ame, maybe the memory footprint would stay within a predictable level... do= n&#39;t know. I&#39;d be interested in experimenting with this, but as far = as I know, nobody knows how to do it.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> I&#39;m very much a layman in this field, but I&#39;m trying to soak up as = much knowledge as possible, and most of it from threads like these. To my u= neducated eyes, an ARC collector does seem like the near-ideal solution -- = assuming, as always, the code is written with the GC in mind. But am I righ= t in gathering that it solves two-thirds of the problem? You don&#39;t need= to scan the managed heap, but when memory is actually freed is still non-d= eterministic and may incur pauses, though not necessarily a program-wide st= op. Aye?<br> </blockquote><div><br></div><div>I&#39;m not sure what you mean. If you mea= n pauses because it&#39;s simply doing work to free things, then that&#39;s= deterministically issued workload just like anything else and can (should)= be budgeted. It doesn&#39;t disappear in a GC context, it just happens in = bursts at unpredictable times.</div> <div>Like you say, it doesn&#39;t stop the world, no need for that. Additio= nally, it leaves room for case-by-case flexibility in destruction approach.= If something takes some time to destroy, and you&#39;re okay to have it ha= ng around for a while, you can easily plonk it in a dead list waiting for s= ome idle time to work through it; thus truly allows the opportunity to use = idle time to clean stuff up since you don&#39;t need to scan the heap every= time, a list of things waiting to be destroyed is already known.</div> <div>You could also add dead objects to a list which can be processed by an= other thread (without stopping all threads).</div><div>There is far more op= portunity to implement the destruction pattern that suits your application,= and for those not interested in implementing their own destruction pattern= s, it would be easy to offer a library function setDestructionPattern(patte= rnType); which would configure a particular policy application-wide, and th= ey&#39;d never have to think about it again.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> At the same time, Lucarella&#39;s dconf slides were very, very attractive. = I gather that allocations themselves may become slower with a concurrent co= llector, but collection times in essence become non-issues. Technically par= allelism doesn&#39;t equate to free CPU time; but that it more or less *is*= assuming there is a cores/thread to spare. Wrong?<br> </blockquote><div><br></div><div>IIRC, it assumes a particular operating sy= stem, and required significant resource overhead. I recall being impressed,= but concluding that it wasn&#39;t a solution that could be merged into D a= cross the board.</div> <div>I&#39;d need to go back and revisit the slides...</div><div><br></div>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Lastly, am I right in understanding precise collectors as identical to the = stop-the-world collector we currently have, but with a smarter allocation s= cheme resulting in a smaller managed heap to scan? With the additional bonu= s of less false pointers. If so, this would seem like a good improvement to= the current implementation, with the next increment in the same direction = being a generational gc.<br> </blockquote><div><br></div><div>I think the reduction in heap to scan is d= ue to only following what are known to be pointers, but I don&#39;t think t= hat reduces the scan volume, the memory footprint of your app is the same (= more because there&#39;s new metadata), but it wouldn&#39;t chase false poi= nters anymore.</div> <div>I recall most agreed that it was a good improvement (although it bench= marked something like 5% slower), I would appreciate it simply for the impr= oved precision... but it doesn&#39;t solve the problem in any way. The prac= tical application for me is that it wouldn&#39;t leak (much) memory.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> I would *dearly* love to have concurrency in whatever we end up with, thoug= h. For a multi-core personal computer threads are free lunches, or close en= ough so. Concurrentgate and all that jazz.<br> </blockquote></div><br></div><div class=3D"gmail_extra">The kicker for me w= ith the whole GC thing, is that as long as I&#39;ve been talking about it a= round here (among a whole bunch of experts), I&#39;ve been saying that an a= cceptable GC would require to 1. not stop the world, 2. support incremental= collection/time-slicing, so I can budget it a maximum amount of time per f= rame.</div> <div class=3D"gmail_extra">I&#39;m yet to hear anyone suggest how they can = even IMAGINE writing a collector like that. As far as I can tell, the conse= nsus is, that it&#39;s impossible.</div><div class=3D"gmail_extra">Certainl= y, nobody has ever given any suggestions how it could possibly be done in t= he future, or steps in that direction, which says a lot to me. I&#39;ve aba= ndoned the idea as unrealistic fantasy.</div> </div> --047d7b5d58b4b8167104f162bca8--
Feb 01 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11330964308ab104f162c5d1
Content-Type: text/plain; charset=UTF-8

On 1 February 2014 22:35, develop32 <develop32 gmail.com> wrote:

 On Saturday, 1 February 2014 at 12:29:10 UTC, Nick Sabalausky wrote:

 Come to think of it, I wonder what language Frostbite 3 uses for
 game-level "scripting" code. From what I've heard about them, Frostbite 3
 and Unreal Engine 4 both sound like they have some notable similarities
 with Unity3D (from the standpoint of the user-experience for game
 developers), although AIUI Unreal Engine 4 still uses C++ for game code
 (unless the engine user wants to to do lua or something on their own). I
 imagine Frostbite's probably the same, C++, but I haven't actually heard
 anything.

Frostbite has an option for both Lua and C++. C++ is the preferred one. Unreal Engine went from Unrealscript to C++ and everyone applauded that.

Correct, this is the general trend I see. idTech also uses C/C++ DLL's for 'scripting'. At Remedy, we were keen to migrate our scripts to native code in the same way, except using D instead of C++. The proof of concept is solid. --001a11330964308ab104f162c5d1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= February 2014 22:35, develop32 <span dir=3D"ltr">&lt;<a href=3D"mailto:dev= elop32 gmail.com" target=3D"_blank">develop32 gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Saturday, 1 February 2014 at 12:29:10 UTC, Nick Sabala= usky wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Come to think of it, I wonder what language Frostbite 3 uses for game-level= &quot;scripting&quot; code. From what I&#39;ve heard about them, Frostbite= 3 and Unreal Engine 4 both sound like they have some notable similarities = with Unity3D (from the standpoint of the user-experience for game developer= s), although AIUI Unreal Engine 4 still uses C++ for game code (unless the = engine user wants to to do lua or something on their own). I imagine Frostb= ite&#39;s probably the same, C++, but I haven&#39;t actually heard anything= .<br> </blockquote> <br></div> Frostbite has an option for both Lua and C++. C++ is the preferred one.<br> <br> Unreal Engine went from Unrealscript to C++ and everyone applauded that.<br=

e general trend I see. idTech also uses C/C++ DLL&#39;s for &#39;scripting&= #39;.</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">= At Remedy, we were keen to migrate our scripts to native code in the same w= ay, except using D instead of C++. The proof of concept is solid.</div> </div> --001a11330964308ab104f162c5d1--
Feb 01 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e013d0dc00c353404f162d694
Content-Type: text/plain; charset=UTF-8

On 2 February 2014 00:33, Frank Bauer <x y.com> wrote:

 On Saturday, 1 February 2014 at 09:27:18 UTC, Adam Wilson wrote:

 For the vast majority of use cases, a GC is the right call and D has to
 cater to the majority if it wants to gain any significant mindshare at all.
 You don't grow by increasing specialization...

So catering to the C++ crowd who shun Java and C# not so much for JITing their code but precisely for their forced use of a GC is a "minority" issue then? Again, please take those of us more serious who don't like a GC to interfere with their business and if it's only for "programming hygiene" reasons. As you consider handling low latency requirements undue "specialisation" anyways. On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:
 To my uneducated eyes, an ARC collector does seem like the near-ideal
 solution

ARC would be a plus for heavily interdependent code. But it doesn't beat unique_ptr semantics in local use that free their memory immediately as they go out of scope or are reassigned.

Why wouldn't ARC do this? I would *dearly* love to have concurrency in whatever we end up with,
 though. For a multi-core personal computer threads are free lunches, or
 close enough so.

No they are not. I want to make good use of all the cores I have available and I don't want to share them a bit with the GC. It's myyy ... precious. As you may have guessed, my criticism is harsh because I love the language :)

--089e013d0dc00c353404f162d694 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 2= February 2014 00:33, Frank Bauer <span dir=3D"ltr">&lt;<a href=3D"mailto:x= y.com" target=3D"_blank">x y.com</a>&gt;</span> wrote:<br><blockquote clas= s=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pad= ding-left:1ex"> <div class=3D"im">On Saturday, 1 February 2014 at 09:27:18 UTC, Adam Wilson= wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> For the vast majority of use cases, a GC is the right call and D has to cat= er to the majority if it wants to gain any significant mindshare at all. Yo= u don&#39;t grow by increasing specialization...<br> </blockquote> <br></div> So catering to the C++ crowd who shun Java and C# not so much for JITing th= eir code but precisely for their forced use of a GC is a &quot;minority&quo= t; issue then? Again, please take those of us more serious who don&#39;t li= ke a GC to interfere with their business and if it&#39;s only for &quot;pro= gramming hygiene&quot; reasons. As you consider handling low latency requir= ements undue &quot;specialisation&quot; anyways.<div class=3D"im"> <br> <br> On Saturday, 1 February 2014 at 12:04:56 UTC, JR wrote:<br> </div><div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> To my uneducated eyes, an ARC collector does seem like the near-ideal solut= ion<br> </blockquote> <br></div> ARC would be a plus for heavily interdependent code. But it doesn&#39;t bea= t unique_ptr semantics in local use that free their memory immediately as t= hey go out of scope or are reassigned.</blockquote><div><br></div><div> Why wouldn&#39;t ARC do this?=C2=A0</div><div><br></div><blockquote class= =3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd= ing-left:1ex"><div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I would *dearly* love to have concurrency in whatever we end up with, thoug= h. For a multi-core personal computer threads are free lunches, or close en= ough so.<br> </blockquote> <br></div> No they are not. I want to make good use of all the cores I have available = and I don&#39;t want to share them a bit with the GC. It&#39;s myyy ... pre= cious.<br> <br> As you may have guessed, my criticism is harsh because I love the language = :)<br> </blockquote></div><br></div></div> --089e013d0dc00c353404f162d694--
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 17:58:01 -0800, Manu <turkeyman gmail.com> wrote:

 On 1 February 2014 22:04, JR <zorael gmail.com> wrote:

 On Saturday, 1 February 2014 at 05:36:44 UTC, Manu wrote:

 I write realtime and memory-constrained software (console games), and  
 for
 me, I think the biggest issue that can never be solved is the
 non-deterministic nature of the collect cycles, and the unknowable  
 memory
 footprint of the application. You can't make any guarantees or  
 predictions
 about the GC, which is fundamentally incompatible with realtime  
 software.

out even worse.) (Maybe this would be better posted in D.learn; if so I'll crosspost.) In your opinion, of how much value would deadlining be? As in, "okay handyman, you may sweep the floor now BUT ONLY FOR 6 MILLISECONDS; whatever's left after that you'll have to take care of next time, your pride as a professional Sweeper be damned"?

This has been my only suggestion for years. Although 6ms is way too much (almost half a frame), 500us is more realistic (a little over 3% of a frame, still quite a lot of time). It obviously doesn't address memory footprint, but you would get the
 illusion of determinism in cases similar to where race-to-idle  
 approaches
 work. Inarguably, this wouldn't apply if the goal is to render as many
 frames per second as possible, such as for non-console shooters where
 tearing is not a concern but latency is very much so.

If it were running a small amount every frame, maybe the memory footprint would stay within a predictable level... don't know. I'd be interested in experimenting with this, but as far as I know, nobody knows how to do it. I'm very much a layman in this field, but I'm trying to soak up as much
 knowledge as possible, and most of it from threads like these. To my
 uneducated eyes, an ARC collector does seem like the near-ideal  
 solution --
 assuming, as always, the code is written with the GC in mind. But am I
 right in gathering that it solves two-thirds of the problem? You don't  
 need
 to scan the managed heap, but when memory is actually freed is still
 non-deterministic and may incur pauses, though not necessarily a
 program-wide stop. Aye?

I'm not sure what you mean. If you mean pauses because it's simply doing work to free things, then that's deterministically issued workload just like anything else and can (should) be budgeted. It doesn't disappear in a GC context, it just happens in bursts at unpredictable times. Like you say, it doesn't stop the world, no need for that. Additionally, it leaves room for case-by-case flexibility in destruction approach. If something takes some time to destroy, and you're okay to have it hang around for a while, you can easily plonk it in a dead list waiting for some idle time to work through it; thus truly allows the opportunity to use idle time to clean stuff up since you don't need to scan the heap every time, a list of things waiting to be destroyed is already known. You could also add dead objects to a list which can be processed by another thread (without stopping all threads). There is far more opportunity to implement the destruction pattern that suits your application, and for those not interested in implementing their own destruction patterns, it would be easy to offer a library function setDestructionPattern(patternType); which would configure a particular policy application-wide, and they'd never have to think about it again. At the same time, Lucarella's dconf slides were very, very attractive. I
 gather that allocations themselves may become slower with a concurrent
 collector, but collection times in essence become non-issues.  
 Technically
 parallelism doesn't equate to free CPU time; but that it more or less  
 *is*
 assuming there is a cores/thread to spare. Wrong?

IIRC, it assumes a particular operating system, and required significant resource overhead. I recall being impressed, but concluding that it wasn't a solution that could be merged into D across the board. I'd need to go back and revisit the slides... Lastly, am I right in understanding precise collectors as identical to the
 stop-the-world collector we currently have, but with a smarter  
 allocation
 scheme resulting in a smaller managed heap to scan? With the additional
 bonus of less false pointers. If so, this would seem like a good
 improvement to the current implementation, with the next increment in  
 the
 same direction being a generational gc.

I think the reduction in heap to scan is due to only following what are known to be pointers, but I don't think that reduces the scan volume, the memory footprint of your app is the same (more because there's new metadata), but it wouldn't chase false pointers anymore. I recall most agreed that it was a good improvement (although it benchmarked something like 5% slower), I would appreciate it simply for the improved precision... but it doesn't solve the problem in any way. The practical application for me is that it wouldn't leak (much) memory. I would *dearly* love to have concurrency in whatever we end up with,
 though. For a multi-core personal computer threads are free lunches, or
 close enough so. Concurrentgate and all that jazz.

The kicker for me with the whole GC thing, is that as long as I've been talking about it around here (among a whole bunch of experts), I've been saying that an acceptable GC would require to 1. not stop the world, 2. support incremental collection/time-slicing, so I can budget it a maximum amount of time per frame. I'm yet to hear anyone suggest how they can even IMAGINE writing a collector like that. As far as I can tell, the consensus is, that it's impossible. Certainly, nobody has ever given any suggestions how it could possibly be done in the future, or steps in that direction, which says a lot to me. I've abandoned the idea as unrealistic fantasy.

It is impossible with a conservative GC such as the current D implementation. Boehm tried with his C++ collector but could never get beyond the basic Mark-LazySweep because C++ enforces a conservative mindset. When you have precision a number of options open up. Firstly, you can now parallelize all phases because pointers are precisely known. And second, you can start to do things like moving (a prereq for generational collection) because you don't have to pin things that you conservatively think are pointers. Now that you have a generational collector, it's trivial to do an incremental collection on one or more generations. So therefore precision is the first step, and more to the point it's already been done in D, VisualD uses a precise version of the stock D GC and special build of the compiler with precise information emitted at compile time. That was what Rainer's talk was about at DConf last year. Getting this merged into the mainline will mean that there is finally enough information for us to go nuts building an actually useful GC in D, but until then you really can't progress beyond what we have right now. It's quite possible, it's just a matter of getting the worked vetted at merged. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Agreed. We must keep the GC in D and not change its semantics 
(certainly its performance to be sure).

I would not even want to go the Rust way (their latest anyway, it 
keeps changing, mostly for the better) of relegating the GC to a 
library type Gc<T>. It should remain part of the language for the 
majority of developers who benefit from it.

Again, I agree, making life harder for people who wish to use GC 
would not be acceptable. I enjoy the GC for non-critical apps.

What I am for is an opt-in solution for GC, not necessarily, as 
the OP implies, favoring smart pointers over GC, but for both 
being on an equal footing.

Let developers who strive for ultimate performance choose their 
tool and let D set the standard for what is possible with a 
modern language.

It might be a minority issue overall, but a significant one here 
in these forums, from what I gather.

The D implementers will decide if the two ways of memory 
management can be reconciled and if it's worth the effort. +1 
from me on both.
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 18:16:07 -0800, Frank Bauer <y z.com> wrote:

 Agreed. We must keep the GC in D and not change its semantics (certainly  
 its performance to be sure).

 I would not even want to go the Rust way (their latest anyway, it keeps  
 changing, mostly for the better) of relegating the GC to a library type  
 Gc<T>. It should remain part of the language for the majority of  
 developers who benefit from it.

 Again, I agree, making life harder for people who wish to use GC would  
 not be acceptable. I enjoy the GC for non-critical apps.

 What I am for is an opt-in solution for GC, not necessarily, as the OP  
 implies, favoring smart pointers over GC, but for both being on an equal  
 footing.

 Let developers who strive for ultimate performance choose their tool and  
 let D set the standard for what is possible with a modern language.

 It might be a minority issue overall, but a significant one here in  
 these forums, from what I gather.

 The D implementers will decide if the two ways of memory management can  
 be reconciled and if it's worth the effort. +1 from me on both.

I am for that as well. GC by default, then opt-in to other semantics, even if via new syntax. That will give the flexibility to do what you need to do when you need to do it without significantly increasing the costs to the common use case... THAT is a laudable position to take. And I'll happily root for support different resource management paradigms, because the one truth is that there is no one right way. But as soon as I start seeing arguments along the lines of "X is stupid because it doesn't work for my use case so we must replace it with Y" I'll start fighting. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Yes, that would be stupid.
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 02:05:16 UTC, Manu wrote:
 On 2 February 2014 00:33, Frank Bauer <x y.com> wrote:
 ARC would be a plus for heavily interdependent code. But it 
 doesn't beat
 unique_ptr semantics in local use that free their memory 
 immediately as
 they go out of scope or are reassigned.

Why wouldn't ARC do this?

Because in a local scope you could decide when to free the object at compile time (i.e. end of scope) and not drag a reference count around when it is not necessary?
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Just to clarify: I would like to have the following pointers. 
I'll start with the GC ones to maybe reach some consensus quicker 
;)

1. GC(T)
Pointer to a heap allocated T, garbage collected.

2. ARC(T)
Reference counted pointer to a heap allocated T, compiler manages 
lifetime automatically via retain/ release.

3. Owning(T)
Pointer to a heap allocated T, freed when pointer goes out of 
scope or is reassigned (some special interactions with 
Borrowed(T) on top of that).

4. Borrowed(T)
Pointer to a T, not necessarily heap allocated, has no impact on 
memory allocation/ deallocation of the T whatsoever, if assigned 
from an Owning(T) prevents that owning pointer from being freed 
or reassigned as long as the borrowed pointer is in scope.

Performance, or, should I say, meddling of the compiler and 
runtime in your code, is from worst to best.

We could argue over which pointer concepts would be granted the 
honor of receiving their own D definition syntax (new, ~,  , &, 
whatever).
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Of course, the big deal is: if I opt for ARC(T), Owning(T) and 
Borrowed(T) exclusively, the GC has no business running for a 
microsecond in my code or the standard library functions I call.
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 19:15:38 -0800, Frank Bauer <y z.com> wrote:

 Of course, the big deal is: if I opt for ARC(T), Owning(T) and  
 Borrowed(T) exclusively, the GC has no business running for a  
 microsecond in my code or the standard library functions I call.

That's not a guarantee that can be reasonably made or demanded. You have no idea if a library you are using is written using the GC and turning those on would have no way to impact the allocation scheme of the library. As long as there is a possibility of a GC, D will have to pass an active GC instance to any libraries that are loaded. The GC only runs right now when a GC allocation is requested. I've been saying this all over the forums but people seem to persist in the mindset that the GC is a black-box that runs at completely random intervals, for random intervals, all determined by voodoo and black magic. It's not, and it doesn't. The GC only runs when an allocation is requested that cannot be filled from the current available heap. If you don't use the GC, by definition it will not run, unless a library you depend on uses it. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 03:38:08 UTC, Adam Wilson wrote:
 You have no idea if a library you are using is written using 
 the GC.

Well, the library writer might tell me, if she used GC(T) (the linker might as well) ...
 If you don't use the GC, by definition it will not run, unless 
 a library you depend on uses it.

big UNLESS!
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu 
wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

 Andrei

Only daydreaming ...
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 20:17:47 -0800, Frank Bauer <y z.com> wrote:

 On Sunday, 2 February 2014 at 03:38:08 UTC, Adam Wilson wrote:
 You have no idea if a library you are using is written using the GC.

Well, the library writer might tell me, if she used GC(T) (the linker might as well) ...
 If you don't use the GC, by definition it will not run, unless a  
 library you depend on uses it.

big UNLESS!

Maybe, but my point is that even with no allocations in phobos, you'd have to verify that ALL the libraries you use don't GC allocate. That's why so many people in the ARC camp want to make it the only GC system in D. Multiple GC systems increase your cognitive load considerably. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu 
wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

The complexity is already there, it is just hidden. But most the capacity to deal with it is already there too: Owning!T is just a struct with ~this() { free(me); }. Referencing counting is just a struct with this(this) { payload.ref++; } ~this() { payload.ref--; } (with a few optimizations to remove ref++; ref--;, which IMO should be there anyway!) Borrowed is a reference we can use or copy, but must not free (delete is deprecated!) and ought not store. GC is a reference that we don't have to free AND is ok to store. The language just conflates the latter two. And the spec even gives us some hope that it isn't supposed to with the scope storage class, though IMO the difference is more of a type thing than a storage class thing. A GC reference should implicitly convert to a borrowed reference (e.g. alias slice this;) but not vice versa. The only way to go from borrowed -> GC (or any other really) would be cowboy it with cast() or copying the contents. BTW I like a lot of the ideas Rust has in this regard, but I don't think we have to go all the way they did. But having: char[] foo = new char[](1024); and char[1024] buffer; char[] foo = buffer[]; both work and yield exactly the same type is trouble.
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 04:26:16 UTC, Adam D. Ruppe wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu 
 wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

The complexity is already there, it is just hidden. But most the capacity to deal with it is already there too: Owning!T is just a struct with ~this() { free(me); }. Referencing counting is just a struct with this(this) { payload.ref++; } ~this() { payload.ref--; } (with a few optimizations to remove ref++; ref--;, which IMO should be there anyway!) Borrowed is a reference we can use or copy, but must not free (delete is deprecated!) and ought not store. GC is a reference that we don't have to free AND is ok to store. The language just conflates the latter two. And the spec even gives us some hope that it isn't supposed to with the scope storage class, though IMO the difference is more of a type thing than a storage class thing. A GC reference should implicitly convert to a borrowed reference (e.g. alias slice this;) but not vice versa. The only way to go from borrowed -> GC (or any other really) would be cowboy it with cast() or copying the contents. BTW I like a lot of the ideas Rust has in this regard, but I don't think we have to go all the way they did. But having: char[] foo = new char[](1024); and char[1024] buffer; char[] foo = buffer[]; both work and yield exactly the same type is trouble.

I see light ... There is one little complication of owning pointers interacting with borrowed pointers that might prevent a straightforward approach (I'm sure there's more): owning pointers must not go out of scope or be reassigned as long as there are outstanding references to them alive (i.e. in scope as well). I don't see ATM how this could be done at zero cost without intrinsic compiler support. Only the compiler could enforce this free of cost as far as I see.
Feb 01 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sat, 01 Feb 2014 20:43:37 -0800, Frank Bauer <y z.com> wrote:

 On Sunday, 2 February 2014 at 04:26:16 UTC, Adam D. Ruppe wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei Alexandrescu wrote:
 Whoa, this won't work without an explosion in language complexity.

The complexity is already there, it is just hidden. But most the capacity to deal with it is already there too: Owning!T is just a struct with ~this() { free(me); }. Referencing counting is just a struct with this(this) { payload.ref++; } ~this() { payload.ref--; } (with a few optimizations to remove ref++; ref--;, which IMO should be there anyway!) Borrowed is a reference we can use or copy, but must not free (delete is deprecated!) and ought not store. GC is a reference that we don't have to free AND is ok to store. The language just conflates the latter two. And the spec even gives us some hope that it isn't supposed to with the scope storage class, though IMO the difference is more of a type thing than a storage class thing. A GC reference should implicitly convert to a borrowed reference (e.g. alias slice this;) but not vice versa. The only way to go from borrowed -> GC (or any other really) would be cowboy it with cast() or copying the contents. BTW I like a lot of the ideas Rust has in this regard, but I don't think we have to go all the way they did. But having: char[] foo = new char[](1024); and char[1024] buffer; char[] foo = buffer[]; both work and yield exactly the same type is trouble.

I see light ... There is one little complication of owning pointers interacting with borrowed pointers that might prevent a straightforward approach (I'm sure there's more): owning pointers must not go out of scope or be reassigned as long as there are outstanding references to them alive (i.e. in scope as well). I don't see ATM how this could be done at zero cost without intrinsic compiler support. Only the compiler could enforce this free of cost as far as I see.

I believe it should be possible to argue for the compiler support as long as it doesn't add any language level complexity. I too like how this works. However, full ARC presents quite a few more problems. You will certainly need to add at least one more keyword to the language ('arc' anyone?) and a metric ton of compiler support. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 04:23:35 UTC, Adam Wilson wrote:
 Maybe, but my point is that even with no allocations in phobos, 
 you'd have to verify that ALL the libraries you use don't GC 
 allocate. That's why so many people in the ARC camp want to 
 make it the only GC system in D. Multiple GC systems increase 
 your cognitive load considerably.

Good point. What we DON'T want is a library that comes in MySuperLib-GC, MySuperLib-ARC, MySuperLib-Owning and MySuperLib-Borrowed flavors. But I don't think that is necessary. A library function would typically take arguments by reference aka as a Borrowed(T). GC(T), ARC(T) and Owning(T) convert equally well to Borrowed(T). So somebody who uses only GC(T) in her code could use the library just as easily as someone who prefers Owning(T). The library would be compatible with the two, because it does not own the (reference to the) passed object. GC(T) or Owning(T) at the call site would free the object automatically (at the next collection cycle or when leaving the scope). Everybody is happy! All that is left for the caller is to choose whether she wants to use a library that uses GC *internally*. Interaction is possible in every combination: 1. I use GC(T), the library uses GC(T) internally -> works (with GC), 2. I use Owning(T), the library uses GC(T) internally -> works (with GC), 3. I use GC(T), the library uses Owning(T) internally -> works (with GC), 4. I use Owning(T), the library uses Owning(T) internally -> works (w/o GC).
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Of course to make everyone in 4 happy (i.e. me;) there would have 
to be a Phobos that uses Owning(T) internally exclusively.
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 05:30:02 UTC, Andrei Alexandrescu 
wrote:
 On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei 
 Alexandrescu wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei

^^ Lets do it while we're young ...
Feb 01 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 05:41:53 UTC, Andrei Alexandrescu 
wrote:
 Sure. While you're young typecheck this:

 class A {
    auto stuff() { ...; return this; }
 }


 Andrei

It would be a Borrowed!A, I would say, no matter how an A is allocated. A GC!A, an ARC!A and an Owning!A convert equally well to a Borrowed!A. As long as the Borrowed!A is in use (i.e. scope) after stuff() returns, the A object must not be freed (compiler enforced). Would that make sense?
Feb 01 2014
prev sibling next sibling parent "Martin Cejp" <minexew gmail.com> writes:
On Sunday, 2 February 2014 at 05:12:05 UTC, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 04:23:35 UTC, Adam Wilson wrote:
 Maybe, but my point is that even with no allocations in 
 phobos, you'd have to verify that ALL the libraries you use 
 don't GC allocate. That's why so many people in the ARC camp 
 want to make it the only GC system in D. Multiple GC systems 
 increase your cognitive load considerably.

Good point. What we DON'T want is a library that comes in MySuperLib-GC, MySuperLib-ARC, MySuperLib-Owning and MySuperLib-Borrowed flavors. But I don't think that is necessary. A library function would typically take arguments by reference aka as a Borrowed(T). GC(T), ARC(T) and Owning(T) convert equally well to Borrowed(T). So somebody who uses only GC(T) in her code could use the library just as easily as someone who prefers Owning(T). The library would be compatible with the two, because it does not own the (reference to the) passed object. GC(T) or Owning(T) at the call site would free the object automatically (at the next collection cycle or when leaving the scope). Everybody is happy! All that is left for the caller is to choose whether she wants to use a library that uses GC *internally*. Interaction is possible in every combination: 1. I use GC(T), the library uses GC(T) internally -> works (with GC), 2. I use Owning(T), the library uses GC(T) internally -> works (with GC), 3. I use GC(T), the library uses Owning(T) internally -> works (with GC), 4. I use Owning(T), the library uses Owning(T) internally -> works (w/o GC).

This exists. It's called C++. I've pretty much given up on GC and do all my allocations in the real-time path manually. In a game, there shouldn't be that much stuff to allocate once live, anyway.
Feb 02 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 2 February 2014 at 00:57:38 UTC, Adam Wilson wrote:
 ...

I often regret that D went so well as "yet another but faster" general-purpose language and is often viewed as such. And this is a niche overloaded with languages. Contrary to that, real-time / barebone stuff is rather narrow niche but there are only few usable languages there (and 0 good languages).
Feb 02 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 2 February 2014 at 07:35:21 UTC, Andrei Alexandrescu 
wrote:
 No.

 What I'm driving at is that right now the type is A, as is in a 
 variety of other situations. It would be a whole different 
 language that would be wholly incompatible with D.

What do you think about my `scope` proposal? It does not introduce any new syntax and should be somewhat in line with existing reference escape analysis. By the way, Sonke Ludwig has recently named scope + unique/isolated qualifiers as one of more promising D improvements for vibe.d : "having this in the core language would open up a lot of potential for performance and safety improvements and I feel that it needs more advertisement to really reach the decision makers - and of course the general community"
Feb 02 2014
prev sibling next sibling parent "JR" <zorael gmail.com> writes:
On Sunday, 2 February 2014 at 05:30:02 UTC, Andrei Alexandrescu 
wrote:
 On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei 
 Alexandrescu wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei

So, going forward, what would you say is the preferred direction to strive toward? I *seem* to remember reading here that you and Walter were increasingly growing to favor ARC, but I can't find the post. (Memory bitrot on my part is more than likely.)
Feb 02 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Sunday, 2 February 2014 at 07:35:21 UTC, Andrei Alexandrescu 
wrote:
 No.

 What I'm driving at is that right now the type is A, as is in a 
 variety of other situations. It would be a whole different 
 language that would be wholly incompatible with D.


 Andrei

Thx for giving it a thought though.
Feb 02 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 2 February 2014 at 11:23:56 UTC, JR wrote:
 I *seem* to remember reading here that you and Walter were 
 increasingly growing to favor ARC, but I can't find the post.

Personally, I think ARC is a waste of time and none of the arguments I've seen for it are convincing. Improving the Gc implementation would be a much bigger win.
Feb 02 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 2 February 2014 at 10:53:36 UTC, Dicebot wrote:
 What do you think about my `scope` proposal? It does not 
 introduce any new syntax and should be somewhat in line with 
 existing reference escape analysis.

What's the link to your proposal? I think I've read it before and liked it (scope would rock) but not sure it is the same thing on my mind.
Feb 02 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Saturday, 1 February 2014 at 19:11:28 UTC, Adam Wilson wrote:
 On Sat, 01 Feb 2014 09:57:41 -0800, Frustrated 
 <c1514843 drdrb.com> wrote:

 On Saturday, 1 February 2014 at 17:30:54 UTC, Sean Kelly wrote:
 On Saturday, 1 February 2014 at 11:40:37 UTC, Frustrated 
 wrote:
 And why does Phobos/runtime require the GC in so many cases 
 when
 it could just use an internal buffer? So much effort has 
 been put
 in to make the GC work that it simply has neglected all those
 that can't use it as it is.

This is the crux of the problem. It's not so much that D uses garbage collection as that Phobos is built in a way that prevents the reuse of existing buffers. It becomes much harder to sell the language to a GC averse group if it turns out they can't use the standard library as well. Though I guess Tango is an option here as a replacement.

Right, and because of the mentality that the GC is one size that fits all, phobos got this way. That mentality is still pervasive. I call it laziness. I think that at some point phobos will need to be rewritten... maybe more .net like(there sense of hierarchy and presentation is excellent). Maybe D needs a proper allocator sub system built in to get to that point?

You do realize that in .NET the BCL is not GC-optional right. It GC-allocates like a fiend. What they have is a monumentally better GC than D. .NET is actually proof that GC's can work quite well in all but the most extreme edge cases. Effectively what you are saying is ".NET does it well with a GC so we need to rewrite Phobos to not use a GC" ... Um wait, what? I am not saying to Phobos needs to allocate as much as it does, just that argument itself fails to met a basic standard of logical coherency.

No, you read way more in that than what I said. I am not talking about the .NET GC(I or I would have said ".NET GC"). You totally fabricated what you think I "effectively said". By writing the phobos like .NET, I mean, in terms of .NET's organization and structure.
Feb 02 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 2 February 2014 at 14:12:39 UTC, Adam D. Ruppe wrote:
 On Sunday, 2 February 2014 at 10:53:36 UTC, Dicebot wrote:
 What do you think about my `scope` proposal? It does not 
 introduce any new syntax and should be somewhat in line with 
 existing reference escape analysis.

What's the link to your proposal? I think I've read it before and liked it (scope would rock) but not sure it is the same thing on my mind.

There is no DIP for it, if you mean that. rvalue reference DIP has been implying it but mostly it is just short generalization for info already spread around the docs: http://forum.dlang.org/post/uopnrbrjwkxgejbcojxa forum.dlang.org It boils down to storage class that binds lifetime of entity to scope + qualifier that guarantees that no references/pointers to that data can be possibly stored at the end of scope in question. May need attribute inference to become usable with Phobos though. I have originally wanted to write it down in DIP form but at that time Andrei has said that implementing scope is out of question. Wondering if his opinion has changed :)
Feb 02 2014
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
 I have originally wanted to write it down in DIP form but at 
 that time Andrei has said that implementing scope is out of 
 question. Wondering if his opinion has changed :)

Feb 02 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 2 February 2014 at 07:35:21 UTC, Andrei Alexandrescu 
wrote:
 What I'm driving at is that right now the type is A, as is in a 
 variety of other situations. It would be a whole different 
 language that would be wholly incompatible with D.

A == Borrowed!A in my view. Consider this: class A { A foo() { return this; } } A a; void crashy() { ubyte[__traits(classInstanceSize, A)] Abuffer; auto localA = emplace!A(Abuffer); a = localA.foo(); } That's obviously wrong, when crashy returns, a is dead... but it has no way of knowing that. Let's consider we wrote a nice little Owned!T. Internally, it keeps that buffer, disables postblit, frees in the destructor and might have a release method to transfer ownership. All stuff we can do in the library today. Owned!A a; void crashy() { Owned!A localA = Owned!A(); a = localA; // prohibited, this(this) is annotated with disable a = localA.foo(); // should ALSO be prohibited, cannot convert A to Owned!A } The class object itself doesn't know how it is stored. If it returns this, it MUST assume it is borrowed unless it explicitly allocates a new thing. For all it knows, it was emplaced into a static array or malloced. A == Borrowed!A. typeof(this) == A. So, what about the global a up there? I think there's two solutions: 1) ideally, Owned!A has something like this: scope A borrow() { return payload; } alias borrow this; A a = localA; // legal, A is a local variable in the same or inner scope globalA = localA; // illegal, since borrow returns scope, it isn't allowed to escape 2) a worse but perhaps doable option would be A globalA; // illegal, non-local variable must not be borrowed struct foo { A a; // now foo is itself considered a borrowed reference type so the rules apply to it } GC!A globalA; // ok cool What's interesting about this is we could, in theory (and I've done a limited proof of concept), implement it in the library using a static assert in RTInfo! anyway i g2g might talk more later
Feb 02 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 2 February 2014 at 14:37:50 UTC, Namespace wrote:
 I have originally wanted to write it down in DIP form but at 
 that time Andrei has said that implementing scope is out of 
 question. Wondering if his opinion has changed :)


Well, wording was a bit different but from DIP36 discussion thread (http://forum.dlang.org/post/ylebrhjnrrcajnvtthtt forum.dlang.org):
 It defines a new language feature instead of improving the 
 existing ones. At this point in the development of the 
 language, our preference should be putting the existing 
 features in good order.

+
 You consider "scope" a new language feature?


I can be wrong but I have interpreted this as "no plans to actually implement scope in any forseeable future"
Feb 02 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Sunday, 2 February 2014 at 16:55:35 UTC, Andrei Alexandrescu
wrote:
 On 2/2/14, 3:23 AM, JR wrote:
 On Sunday, 2 February 2014 at 05:30:02 UTC, Andrei 
 Alexandrescu wrote:
 On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei 
 Alexandrescu wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei

So, going forward, what would you say is the preferred direction to strive toward? I *seem* to remember reading here that you and Walter were increasingly growing to favor ARC, but I can't find the post. (Memory bitrot on my part is more than likely.)

I think of the following foci for the first half of 2014: 1. Add nullable and provide a -nullable compiler flag to verify it. The attribute is inferred locally and for white-box functions (lambdas, templates), and required as annotation otherwise. References not annotated with nullable are statically enforced to never be null. 2. Work on Phobos to see what can be done about avoiding unnecessary allocation. Most likely we'll need to also add a nogc flag. 3. Work on adding tracing capabilities to allocators and see how to integrate them with the language and Phobos. 4. Work on the core language and druntime to see how to seamlessly accommodate alternate GC mechanisms such as reference counting. Andrei

Sounds like a step in the right direction. By having the nogc, phobos could be progressively converted and one can verify when modules, functions, and aggregates are gc dependent.
Feb 02 2014
prev sibling next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Sunday, 2 February 2014 at 16:55:35 UTC, Andrei Alexandrescu 
wrote:
 On 2/2/14, 3:23 AM, JR wrote:
 On Sunday, 2 February 2014 at 05:30:02 UTC, Andrei 
 Alexandrescu wrote:
 On 2/1/14, 8:18 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 03:38:03 UTC, Andrei 
 Alexandrescu wrote:
 Whoa, this won't work without an explosion in language 
 complexity.

 Andrei

Only daydreaming ...

No, it's a nightmare. Andrei

So, going forward, what would you say is the preferred direction to strive toward? I *seem* to remember reading here that you and Walter were increasingly growing to favor ARC, but I can't find the post. (Memory bitrot on my part is more than likely.)

I think of the following foci for the first half of 2014: 1. Add nullable and provide a -nullable compiler flag to verify it. The attribute is inferred locally and for white-box functions (lambdas, templates), and required as annotation otherwise. References not annotated with nullable are statically enforced to never be null. 2. Work on Phobos to see what can be done about avoiding unnecessary allocation. Most likely we'll need to also add a nogc flag. 3. Work on adding tracing capabilities to allocators and see how to integrate them with the language and Phobos. 4. Work on the core language and druntime to see how to seamlessly accommodate alternate GC mechanisms such as reference counting. Andrei

Sounds good. But why nullable instead of C# choice of "Type?" ?
Feb 02 2014
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/2/14, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 2. Work on Phobos to see what can be done about avoiding unnecessary
 allocation. Most likely we'll need to also add a  nogc flag.

See Johannes'es pull here: https://github.com/D-Programming-Language/dmd/pull/1886 Where he comments: ``` Note: This could also be a start to implement a -nogc parameter which would check a whole module not to use the GC. This is less intrusive than a nogc attribute. ```
Feb 02 2014
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Sun, 2 Feb 2014 18:36:29 +0100
schrieb Andrej Mitrovic <andrej.mitrovich gmail.com>:

 On 2/2/14, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:
 2. Work on Phobos to see what can be done about avoiding unnecessary
 allocation. Most likely we'll need to also add a  nogc flag.

See Johannes'es pull here: https://github.com/D-Programming-Language/dmd/pull/1886 Where he comments: ``` Note: This could also be a start to implement a -nogc parameter which would check a whole module not to use the GC. This is less intrusive than a nogc attribute. ```

The last iteration actually implements -nogc and the pull request also provides almost everything needed for nogc. The attribute should probably be added in a second pull request as I expect more discussions there.
Feb 02 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 02 Feb 2014 06:48:28 -0800, Paulo Pinto <pjmlp progtools.org>  
wrote:

 Am 02.02.2014 15:11, schrieb Adam D. Ruppe:
 On Sunday, 2 February 2014 at 11:23:56 UTC, JR wrote:
 I *seem* to remember reading here that you and Walter were
 increasingly growing to favor ARC, but I can't find the post.

Personally, I think ARC is a waste of time and none of the arguments I've seen for it are convincing. Improving the Gc implementation would be a much bigger win.

It might be that ARC will be a solution given the recent work in Objective-C, ParaSail, Rust and the misterious M#. I was even surprised to discover that Cedar actually used RC with a GC just for collecting cycles. What I don't like is using Objective-C as a poster child for ARC, because the only reason it exists, is that Apple engineers weren't able to write a GC for Objective-C that didn't core dump left and right. Apple of course sold the history in a different way. -- Paulo

I just want to point out that in the comments of Joe Duffy's latest post on M# he explicitly stated that M# is still C# semantics, and specifically he says that all existing C# code will compile with M#, since ARC would by very fact of it's nature require the language semantics to change, it's extremely unlikely that M# will use ARC. Especially since they have access to a GC guru. For a ridiculously easy example, what do you do when one object implements .Dispose() and another one does not? -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 02 2014
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/2/14, Johannes Pfau <nospam example.com> wrote:
 The last iteration actually implements -nogc and the pull request
 also provides almost everything needed for  nogc.

Excellent. 2014 is looking great so far. :>
Feb 02 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
+1 on making Phobos GC-free.

On Sunday, 2 February 2014 at 07:35:21 UTC, Andrei Alexandrescu 
wrote:
 On 2/1/14, 10:00 PM, Frank Bauer wrote:
 On Sunday, 2 February 2014 at 05:41:53 UTC, Andrei 
 Alexandrescu wrote:
 Sure. While you're young typecheck this:

 class A {
   auto stuff() { ...; return this; }
 }


 Andrei

It would be a Borrowed!A, I would say, no matter how an A is allocated. A GC!A, an ARC!A and an Owning!A convert equally well to a Borrowed!A. As long as the Borrowed!A is in use (i.e. scope) after stuff() returns, the A object must not be freed (compiler enforced). Would that make sense?

No. What I'm driving at is that right now the type is A, as is in a variety of other situations. It would be a whole different language that would be wholly incompatible with D. Andrei

Why is that? I propose the new memory regime to be 100% compatible with all existing D code. It is way more powerful than anything scope can do. And it seems a lot less work to me than going through GC optimization and custom allocator tracing hell. Let's see how far we can get if we require none of the existing D code to break. Example: this is current D code that runs unchanged and with the same semantics under the new memory regime. GC(T) and Borrowed(T) remain under the hood: class A { A foo() { return this; } // returns Borrowed!A } void main () { A m1 = new A; // m1 is a GC!A A m2 = m1; // m2 is a Borrowed!A A m3; // Borrowed!A m3 = m1; // fine: assignment from GC!A to Borrowed!A m3 = bar(m1); // fine: m1 converted from GC!A to Borrowed!A, assignment from Borrowed!A to Borrowed!A m3 = bar(m2); // fine: m2 already Borrowed!A, assignment from Borrowed!A to Borrowed!A m1 = new A; // fine: can reasssign GC!A although there are outstanding references m1 = m2; // fine: can reassign GC!A to a Borrowed!A } A bar(A a) { // takes and returns Borrowed!A A b1 = a; return b1; } Now here it comes: if, on the other hand, I replace the first 'new' with, say e.g., 'newOwning' (or '~', whatever), I get GC-free code: A m1 = newOwning A; // m1 is an Owning!A, allocating on the heap and freeing when it goes out of scope Then these lines above would be in error: m1 = new A; // error: can't reassign to m1 because m2 and m3 still referencing m1 (same for newOwning) m1 = m2; // error: can't assign Borrowed!A to Owning!A ARC(T) could be implemented similarly. Nothing changes one bit for those who want to use the GC as before. ARC(T) and Owning(T) would be *ADDED* on top of the existing language spec. Emplacement new, as Adam mentioned, is not a problem: you always get to the emplaced object with a Borrowed!T. The underlying memory is managed separately and automatically as a GC!M, ARC!M or Owning!M (M could be an array of bytes, e.g.).
Feb 02 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e015365e6f3a75804f177b2f9
Content-Type: text/plain; charset=UTF-8

On 2 February 2014 06:07, Adam Wilson <flyboynw gmail.com> wrote:

 On Sat, 01 Feb 2014 02:15:42 -0800, Manu <turkeyman gmail.com> wrote:

 Why is ARC any worse than GC? Why is it even a compromise at the high
 level?
 Major players have been moving away from GC to ARC in recent years. It's
 still a perfectly valid method of garbage collection, and it has the
 advantage that it's intrinsically real-time compatible.

doing ARC for almost a decade, and IIRC, like GC's, it's not universally loved there either. Microsoft is desperately trying to get people to move back to C++ but so far the community has spoken with a resounding "You can pry C#/.NET from our cold, dead, hands." Java has shown no particular interest in moving away from GC's probably because their GC is best in class. Even games are starting to bring in GC's (The Witcher 2 for example, and almost all of the mobile/casual games market, which is actually monetarily bigger than the triple-A games market.)

You didn't say why ARC is worse than GC? I have my suspicious that the witcher guys would have been perfectly (perhaps MORE happy) with ARC, if it were possible. It's possible to write a GC in C++, but it's impossible to use ARC without compiler support, which C++ does not and never will have, and as a result, it wasn't an option to them. You can't say they picked one over the other when one simply wasn't available. Their use of a GC shows they appreciate the convenience of garbage collection, but it doesn't say anything about GC vs ARC. I don't think realtime software is becoming an edge case by any means,
 maybe 'extreme' realtime is, but that doesn't change the fact that the GC
 still causes problems for all realtime software.

as a percentage of all software written, which, by definition, makes it an edge-case. Even vehicle control software is no longer completely real-time. [I just happen to know that because that's the industry I work in. Certain aspects are, with the rest scheduled out.] And more to the point, D has made no claim about it's suitability for RT software and I have seen little demand for it outside a very small very vocal minority that is convinced that it has the dynamic resource management panacea if everyone would just do as they say.

So answer, why is a GC so much preferable to ARC? GC forces you into one (poorly implemented) paradigm, which is fundamentally incompatible with many tasks. ARC might force you to think about weak references... but is that really as hard as you say? It's possible an ARC solution might optionally have a GC in the background to collect cycles (for those not interested in dealing with the cognitive load of circular reference as you are worried). This is fine, as long as the aggressive users can turn that off, and deal with their circular reference issues directly. Under an approach like this there are far more options in terms of how different users can approach the garbage collection in their apps, and ALL USERS would still have access to phobos. I don't see them as mutually exclusive. They are quite complimentary. I personally believe latency and stuttering is one of the biggest usability
 hindrances in modern computing, and it will become a specific design focus
 in software of the future. People are becoming less and less tolerant of
 latency in all forms; just consider the success of iPhone compared to
 competition, almost entirely attributable to the silky smooth UI
 experience. It may also be a telling move that Apple switched to ARC
 around
 the same time, but I don't know the details.

some C# code. I can happily report that you are *ahem* wrong. Even Visual Studio 2013 doesn't stutter often, and only when I am pulling in some massive third-party module that may or may not be well written. Ironically, it is VisualD, it of D code fame that most slows down VS for me. I write software in C# every day I can happily report that I have yet to have a problem with stuttering in my code that wasn't of my own devise. (If you forget to asynchronously call that web-service it WILL stutter) And that's a point. I can write software in C# that works will without having to worry about circular references or if my data prematurely falls out of scope or any other of the details that are associated with ARC. And for my not-effort, I pay an effective cost of 0. Win-win. You're demanding that to suit your needs, we make a massive philosophical and language change to D that will incur HIGHER cognitive load on programmers for something that will not measurably improve the general use case? Ahem, that's good for D how?

Can you tell me your experience with C# on embedded hardware with bugger-all memory? I can; it doesn't work (although to be fair, the GC is not the only issue there). It's also like you seem to think D's GC experience is comparable to C#. It's not, D has a terrible GC, and there seems to be almost no motion to fix or improve it. Can it EVER be anywhere near as good as C# or Java? Where are the proposals to get it there? And even then, what is the resource overhead? How much ambient memory overhead/loss will I suffer as a consequence? What if my platform is only single-core? I also firmly believe that if D - a native compiled language familiar to
 virtually all low-level programmers - doesn't have any ambition to service
 the embedded space in the future, what will? And why not?
 The GC is the only thing inhibiting D from being a successful match in
 that
 context. ARC is far more appropriate, and will see it enter a lot more
 fields.
 What's the loss?

per line of code. Ease-of-use. A GC is easier to use in practice. You can say well they should learn to use ARC because it's better (for certain definitions of better) but they won't. They'll just move on. I'd say that's a pretty big loss. And based on my conversations with Walter, I don't think that D was ever intended to make a play for the embedded space. If D can be made to work there great, but Walter, as near as I can tell, has no interest in tying the language in knots to make it happen. So that's a non-issue. And let's be honest, the requirements of that space are fairly extreme.

But D ticks all the boxes, except that one... and it's an important field that isn't covered by the rest of the landscape of emerging or trendy languages. I think it's also telling that newcomers constantly raise it as a massive
 concern, or even a deal-breaker. Would they feel the same about ARC? I
 seriously doubt it. I wonder if a poll is in order...
 Conversely, would any of the new-comers who are pro-GC feel any less happy
 if it were ARC instead? $100 says they probably wouldn't even know, and
 almost certainly wouldn't care.

I DON'T see a majority of newcomers raising an issue with the GC, I only see it from newcomers with some pretty extreme latency requirements, primarily for the real-time crowd. The majority of newcomers aren't interested in RT work. I think you're falling prey to confirmation bias here.

I never said anything about a majority, re-read my statement. But it does come up all the time. Are you surprised that the RT crowd are attracted to D? It's precisely what they've been waiting for all these years. There's just this one single stumbling block that seems to make every single one of them very nervous. I attended a small graphics/systems programming event yesterday, there were about 40-50 people there. I hadn't caught up with the organisers for years, they asked me what I had been up to, I mentioned I had been working a lot with D in my spare time. They were interested, conversations went a lot like this: [among other comments] "Oh yeah, I heard about that... we'd love you to come and give a talk about it some time." (more people overheard and approached the conversation) "I was really interested in D, but then I realised that the GC isn't really optional like the website claims, and stopped" (more people approach) "I tried it once, but I don't think it's practical for the same reason I had to give up C#" (small crowd, ~10 interested in hearing about it) "I love C#, if I could use it for work, I would!" (note: for my money, D actually offers to my kind what C# teases you with, but hangs just out of reach) "so, can you actually use D without the GC?" me: "Erm. Well..." Etc. There was really surprising interest, and I ended out drinking beers with 4 guys who were really interested to hear more about it. Many of them HAD heard about D, but immediately wrote it off, like C#. There is a massive potential audience here, I've been saying for a long time, D is the language that game devs and other realtime programmers want, but they're very conservative, and this particular issue is an absolute red flag. --089e015365e6f3a75804f177b2f9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 2= February 2014 06:07, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> On Sat, 01 Feb 2014 02:15:42 -0800, Manu &lt;<a href=3D"mailto:turkeyman gm= ail.com" target=3D"_blank">turkeyman gmail.com</a>&gt; wrote:<br><blockquot= e class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc sol= id;padding-left:1ex"> <div><div class=3D"h5"><br></div></div><div class=3D"im"> Why is ARC any worse than GC? Why is it even a compromise at the high level= ?<br> Major players have been moving away from GC to ARC in recent years. It&#39;= s<br> still a perfectly valid method of garbage collection, and it has the<br> advantage that it&#39;s intrinsically real-time compatible.<br> <br> </div></blockquote> <br> Define Major Players? Because I only know about Apple, but they&#39;ve been= doing ARC for almost a decade, and IIRC, like GC&#39;s, it&#39;s not unive= rsally loved there either. Microsoft is desperately trying to get people to= move back to C++ but so far the community has spoken with a resounding &qu= ot;You can pry C#/.NET from our cold, dead, hands.&quot; Java has shown no = particular interest in moving away from GC&#39;s probably because their GC = is best in class. Even games are starting to bring in GC&#39;s (The Witcher= 2 for example, and almost all of the mobile/casual games market, which is = actually monetarily bigger than the triple-A games market.)</blockquote> <div><br></div><div>You didn&#39;t say why ARC is worse than GC?</div><div>= <br></div><div>I have my suspicious that the witcher guys would have been p= erfectly (perhaps MORE happy) with ARC, if it were possible.</div><div> It&#39;s possible to write a GC in C++, but it&#39;s impossible to use ARC = without compiler support, which C++ does not and never will have, and as a = result, it wasn&#39;t an option to them.</div><div>You can&#39;t say they p= icked one over the other when one simply wasn&#39;t available. Their use of= a GC shows they appreciate the convenience of garbage collection, but it d= oesn&#39;t say anything about GC vs ARC.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"><div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I don&#39;t think realtime software is becoming an edge case by any means,<= br> maybe &#39;extreme&#39; realtime is, but that doesn&#39;t change the fact t= hat the GC<br> still causes problems for all realtime software.<br> <br> </blockquote> <br></div> Yes, but my point is that there is very little real-time software written a= s a percentage of all software written, which, by definition, makes it an e= dge-case. Even vehicle control software is no longer completely real-time. = [I just happen to know that because that&#39;s the industry I work in. Cert= ain aspects are, with the rest scheduled out.] And more to the point, D has= made no claim about it&#39;s suitability for RT software and I have seen l= ittle demand for it outside a very small very vocal minority that is convin= ced that it has the dynamic resource management panacea if everyone would j= ust do as they say.</blockquote> <div><br></div><div>So answer, why is a GC so much preferable to ARC?</div>= <div>GC forces you into one (poorly implemented) paradigm, which is fundame= ntally incompatible with many tasks.</div><div>ARC might force you to think= about weak references... but is that really as hard as you say?</div> <div>It&#39;s possible an ARC solution might optionally have a GC in the ba= ckground to collect cycles (for those not interested in dealing with the co= gnitive load of circular reference as you are worried). This is fine, as lo= ng as the aggressive users can turn that off, and deal with their circular = reference issues directly. Under an approach like this there are far more o= ptions in terms of how different users can approach the garbage collection = in their apps, and ALL USERS would still have access to phobos.</div> <div>I don&#39;t see them as mutually exclusive. They are quite complimenta= ry.</div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I personally believe latency and stuttering is one of the biggest usability= <br> hindrances in modern computing, and it will become a specific design focus<= br> in software of the future. People are becoming less and less tolerant of<br=

competition, almost entirely attributable to the silky smooth UI<br> experience. It may also be a telling move that Apple switched to ARC around= <br> the same time, but I don&#39;t know the details.<br> <br> </blockquote> <br></div> I use .NET every day, seriously not one day goes by when I haven&#39;t touc= hed some C# code. I can happily report that you are *ahem* wrong. Even Visu= al Studio 2013 doesn&#39;t stutter often, and only when I am pulling in som= e massive third-party module that may or may not be well written. Ironicall= y, it is VisualD, it of D code fame that most slows down VS for me. I write= software in C# every day I can happily report that I have yet to have a pr= oblem with stuttering in my code that wasn&#39;t of my own devise. (If you = forget to asynchronously call that web-service it WILL stutter)<br> <br> And that&#39;s a point. I can write software in C# that works will without = having to worry about circular references or if my data prematurely falls o= ut of scope or any other of the details that are associated with ARC. And f= or my not-effort, I pay an effective cost of 0. Win-win. You&#39;re demandi= ng that to suit your needs, we make a massive philosophical and language ch= ange to D that will incur HIGHER cognitive load on programmers for somethin= g that will not measurably improve the general use case? Ahem, that&#39;s g= ood for D how?<br> </blockquote><div><br></div><div>Can you tell me your experience with C# on= embedded hardware with bugger-all memory? I can; it doesn&#39;t work (alth= ough to be fair, the GC is not the only issue there).</div><div>It&#39;s al= so like you seem to think D&#39;s GC experience is comparable to C#. It&#39= ;s not, D has a terrible GC, and there seems to be almost no motion to fix = or improve it.</div> <div><br></div><div>Can it EVER be anywhere near as good as C# or Java? Whe= re are the proposals to get it there?</div><div>And even then, what is the = resource overhead? How much ambient memory overhead/loss will I suffer as a= consequence? What if my platform is only single-core?</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im"> I also firmly believe that if D - a native compiled language familiar to<br=

ice<br></div><div class=3D"im"> the embedded space in the future, what will? And why not?<br> The GC is the only thing inhibiting D from being a successful match in that= <br> context. ARC is far more appropriate, and will see it enter a lot more<br> fields.<br> What&#39;s the loss?<br> <br> </div></blockquote> <br> Cognitive load. How many details does the programmer have to worry about pe= r line of code. Ease-of-use. A GC is easier to use in practice. You can say= well they should learn to use ARC because it&#39;s better (for certain def= initions of better) but they won&#39;t. They&#39;ll just move on. I&#39;d s= ay that&#39;s a pretty big loss.<br> <br> And based on my conversations with Walter, I don&#39;t think that D was eve= r intended to make a play for the embedded space. If D can be made to work = there great, but Walter, as near as I can tell, has no interest in tying th= e language in knots to make it happen. So that&#39;s a non-issue. And let&#= 39;s be honest, the requirements of that space are fairly extreme.</blockqu= ote> <div><br></div><div>But D ticks all the boxes, except that one... and it&#3= 9;s an important field that isn&#39;t covered by the rest of the landscape = of emerging or trendy languages.</div><div><br></div><blockquote class=3D"g= mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l= eft:1ex"> <div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I think it&#39;s also telling that newcomers constantly raise it as a massi= ve<br> concern, or even a deal-breaker. Would they feel the same about ARC? I<br> seriously doubt it. I wonder if a poll is in order...<br> Conversely, would any of the new-comers who are pro-GC feel any less happy<= br> if it were ARC instead? $100 says they probably wouldn&#39;t even know, and= <br> almost certainly wouldn&#39;t care.<br> </blockquote> <br></div> I DON&#39;T see a majority of newcomers raising an issue with the GC, I onl= y see it from newcomers with some pretty extreme latency requirements, prim= arily for the real-time crowd. The majority of newcomers aren&#39;t interes= ted in RT work. I think you&#39;re falling prey to confirmation bias here.<= /blockquote> <div><br></div><div>I never said anything about a majority, re-read my stat= ement. But it does come up all the time.</div><div>Are you surprised that t= he RT crowd are attracted to D? It&#39;s precisely what they&#39;ve been wa= iting for all these years.</div> <div>There&#39;s just this one single stumbling block that seems to make ev= ery single one of them very nervous.</div><div><br></div><div>I attended a = small graphics/systems programming event yesterday, there were about 40-50 = people there. I hadn&#39;t caught up with the organisers for years, they as= ked me what I had been up to, I mentioned I had been working a lot with D i= n my spare time. They were interested, conversations went a lot like this:<= /div> <div><br></div><div>[among other comments]</div><div>&quot;Oh yeah, I heard= about that... we&#39;d love you to come and give a talk about it some time= .&quot;</div><div>(more people overheard and approached the conversation)</= div> <div>&quot;I was really interested in D, but then I realised that the GC is= n&#39;t really optional like the website claims, and stopped&quot;</div><di= v>(more people approach)</div><div>&quot;I tried it once, but I don&#39;t t= hink it&#39;s practical for the same reason I had to give up C#&quot;</div> <div>(small crowd, ~10 interested in hearing about it)</div><div>&quot;I lo= ve C#, if I could use it for work, I would!&quot; (note: for my money, D ac= tually offers to my kind what C# teases you with, but hangs just out of rea= ch)</div> <div>&quot;so, can you actually use D without the GC?&quot;</div><div>me: &= quot;Erm. Well...&quot;</div><div><br></div><div>Etc.</div><div><br></div><= div>There was really surprising interest, and I ended out drinking beers wi= th 4 guys who were really interested to hear more about it.</div> <div>Many of them HAD heard about D, but immediately wrote it off, like C#.= </div><div>There is a massive potential audience here, I&#39;ve been saying= for a long time, D is the language that game devs and other realtime progr= ammers want, but they&#39;re very conservative, and this particular issue i= s an absolute red flag.</div> </div></div></div> --089e015365e6f3a75804f177b2f9--
Feb 02 2014
prev sibling next sibling parent "Frank Bauer" <x y.com> writes:
On Monday, 3 February 2014 at 02:58:59 UTC, Manu wrote:
 But D ticks all the boxes, except that one... and it's an 
 important field
 that isn't covered by the rest of the landscape of emerging or 
 trendy
 languages.

  I think it's also telling that newcomers constantly raise it 
 as a massive
 concern, or even a deal-breaker. Would they feel the same 
 about ARC? I
 seriously doubt it. I wonder if a poll is in order...



Agree with Manu 100% there and the rest of his post as well. I imagine standing in that little crowd he talked to. Anyone asking for the addition of ARC or owning pointers to D, gets pretty much ignored. The topic is "Smart pointers instead of GC?", remember? People here seem to be more interested in diverting to nullable, scope and GC optimization. Telling, indeed. And, yes, as I posted, I believe one could keep D's syntax unchanged, including GC allocation via new, by changing the memory regime under the hood and providing for ARCs and owning pointers on top of GC. It would be a lot of work taking into account storage classes, type construction, pointer interaction and what not), I understand. But it would be better spent than on more of the same garbage ... err ... collection. So it's a done deal, then? D didn't go all the way to become a systems or highest performance language? Instead, it wants to grab a piece of that Java / C# pie? Good luck with that. I don't think those guys are as open to change as the C++ guys, also given the higher complexity (i.e. richness) of D compared to C# / Java. IMHO, the D implementers should really reconsider what language features they would like to concentrate on if they want to be around in the years to come.
Feb 03 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Monday, 3 February 2014 at 14:57:35 UTC, Frank Bauer wrote:
 On Monday, 3 February 2014 at 02:58:59 UTC, Manu wrote:
 But D ticks all the boxes, except that one... and it's an 
 important field
 that isn't covered by the rest of the landscape of emerging or 
 trendy
 languages.

 I think it's also telling that newcomers constantly raise it 
 as a massive
 concern, or even a deal-breaker. Would they feel the same 
 about ARC? I
 seriously doubt it. I wonder if a poll is in order...



Agree with Manu 100% there and the rest of his post as well. I imagine standing in that little crowd he talked to. Anyone asking for the addition of ARC or owning pointers to D, gets pretty much ignored. The topic is "Smart pointers instead of GC?", remember? People here seem to be more interested in diverting to nullable, scope and GC optimization. Telling, indeed. And, yes, as I posted, I believe one could keep D's syntax unchanged, including GC allocation via new, by changing the memory regime under the hood and providing for ARCs and owning pointers on top of GC. It would be a lot of work taking into account storage classes, type construction, pointer interaction and what not), I understand. But it would be better spent than on more of the same garbage ... err ... collection. So it's a done deal, then? D didn't go all the way to become a systems or highest performance language? Instead, it wants to grab a piece of that Java / C# pie? Good luck with that. I don't think those guys are as open to change as the C++ guys, also given the higher complexity (i.e. richness) of D compared to C# / Java. IMHO, the D implementers should really reconsider what language features they would like to concentrate on if they want to be around in the years to come.

It's pure laziness. That is the hole point of AGC, right? So you don't have to pay attention to what you are doing. Just wait. Google is working on it! They are working on cars that drive themselves so you don't have to. There is no way to override it. Google decides where you need to go and how fast. You have no say so in the matter and if you want to take the scenic route then you'll just have to get out and walk(from a moving car BTW). If it so happens there is construction work and you gotta take a detour, well, tell that to the judge or your insurance company... it's not googles fault for not making the car smart enough or giving you alternatives(remember, you are too stupid to know anything).
Feb 03 2014
prev sibling next sibling parent "Frank Bauer" <x y.com> writes:
On Monday, 3 February 2014 at 15:09:08 UTC, Frustrated wrote:
 It's pure laziness. That is the hole point of AGC, right? So you
 don't have to pay attention to what you are doing.

Well, aren't we all programmers because we are lazy? Othherwise we would be moving physical stuff around ;) Certainly, Walter, Andrei and the rest were anything but lazy. Quite the opposite. But considering what they accomplished already, maybe they decide to go that extra mile in the coming years. I actually enjoy the comfort of a GC cleaning up after me, if I don't have critical stuff to do. Other than that, you are right: if you know what you do in your code and given powerful tools, you can accomplish more (performance).
Feb 03 2014
prev sibling next sibling parent "Joseph Cassman" <jc7919 outlook.com> writes:
On Sunday, 2 February 2014 at 16:55:35 UTC, Andrei Alexandrescu 
wrote:
 I think of the following foci for the first half of 2014:

 1. Add  nullable and provide a -nullable compiler flag to 
 verify it. The attribute is inferred locally and for white-box 
 functions (lambdas, templates), and required as annotation 
 otherwise. References not annotated with  nullable are 
 statically enforced to never be null.

 2. Work on Phobos to see what can be done about avoiding 
 unnecessary allocation. Most likely we'll need to also add a 
  nogc flag.

 3. Work on adding tracing capabilities to allocators and see 
 how to integrate them with the language and Phobos.

 4. Work on the core language and druntime to see how to 
 seamlessly accommodate alternate GC mechanisms such as 
 reference counting.


 Andrei

This is an excellent direction to take the language in the near-term. It seems that some who would like to use D are afraid of the GC. Others have legitimate use-cases where the effort of manually managing memory is warranted, such as a memory-constrained environment. Being able to accommodate both, provide an integrated approach to allocation, and allow for ref-counting as a viable alternative to GC is a home run. Also will be nice to see Phobos memory usage trimmed down. I feel that these items mentioned above combined will put D into a niche that other languages will find it difficult to compete with: end-to-end fine-grained memory control. Nice! Just my 10 cents. Joseph
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

Andrei, I am sorry to report that anything other than complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 12:21:14 -0800, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu  
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

Andrei, I am sorry to report that anything other than complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

That said, I do think that it should be possible to implement the two side-by-side, although it might require some new keywords. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 3 February 2014 at 20:02:30 UTC, Andrei Alexandrescu 
wrote:
 I thought I made it clear that GC avoidance (which includes 
 considering built-in reference counting) is a major focus of 
 2014.

Please make it so that you can easily write thread-level allocator agnostic code without passing the allocator around. E.g. so that you can start writing code with GC, but properly release memory, then when you are certain that your model does not introduce cycles replace it with a cheaper allocation model with no additional code changes.
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 12:40:20 -0800, Dmitry Olshansky  =

<dmitry.olsh gmail.com> wrote:

 04-Feb-2014 00:21, Adam Wilson =D0=BF=D0=B8=D1=88=D0=B5=D1=82:
 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets=




 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei


 Sadly, although written as hyperbole, I feel that the above is fairly=


 close to the actual position of the ARC crowd.

I won't be surprised that half of current GC problems are because it's=

 simply too stupid as an allocator. I had some numbers that I'd need to=

 dig up were GC.malloc was ~2x slower then malloc even with garbage  =

 collection disabled.

 With that said I'd focus on ref-counting somehow coexisting with traci=

 GC. It doesn't go well at the moment - try creating dynamic arrays of =

 std.stdio.File (that is ref-counted).

I will not defend the current GC, it is about as stupid as you can make = = one and still have a functioning memory management system. It implements= = exactly none of the optimizations recommended for Mark-Sweep GC's in the= = GC Handbook. That said, I firmly believe that wholesale replacement of the GC is = throwing the baby out with the bathwater. Effectively, the poor D GC = implementation has become an excuse to launch a crusade against all GC's= = everywhere, never mind that Java and the .NET GC's are consistent exampl= es = of just how good GC's can actually be. -- = Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Adam Wilson:

 I will not defend the current GC, it is about as stupid as you 
 can make one and still have a functioning memory management 
 system. It implements exactly none of the optimizations 
 recommended for Mark-Sweep GC's in the GC Handbook.

Why don't you open an enhancement request for each of those recommended optimizations (explaining as well as possible what to do and how and what to expect from them)? I have opened several "obvious" enhancement requests, and I have seen them fixed in some time. Bye, bearophile
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 12:51:20 -0800, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 12:21 PM, Adam Wilson wrote:
 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

Andrei, I am sorry to report that anything other than complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users.

Why would you be sorry? "It is what it is", "customer is always right" etc. If we deem that important then we should look into supporting that scenario.

There is a big difference between supporting a scenario, and forcing it down everyone's throats as The One True Path. I would actually get out an pull for supporting both, equally. Let me choose, I know more about the memory usage patterns of my cloud-based service than a game developer does. In some cases ARC is more efficient, in other cases, the extra effort required is more than I care to pay my programmers for. In fact, for all of our projects, the efficiency cost of the GC is microscopic compared to having to pay the developer/time cost of ARC or Manual management. There is a reason we are an exclusively C# shop right now. And Apple/Obj-C has consistently proved that all the hype about ARC being more efficient for free is untrue. There are plenty of complaints in that camp about having to hunt down heisenleaks based on ARC leaking because somebody forgot to mark something as weak.
 No arguments
 can be made to otherwise, regardless of validity. As far as they are
 concerned the discussion of ARC vs. GC is closed and decided. ARC is the
 only path forward to the bright and glorious future of D. ARC most
 efficiently solves all memory management problems ever encountered.
 Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL  
 ARC!

 Sadly, although written as hyperbole, I feel that the above is fairly
 close to the actual position of the ARC crowd.

I'm going on a limb here, but it seems you're not convinced :o). Andrei

Hehe, you might say that. I did buy the GC Handbook to learn more about GC's after all, not replace them with something that has demonstrable drawbacks. :-) -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Monday, 3 February 2014 at 20:21:14 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to 
 D, gets
 pretty much ignored. The topic is "Smart pointers instead of 
 GC?",
 remember? People here seem to be more interested in diverting 
 to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

Andrei, I am sorry to report that anything other than complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

No!! That seems quite arrogant! No one is trying to take your toys away from you! We just ask that you allow us to play with ours as we have done the same for you. It should be a two way street. Just because you drive down it doesn't mean you own it. Of course no one is demanding it be done, Anyone can choose to do what they want. But many love D and what it does but are forced to take the good with the bad... and there is no great reason for that. D has the potential to be one of those languages that have it all. It can't when you keep it from growing. Removing the AGC dependence that D has will be like giving water to a seed... and we all want D to be successful, don't you? After all, what's the harm in us both being able to play with our toys? Or are you just the jealous time that doesn't like to see anyone else happy?
Feb 03 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Andrei Alexandrescu wrote:

 2. Work on Phobos to see what can be done about avoiding 
 unnecessary allocation. Most likely we'll need to also add a 
  nogc flag.

 ...

 4. Work on the core language and druntime to see how to 
 seamlessly accommodate alternate GC mechanisms such as 
 reference counting.

 ...

 I thought I made it clear that GC avoidance (which includes 
 considering built-in reference counting) is a major focus of 
 2014.

 Andrei

My apologies then. Not apologizing for stressing this point over and over again, though :) Just to steal some people's thunder: I want the GC to remain in D as it is. This is not hyperbole or cynicism. I mean it. Having the GC around to clean up after me for noncritical apps is a blessing. Adding ARC and owning pointers would make my (and others) day though. BTW: Why is most everybody who doesn't like GC obsessed with ARC pointers? I like them for where they are appropriate. But what about a simpler owning pointer that frees automatically when it goes out of scope. It comes at exactly zero runtime cost, just as a "dumb" pointer. You don't have to drag a reference count around with you. It is just a pointer in memory. All its semantics are enforced at compile time. Throw borrowed pointers in the mix, i.e. references to owning pointers that become "frozen" while there are outstanding references to them, and you're all set. I think this would really be necessary (and sufficient!) to get Rust off your back and whatever half baked language stuff MS is intentionally leaking these days. At least IMHO and others from the *vocal minority*. I'm not quite sure that I understand what you mean by GC avoidance being a major focus of 2014 though. In the long term, can I look forward to writing an absolutely, like in 100 %, like in guaranteed, GC free D app with all of current D's and Phobos' features if I choose to? Or does it mean: well for the most part it will avoid the GC, but if you're unlucky the GC might still kick in if you don't pay attention and when you least expect it?
Feb 03 2014
prev sibling next sibling parent "woh" <wojw yahoo.com> writes:
  Any system that forces a single way of handling memory as the 
only viable method, be it GC( as D currently does), or ARC, is 
undesirable to me

   Rust seems to have found a very nice model, and even cpp with 
value/unique/rc/weak is IMO superior to what D currently offers

On Monday, 3 February 2014 at 22:51:50 UTC, Frank Bauer wrote:
 Andrei Alexandrescu wrote:

 2. Work on Phobos to see what can be done about avoiding 
 unnecessary allocation. Most likely we'll need to also add a 
  nogc flag.

 ...

 4. Work on the core language and druntime to see how to 
 seamlessly accommodate alternate GC mechanisms such as 
 reference counting.

 ...

 I thought I made it clear that GC avoidance (which includes 
 considering built-in reference counting) is a major focus of 
 2014.

 Andrei

My apologies then. Not apologizing for stressing this point over and over again, though :) Just to steal some people's thunder: I want the GC to remain in D as it is. This is not hyperbole or cynicism. I mean it. Having the GC around to clean up after me for noncritical apps is a blessing. Adding ARC and owning pointers would make my (and others) day though. BTW: Why is most everybody who doesn't like GC obsessed with ARC pointers? I like them for where they are appropriate. But what about a simpler owning pointer that frees automatically when it goes out of scope. It comes at exactly zero runtime cost, just as a "dumb" pointer. You don't have to drag a reference count around with you. It is just a pointer in memory. All its semantics are enforced at compile time. Throw borrowed pointers in the mix, i.e. references to owning pointers that become "frozen" while there are outstanding references to them, and you're all set. I think this would really be necessary (and sufficient!) to get Rust off your back and whatever half baked language stuff MS is intentionally leaking these days. At least IMHO and others from the *vocal minority*. I'm not quite sure that I understand what you mean by GC avoidance being a major focus of 2014 though. In the long term, can I look forward to writing an absolutely, like in 100 %, like in guaranteed, GC free D app with all of current D's and Phobos' features if I choose to? Or does it mean: well for the most part it will avoid the GC, but if you're unlucky the GC might still kick in if you don't pay attention and when you least expect it?

Feb 03 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
An owning pointer is the perfect garbage collector: it gets rid 
of the garbage exactly at the moment it goes out of scope.
Feb 03 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor
wrote:
 You can always force the GC to run between cycles in your game, 
 and
 turn off automatic sweeps.  This is how most games operate 
 nowadays.
 It's also probably possible to create a drop-in replacement for 
 the GC
 to do something else.   I could see if being *VERY* useful to 
 make the
 GC take a compile-time parameter to select which GC engine is 
 used.

This is just non-sense. Maybe this is why modern games suck then? How do you guarantee that the GC won't kick in at the most inopportune times? Oh, you manually trigger it? When? Right at the moment when the player is about to down the boss after a 45 min fight? Oh, right.. you just segfault cause there is no memory left. On Monday, 3 February 2014 at 22:51:50 UTC, Frank Bauer wrote:
 I'm not quite sure that I understand what you mean by GC 
 avoidance being a major focus of 2014 though. In the long term, 
 can I look forward to writing an absolutely, like in 100 %, 
 like in guaranteed, GC free D app with all of current D's and 
 Phobos' features if I choose to? Or does it mean: well for the 
 most part it will avoid the GC, but if you're unlucky the GC 
 might still kick in if you don't pay attention and when you 
 least expect it?

It's either got to be 100% or nothing. The only issue of the GC is the non-determinism.... or if you do corner it and trigger it manually you end up with exactly the types of problems Mr. Chancellor thinks doesn't exist... i.e., the longer you have to put off the GC the worse it becomes(the more time it takes to run or the less memory you have to work with). It might work ok with some concurrent AGC that you can use for non-critical parts. e.g., have phobo's use the GC for non-real time sections of the app(boot up, menu's for games, etc...) then disable it and use ARC for when the app is meant for optimal and/or deterministic performance. One could argue that if one goes this around why not use ARC or manual memory mangement(MMM?) the whole time... but by using the GC during the non-critical parts of the program one can focus less on memory leaks as usual with the GC. What would be nice is to be able to write code that is oblivious to how it's memory is managed, but be able to switch between different methods. [Sartup->menu's and other non-performance section of the game] <- use GC [real-time areas of game] <- use ARC or manual
Feb 03 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Monday, 3 February 2014 at 23:13:07 UTC, woh wrote:

   Rust seems to have found a very nice model ...

How cool would that be: a language with the memory allocation features of Rust but with stronger syntax and semantics roots in the C / C++ world?
Feb 03 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 31 December 2012 at 12:14:22 UTC, Sven Over wrote:
 In my job I'm writing backend services that power a big web 
 site. Perfomance is key, as the response time of the data 
 service in most cases directly adds to the page load time. The 
 bare possibility that the whole service pauses for, say, 100ms 
 is making me feel very uncomfortable.

Back to the original scenario: Current CPUs have a measured peak throughput of 2-4GB per 100ms. So if you keep pointers clustered on cache lines in a way that is compatible with the AGCs prefetching patterns, then you ought to make it for most applications if the collector is optimized? If your backend service (data) is replicated on multiple servers you could, of course, let the servers run the GC at different times organized with a load-balancer/router/task scheduler. So if GC makes you more productive on a web server then I don't really see why you don't use it. Now, that might not work out in the future though. If CPU caches keep growing to the extent that most of your working dataset sits in the cache and you have lots of rather slow main memory then the AGC process will become sloooow no matter how good the GC is.
Feb 03 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 00:08:28 UTC, Frank Bauer wrote:
 How cool would that be: a language with the memory allocation 
 features of Rust but with stronger syntax and semantics roots 
 in the C / C++ world?

It is really tempting to write a new parser for Rust, isn't it? :-)
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 15:26:22 -0800, Frustrated <c1514843 drdrb.com> wrote:

 On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor
 wrote:
 You can always force the GC to run between cycles in your game, and
 turn off automatic sweeps.  This is how most games operate nowadays.
 It's also probably possible to create a drop-in replacement for the GC
 to do something else.   I could see if being *VERY* useful to make the
 GC take a compile-time parameter to select which GC engine is used.

This is just non-sense. Maybe this is why modern games suck then? How do you guarantee that the GC won't kick in at the most inopportune times? Oh, you manually trigger it? When? Right at the moment when the player is about to down the boss after a 45 min fight? Oh, right.. you just segfault cause there is no memory left. On Monday, 3 February 2014 at 22:51:50 UTC, Frank Bauer wrote:
 I'm not quite sure that I understand what you mean by GC avoidance  
 being a major focus of 2014 though. In the long term, can I look  
 forward to writing an absolutely, like in 100 %, like in guaranteed, GC  
 free D app with all of current D's and Phobos' features if I choose to?  
 Or does it mean: well for the most part it will avoid the GC, but if  
 you're unlucky the GC might still kick in if you don't pay attention  
 and when you least expect it?

It's either got to be 100% or nothing. The only issue of the GC is the non-determinism.... or if you do corner it and trigger it manually you end up with exactly the types of problems Mr. Chancellor thinks doesn't exist... i.e., the longer you have to put off the GC the worse it becomes(the more time it takes to run or the less memory you have to work with).

Why is this myth of non-determinism still alive? The only truly non-deterministic GC's are concurrent collectors, but alas concurrent collects don't routinely stop-the-world either, so there really aren't any pauses to complain about. D's Mark-Sweep GC is *perfectly* deterministic. It can *only* pause on allocation. Ergo you can determine exactly which allocation caused the problem. You might not expect the function you called to GC-allocate, but that doesn't make it non-deterministic, just not what you expected. Please, stop blaming your missed expectations on the GC. This non-determinism thing is a red herring that is repeated over and over by people who obviously have no idea what they are talking about.
 It might work ok with some concurrent AGC that you can use for
 non-critical parts. e.g., have phobo's use the GC for non-real
 time sections of the app(boot up, menu's for games, etc...) then
 disable it and use ARC for when the app is meant for optimal
 and/or deterministic performance.

 One could argue that if one goes this around why not use ARC or
 manual memory mangement(MMM?) the whole time... but by using the
 GC during the non-critical parts of the program one can focus
 less on memory leaks as usual with the GC.

 What would be nice is to be able to write code that is oblivious
 to how it's memory is managed, but be able to switch between
 different methods.

 [Sartup->menu's and other non-performance section of the game] <-
 use GC
 [real-time areas of game] <- use ARC or manual

-- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Tuesday, 4 February 2014 at 00:19:53 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 15:26:22 -0800, Frustrated 
 <c1514843 drdrb.com> wrote:

 On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor
 wrote:
 You can always force the GC to run between cycles in your 
 game, and
 turn off automatic sweeps.  This is how most games operate 
 nowadays.
 It's also probably possible to create a drop-in replacement 
 for the GC
 to do something else.   I could see if being *VERY* useful to 
 make the
 GC take a compile-time parameter to select which GC engine is 
 used.

This is just non-sense. Maybe this is why modern games suck then? How do you guarantee that the GC won't kick in at the most inopportune times? Oh, you manually trigger it? When? Right at the moment when the player is about to down the boss after a 45 min fight? Oh, right.. you just segfault cause there is no memory left. On Monday, 3 February 2014 at 22:51:50 UTC, Frank Bauer wrote:
 I'm not quite sure that I understand what you mean by GC 
 avoidance being a major focus of 2014 though. In the long 
 term, can I look forward to writing an absolutely, like in 
 100 %, like in guaranteed, GC free D app with all of current 
 D's and Phobos' features if I choose to? Or does it mean: 
 well for the most part it will avoid the GC, but if you're 
 unlucky the GC might still kick in if you don't pay attention 
 and when you least expect it?

It's either got to be 100% or nothing. The only issue of the GC is the non-determinism.... or if you do corner it and trigger it manually you end up with exactly the types of problems Mr. Chancellor thinks doesn't exist... i.e., the longer you have to put off the GC the worse it becomes(the more time it takes to run or the less memory you have to work with).

Why is this myth of non-determinism still alive? The only truly non-deterministic GC's are concurrent collectors, but alas concurrent collects don't routinely stop-the-world either, so there really aren't any pauses to complain about. D's Mark-Sweep GC is *perfectly* deterministic. It can *only* pause on allocation. Ergo you can determine exactly which allocation caused the problem. You might not expect the function you called to GC-allocate, but that doesn't make it non-deterministic, just not what you expected. Please, stop blaming your missed expectations on the GC. This non-determinism thing is a red herring that is repeated over and over by people who obviously have no idea what they are talking about.

What I want you to do then is tell me exactly which allocation will stop the world. You claim it is deterministic so it should be obvious... hell, it shouldn't even require any code to monitor what the GC is doing. With manual allocation and deallocation I can see exactly when the memory will be allocated and free'ed. That is deterministic. Just because you want to use an all encompassing definition that is meaningless doesn't mean you are right. RNG's are deterministic using your definition, but that is a useless definition.
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e01293f0eaeda7004f18a3728
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

  On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

Don't be a dick. I get the impression you don't actually read my posts. And I also feel like you're a lot more dogmatic about this than you think I am. I'm absolutely fine with GC in most applications, I really couldn't give any shits if most people want a GC. I'm not dogmatic about it, and I've **honestly** tried to love the GC for years now. What I'm concerned about is that I have _no option_ to use D uninhibited when I need to not have the GC. These are the problems: * GC stalls for long periods time at completely un-predictable moments. * GC stalls become longer *and* more frequent as memory becomes less available, and the working pool becomes larger (what a coincidence). * Memory footprint is unknowable, what if you don't have a virtual memory manager? What if your total memory is measured in megabytes? * It's not possible to know when destruction of an object will happen, which has known workarounds (like in C#) but is also annoying in many cases, and supports the prior point. Conclusion: GC is unfit for embedded systems. One of the most significant remaining and compelling uses for a native systems language. The only realistic path I am aware of is to use ARC, which IS a form of GC, and allows a lot more flexibility in the front-end. GC forces one very particular paradigm upon you. ARC is a GC, but it has some complex properties __which can be addressed in various ways__. Unlike a GC which is entirely inflexible. You're not happy with ARC's cleaning objects up on the spot? Something that many people WANT, but I understand zero cleanup times in the running context is in other occasions a strength of GC; fine, just stick the pointer on a dead list, and free it either later during idle time, or on another thread. On the contrary, I haven't heard any proposal for a GC that would allow it to operate in carefully controlled time-slices, or strictly during idle-time. Cycles are a problem with ARC? True, how much effort are you willing to spend to mitigate the problem? None: run a secondary GC in the background to collect cycles (yes, there is still a GC, but it has much less work to do). Some: Disable background GC, manually require user specified weak references and stuff. Note: A user-preferred combination of the 2 could severely mitigate the workload of the background GC if it is still desired to handle some complex situations, or user errors. Are there any other disadvantages to ARC? I don't know of them if there are. Is far as I can tell, an ARC collector could provide identical convenience as the existing GC for anyone that simply doesn't care. It would also seem that it could provide significantly more options and control for those that do. I am _yet to hear anyone present a realistic path forwards using any form of GC_, so what else do I have to go with? Until I know of any other path forward, I'll stand behind the only one I can see. You're just repeating "I don't care about something that a significant subset of D developers do care about, and I don't think any changes should be made to support them". As far as I know, a switch to ARC could be done in a way that 'regular' users don't lose anything, or even notice... why is that so offensive? --089e01293f0eaeda7004f18a3728 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 06:21, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On Mon, 03 Feb 2014 12:02:29 -0800,= Andrei Alexandrescu &lt;<a href=3D"mailto:SeeWebsiteForEmail erdani.org" t= arget=3D"_blank">SeeWebsiteForEmail erdani.org</a><u></u>&gt; wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 2/3/14, 6:57 AM, Frank Bauer wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Anyone asking for the addition of ARC or owning pointers to D, gets<br> pretty much ignored. The topic is &quot;Smart pointers instead of GC?&quot;= ,<br> remember? People here seem to be more interested in diverting to<br> nullable, scope and GC optimization. Telling, indeed.<br> </blockquote> <br> I thought I made it clear that GC avoidance (which includes considering bui= lt-in reference counting) is a major focus of 2014.<br> <br> Andrei<br> <br> </blockquote> <br></div></div> Andrei, I am sorry to report that anything other than complete removal of t= he GC and replacement with compiler generated ARC will be unacceptable to a= certain, highly vocal, subset of D users. No arguments can be made to othe= rwise, regardless of validity. As far as they are concerned the discussion = of ARC vs. GC is closed and decided. ARC is the only path forward to the br= ight and glorious future of D. ARC most efficiently solves all memory manag= ement problems ever encountered. Peer-Reviewed Research and the Scientific = Method be damned! ALL HAIL ARC!<br> <br> Sadly, although written as hyperbole, I feel that the above is fairly close= to the actual position of the ARC crowd.</blockquote><div><br></div><div>D= on&#39;t be a dick.</div><div>I get the impression you don&#39;t actually r= ead my posts. And I also feel like you&#39;re a lot more dogmatic about thi= s than you think I am.</div> <div><br></div><div>I&#39;m absolutely fine with GC in most applications, I= really couldn&#39;t give any shits if most people want a GC. I&#39;m not d= ogmatic about it, and I&#39;ve **honestly** tried to love the GC for years = now.</div> <div>What I&#39;m concerned about is that I have _no option_ to use D uninh= ibited when I need to not have the GC.</div><div><br></div><div>These are t= he problems:</div><div>=C2=A0* GC stalls for long periods time at completel= y un-predictable moments.</div> <div>=C2=A0* GC stalls become longer *and* more frequent as memory becomes = less available, and the working pool becomes larger (what a coincidence).</= div><div>=C2=A0* Memory footprint is unknowable, what if you don&#39;t have= a virtual memory manager? What if your total memory is measured in megabyt= es?</div> <div>=C2=A0* It&#39;s not possible to know when destruction of an object wi= ll happen, which has known workarounds (like in C#) but is also annoying in= many cases, and supports the prior point.</div><div><br></div><div>Conclus= ion:</div> <div>=C2=A0 GC is unfit for embedded systems. One of the most significant r= emaining and compelling uses for a native systems language.</div><div><br><= /div><div>The only realistic path I am aware of is to use ARC, which IS a f= orm of GC, and allows a lot more flexibility in the front-end.</div> <div>GC forces one very particular paradigm upon you.</div><div>ARC is a GC= , but it has some complex properties __which can be addressed in various wa= ys__. Unlike a GC which is entirely inflexible.</div><div><br></div><div> You&#39;re not happy with ARC&#39;s cleaning objects up on the spot? Someth= ing that many people WANT, but I understand zero cleanup times in the runni= ng context is in other occasions a strength of GC; fine, just stick the poi= nter on a dead list, and free it either later during idle time, or on anoth= er thread. On the contrary, I haven&#39;t heard any proposal for a GC that = would allow it to operate in carefully controlled time-slices, or strictly = during idle-time.</div> <div>Cycles are a problem with ARC? True, how much effort are you willing t= o spend to mitigate the problem? None: run a secondary GC in the background= to collect cycles (yes, there is still a GC, but it has much less work to = do). Some: Disable background GC, manually require user specified weak refe= rences and stuff. Note: A user-preferred combination of the 2 could severel= y mitigate the workload of the background GC if it is still desired to hand= le some complex situations, or user errors.<br> </div><div>Are there any other disadvantages to ARC? I don&#39;t know of th= em if there are.</div><div><br></div><div>Is far as I can tell, an ARC coll= ector could provide identical convenience as the existing GC for anyone tha= t simply doesn&#39;t care. It would also seem that it could provide signifi= cantly more options and control for those that do.</div> <div><br></div><div>I am _yet to hear anyone present a realistic path forwa= rds using any form of GC_, so what else do I have to go with? Until I know = of any other path forward, I&#39;ll stand behind the only one I can see.</d= iv> <div>You&#39;re just repeating &quot;I don&#39;t care about something that = a significant subset of D developers do care about, and I don&#39;t think a= ny changes should be made to support them&quot;.</div><div>As far as I know= , a switch to ARC could be done in a way that &#39;regular&#39; users don&#= 39;t lose anything, or even notice... why is that so offensive?</div> </div></div></div> --089e01293f0eaeda7004f18a3728--
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c2e60a35b2d504f18a4c7c
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 4 February 2014 06:52, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 12:40:20 -0800, Dmitry Olshansky <
 dmitry.olsh gmail.com> wrote:

  04-Feb-2014 00:21, Adam Wilson =D0=BF=D0=B8=D1=88=D0=B5=D1=82:
 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

  On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei


 Sadly, although written as hyperbole, I feel that the above is fairly
 close to the actual position of the ARC crowd.

simply too stupid as an allocator. I had some numbers that I'd need to d=


 up were GC.malloc was ~2x slower then malloc even with garbage collectio=


 disabled.

 With that said I'd focus on ref-counting somehow coexisting with tracing
 GC. It doesn't go well at the moment - try creating dynamic arrays of
 std.stdio.File (that is ref-counted).

one and still have a functioning memory management system. It implements exactly none of the optimizations recommended for Mark-Sweep GC's in the =

 Handbook.

 That said, I firmly believe that wholesale replacement of the GC is
 throwing the baby out with the bathwater. Effectively, the poor D GC
 implementation has become an excuse to launch a crusade against all GC's
 everywhere, never mind that Java and the .NET GC's are consistent example=

 of just how good GC's can actually be.

Point me at a proposal for a GC that satisfies the problems in my prior email, which is realistically implement-able in D, and I'll shut up. --001a11c2e60a35b2d504f18a4c7c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 06:52, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5">On Mon, 03 Feb 2014 12:40:20 -0800,= Dmitry Olshansky &lt;<a href=3D"mailto:dmitry.olsh gmail.com" target=3D"_b= lank">dmitry.olsh gmail.com</a>&gt; wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> 04-Feb-2014 00:21, Adam Wilson =D0=BF=D0=B8=D1=88=D0=B5=D1=82:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu<br> &lt;<a href=3D"mailto:SeeWebsiteForEmail erdani.org" target=3D"_blank">SeeW= ebsiteForEmail erdani.org</a><u></u>&gt; wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 2/3/14, 6:57 AM, Frank Bauer wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Anyone asking for the addition of ARC or owning pointers to D, gets<br> pretty much ignored. The topic is &quot;Smart pointers instead of GC?&quot;= ,<br> remember? People here seem to be more interested in diverting to<br> nullable, scope and GC optimization. Telling, indeed.<br> </blockquote> <br> I thought I made it clear that GC avoidance (which includes<br> considering built-in reference counting) is a major focus of 2014.<br> <br> Andrei<br> <br> </blockquote> <br> </blockquote> ...<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Sadly, although written as hyperbole, I feel that the above is fairly<br> close to the actual position of the ARC crowd.<br> <br> </blockquote> <br> I won&#39;t be surprised that half of current GC problems are because it&#3= 9;s simply too stupid as an allocator. I had some numbers that I&#39;d need= to dig up were GC.malloc was ~2x slower then malloc even with garbage coll= ection disabled.<br> <br> With that said I&#39;d focus on ref-counting somehow coexisting with tracin= g GC. It doesn&#39;t go well at the moment - try creating dynamic arrays of= std.stdio.File (that is ref-counted).<br> <br> </blockquote> <br></div></div> I will not defend the current GC, it is about as stupid as you can make one= and still have a functioning memory management system. It implements exact= ly none of the optimizations recommended for Mark-Sweep GC&#39;s in the GC = Handbook.<br> <br> That said, I firmly believe that wholesale replacement of the GC is throwin= g the baby out with the bathwater. Effectively, the poor D GC implementatio= n has become an excuse to launch a crusade against all GC&#39;s everywhere,= never mind that Java and the .NET GC&#39;s are consistent examples of just= how good GC&#39;s can actually be.</blockquote> <div><br></div><div>Point me at a proposal for a GC that satisfies the prob= lems in my prior email, which is realistically implement-able in D, and I&#= 39;ll shut up.</div></div></div></div> --001a11c2e60a35b2d504f18a4c7c--
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7bd6c03e5c740b04f18a573e
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 06:52, Adam Wilson <flyboynw gmail.com> wrote:


 actually be.

Also, neither languages are systems languages, or practical/appropriate on embedded, or memory limited systems. What is your argument? That D not be a systems language? Additionally, if you are going to beat that drum, you need to prove that either languages GC technology is even possible in the context of D, otherwise it's a red herring. --047d7bd6c03e5c740b04f18a573e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 06:52, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5"><span style=3D"color:rgb(34,34,34)"=
=C2=A0</span><br></div></div></blockquote><blockquote class=3D"gmail_quote=

ava and the .NET GC&#39;s are consistent examples of just how good GC&#39;s= can actually be.</blockquote> <div><br></div><div>Also, neither languages are systems languages, or pract= ical/appropriate on embedded, or memory limited systems. What is your argum= ent? That D not be a systems language?</div><div>Additionally, if you are g= oing to beat that drum, you need to prove that either languages GC technolo= gy is even possible in the context of D, otherwise it&#39;s a red herring.<= /div> </div></div></div> --047d7bd6c03e5c740b04f18a573e--
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0112c4bcd2723304f18a9f75
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 10:19, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 15:26:22 -0800, Frustrated <c1514843 drdrb.com> wrote:

  On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor
 wrote:

 You can always force the GC to run between cycles in your game, and
 turn off automatic sweeps.  This is how most games operate nowadays.
 It's also probably possible to create a drop-in replacement for the GC
 to do something else.   I could see if being *VERY* useful to make the
 GC take a compile-time parameter to select which GC engine is used.

This is just non-sense. Maybe this is why modern games suck then? How do you guarantee that the GC won't kick in at the most inopportune times? Oh, you manually trigger it? When? Right at the moment when the player is about to down the boss after a 45 min fight? Oh, right.. you just segfault cause there is no memory left. On Monday, 3 February 2014 at 22:51:50 UTC, Frank Bauer wrote: I'm not quite sure that I understand what you mean by GC avoidance being
 a major focus of 2014 though. In the long term, can I look forward to
 writing an absolutely, like in 100 %, like in guaranteed, GC free D app
 with all of current D's and Phobos' features if I choose to? Or does it
 mean: well for the most part it will avoid the GC, but if you're unlucky
 the GC might still kick in if you don't pay attention and when you least
 expect it?

It's either got to be 100% or nothing. The only issue of the GC is the non-determinism.... or if you do corner it and trigger it manually you end up with exactly the types of problems Mr. Chancellor thinks doesn't exist... i.e., the longer you have to put off the GC the worse it becomes(the more time it takes to run or the less memory you have to work with).

non-deterministic GC's are concurrent collectors, but alas concurrent collects don't routinely stop-the-world either, so there really aren't any pauses to complain about. D's Mark-Sweep GC is *perfectly* deterministic. It can *only* pause on allocation. Ergo you can determine exactly which allocation caused the problem. You might not expect the function you called to GC-allocate, but that doesn't make it non-deterministic, just not what you expected. Please, stop blaming your missed expectations on the GC. This non-determinism thing is a red herring that is repeated over and over by people who obviously have no idea what they are talking about.

Your assertion makes the assumption that people who write huge complex programs have completely control over their code, and/or depend on zero libraries, which is a ridiculous notion. I'm quite sick of people making claims like that. If the default is to do something incompatible with my use case, and I depend on any libraries (including phobos), then it's safe to say, I'm running code that is incompatible with my use case. What are my options then? Additionally, in D, you don't have to type 'new' to allocate memory, it happens all the time... a closure here, a concatenate there. These are core language features, and to say that I'm meant to avoid them in my million loc program, authored by perhaps hundreds of programmers, because I need to be certain where every alloc is being issued from, is quite unrealistic to say the least. --089e0112c4bcd2723304f18a9f75 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 10:19, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"im">On Mon, 03 Feb 2014 15:26:22 -0800, Frustrated &lt;<a hre= f=3D"mailto:c1514843 drdrb.com" target=3D"_blank">c1514843 drdrb.com</a>&gt= ; wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On Monday, 3 February 2014 at 21:42:59 UTC, Shammah Chancellor<br> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> You can always force the GC to run between cycles in your game, and<br> turn off automatic sweeps. =C2=A0This is how most games operate nowadays.<b= r> It&#39;s also probably possible to create a drop-in replacement for the GC<= br> to do something else. =C2=A0 I could see if being *VERY* useful to make the= <br> GC take a compile-time parameter to select which GC engine is used.<br> </blockquote> <br> <br> This is just non-sense. Maybe this is why modern games suck then?<br> How do you guarantee that the GC won&#39;t kick in at the most<br> inopportune times? Oh, you manually trigger it? When? Right at<br> the moment when the player is about to down the boss after a 45<br> min fight?<br> <br> Oh, right.. you just segfault cause there is no memory left.<br> <br> On Monday, 3 February 2014 at 22:51:50 UTC, Frank Bauer wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;m not quite sure that I understand what you mean by GC avoidance bein= g a major focus of 2014 though. In the long term, can I look forward to wri= ting an absolutely, like in 100 %, like in guaranteed, GC free D app with a= ll of current D&#39;s and Phobos&#39; features if I choose to? Or does it m= ean: well for the most part it will avoid the GC, but if you&#39;re unlucky= the GC might still kick in if you don&#39;t pay attention and when you lea= st expect it?<br> </blockquote> <br> It&#39;s either got to be 100% or nothing. The only issue of the GC<br> is the non-determinism.... or if you do corner it and trigger it<br> manually you end up with exactly the types of problems Mr.<br> Chancellor thinks doesn&#39;t exist... i.e., the longer you have to<br> put off the GC the worse it becomes(the more time it takes to run<br> or the less memory you have to work with).<br> <br> </blockquote> <br></div> Why is this myth of non-determinism still alive? The only truly non-determi= nistic GC&#39;s are concurrent collectors, but alas concurrent collects don= &#39;t routinely stop-the-world either, so there really aren&#39;t any paus= es to complain about. D&#39;s Mark-Sweep GC is *perfectly* deterministic. I= t can =C2=A0*only* pause on allocation. Ergo you can determine exactly whic= h allocation caused the problem. You might not expect the function you call= ed to GC-allocate, but that doesn&#39;t make it non-deterministic, just not= what you expected. Please, stop blaming your missed expectations on the GC= . This non-determinism thing is a red herring that is repeated over and ove= r by people who obviously have no idea what they are talking about.</blockq= uote> <div><br></div><div>Your assertion makes the assumption that people who wri= te huge complex programs have completely control over their code, and/or de= pend on zero libraries, which is a ridiculous notion.</div><div>I&#39;m qui= te sick of people making claims like that. If the default is to do somethin= g incompatible with my use case, and I depend on any libraries (including p= hobos), then it&#39;s safe to say, I&#39;m running code that is incompatibl= e with my use case.</div> <div>What are my options then?</div><div><br></div><div>Additionally, in D,= you don&#39;t have to type &#39;new&#39; to allocate memory, it happens al= l the time... a closure here, a concatenate there. These are core language = features, and to say that I&#39;m meant to avoid them in my million loc pro= gram, authored by perhaps hundreds of programmers, because I need to be cer= tain where every alloc is being issued from, is quite unrealistic to say th= e least.</div> </div></div></div> --089e0112c4bcd2723304f18a9f75--
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 17:04:08 -0800, Manu <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

  On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

Don't be a dick. I get the impression you don't actually read my posts. And I also feel like you're a lot more dogmatic about this than you think I am. I'm absolutely fine with GC in most applications, I really couldn't give any shits if most people want a GC. I'm not dogmatic about it, and I've **honestly** tried to love the GC for years now. What I'm concerned about is that I have _no option_ to use D uninhibited when I need to not have the GC. These are the problems: * GC stalls for long periods time at completely un-predictable moments. * GC stalls become longer *and* more frequent as memory becomes less available, and the working pool becomes larger (what a coincidence). * Memory footprint is unknowable, what if you don't have a virtual memory manager? What if your total memory is measured in megabytes? * It's not possible to know when destruction of an object will happen, which has known workarounds (like in C#) but is also annoying in many cases, and supports the prior point. Conclusion: GC is unfit for embedded systems. One of the most significant remaining and compelling uses for a native systems language. The only realistic path I am aware of is to use ARC, which IS a form of GC, and allows a lot more flexibility in the front-end. GC forces one very particular paradigm upon you. ARC is a GC, but it has some complex properties __which can be addressed in various ways__. Unlike a GC which is entirely inflexible. You're not happy with ARC's cleaning objects up on the spot? Something that many people WANT, but I understand zero cleanup times in the running context is in other occasions a strength of GC; fine, just stick the pointer on a dead list, and free it either later during idle time, or on another thread. On the contrary, I haven't heard any proposal for a GC that would allow it to operate in carefully controlled time-slices, or strictly during idle-time. Cycles are a problem with ARC? True, how much effort are you willing to spend to mitigate the problem? None: run a secondary GC in the background to collect cycles (yes, there is still a GC, but it has much less work to do). Some: Disable background GC, manually require user specified weak references and stuff. Note: A user-preferred combination of the 2 could severely mitigate the workload of the background GC if it is still desired to handle some complex situations, or user errors. Are there any other disadvantages to ARC? I don't know of them if there are. Is far as I can tell, an ARC collector could provide identical convenience as the existing GC for anyone that simply doesn't care. It would also seem that it could provide significantly more options and control for those that do. I am _yet to hear anyone present a realistic path forwards using any form of GC_, so what else do I have to go with? Until I know of any other path forward, I'll stand behind the only one I can see. You're just repeating "I don't care about something that a significant subset of D developers do care about, and I don't think any changes should be made to support them". As far as I know, a switch to ARC could be done in a way that 'regular' users don't lose anything, or even notice... why is that so offensive?

I am not trying to be a dick. But I do feel like a small number of people are trying to gang up on me for daring to point out that the solution they've proposed solution might have bigger problems for other people than they care to admit. You still haven't dealt with the cyclic reference problem in ARC. There is absolutely no way ARC can handle that without programmer input, therefore, it is simply not possible to switch D to ARC without adding some language support to deal with cyclic-refs. Ergo, it is simply not possible to seamlessly switch D to ARC without creating all kinds of havoc as people now how memory leaks where they didn't before. In order to support ARC the D language will necessarily have to grow/change to accommodate it. Apple devs constantly have trouble with cyclic-refs to this day. I am not against supporting ARC side-by-side with the GC (I'm actually quite for it, I would love the flexibility), but it is unrealistic to make ARC the default option in D as that would subtly break all existing D code, something that Walter has point-blank refused to do in much smaller easier to find+fix cases. You can't grep for a weak-ref. So if that is what you are asking for, then yes, it will never happen in D. Also, I don't think you've fully considered what the perf penalty actually is for a *good* ARC implementation. I just leafed through the P-Code in the GC Handbook for their ARC implementation, it's about 4x longer than what their best P-Code Mark-Sweep implementation is. I would also like to point out that the GC Handbook points out six scientifically confirmed problems with ARC. (See Page 59) 1. RC imposes a time overhead on mutators in order to manipulate the counter. 2. Both the counter manipulation and pointer load/store operations MUST be atomic to prevent races. 3. Naive RC turns read ops into store ops to update the count. 4. No RC can reclaim cyclic data structures, which are much more common than is typically understood. [Bacon and Rajan 2001] 5. Counter must be the same size as the pointer, which can result in significant overhead for small objects. 6. RC can still pause. When the last head to a large pointer structure is deleted, RC MUST delete each descendant node. Note that these are paraphrases of the book, not me talking. And these apply equally to ARC and vanilla RC. Boehm demonstrated in one of his papers (2004) that thread-safe ARC may even lead to longer maximum pause times than a standard Tracing GC. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu <turkeyman gmail.com> 
 wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to 
 D, gets
 pretty much ignored. The topic is "Smart pointers instead 
 of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

Don't be a dick. I get the impression you don't actually read my posts. And I also feel like you're a lot more dogmatic about this than you think I am. I'm absolutely fine with GC in most applications, I really couldn't give any shits if most people want a GC. I'm not dogmatic about it, and I've **honestly** tried to love the GC for years now. What I'm concerned about is that I have _no option_ to use D uninhibited when I need to not have the GC. These are the problems: * GC stalls for long periods time at completely un-predictable moments. * GC stalls become longer *and* more frequent as memory becomes less available, and the working pool becomes larger (what a coincidence). * Memory footprint is unknowable, what if you don't have a virtual memory manager? What if your total memory is measured in megabytes? * It's not possible to know when destruction of an object will happen, which has known workarounds (like in C#) but is also annoying in many cases, and supports the prior point. Conclusion: GC is unfit for embedded systems. One of the most significant remaining and compelling uses for a native systems language. The only realistic path I am aware of is to use ARC, which IS a form of GC, and allows a lot more flexibility in the front-end. GC forces one very particular paradigm upon you. ARC is a GC, but it has some complex properties __which can be addressed in various ways__. Unlike a GC which is entirely inflexible. You're not happy with ARC's cleaning objects up on the spot? Something that many people WANT, but I understand zero cleanup times in the running context is in other occasions a strength of GC; fine, just stick the pointer on a dead list, and free it either later during idle time, or on another thread. On the contrary, I haven't heard any proposal for a GC that would allow it to operate in carefully controlled time-slices, or strictly during idle-time. Cycles are a problem with ARC? True, how much effort are you willing to spend to mitigate the problem? None: run a secondary GC in the background to collect cycles (yes, there is still a GC, but it has much less work to do). Some: Disable background GC, manually require user specified weak references and stuff. Note: A user-preferred combination of the 2 could severely mitigate the workload of the background GC if it is still desired to handle some complex situations, or user errors. Are there any other disadvantages to ARC? I don't know of them if there are. Is far as I can tell, an ARC collector could provide identical convenience as the existing GC for anyone that simply doesn't care. It would also seem that it could provide significantly more options and control for those that do. I am _yet to hear anyone present a realistic path forwards using any form of GC_, so what else do I have to go with? Until I know of any other path forward, I'll stand behind the only one I can see. You're just repeating "I don't care about something that a significant subset of D developers do care about, and I don't think any changes should be made to support them". As far as I know, a switch to ARC could be done in a way that 'regular' users don't lose anything, or even notice... why is that so offensive?

I am not trying to be a dick. But I do feel like a small number of people are trying to gang up on me for daring to point out that the solution they've proposed solution might have bigger problems for other people than they care to admit.

The problem is that you are selfish. You won't let anyone in the sand box to play with their toys... it's your sand box, even though it's not. It's your toys, even though it's not. You pitch a fit when someone comes into the sand box because you don't want them their. They might mess up the sand. Why can't you get it through your thick skull that just because people are suggesting alternative methods does not mean that you can't have your precious AGC? Also, why is it so difficult that just because you have no need for real-time applications that others don't either? Your method of thinking is very arrogant and I wish you the best of luck at the center of the universe... I hear it's a very unfriendly place.
Feb 03 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 1. RC imposes a time overhead on mutators in order to 
 manipulate the counter.

Usually you just do a retain() when your thread attain ownership (at the root of the call tree). If you use embedded counters you most likely want the data in the cache anyway, but the cacheline is made dirty. So it cost you a write back. That affects reads of small objects more than writes. (The C++ implementation does not use embedded counters.)
 2. Both the counter manipulation and pointer load/store 
 operations MUST be atomic to prevent races.

Not with transactional memory. You revert to locking, and you probably want that kind of synchronization anyway if you run multi threaded.
 3. Naive RC turns read ops into store ops to update the count

If we assume naive RC, then we should assume naive GC.
 4. No RC can reclaim cyclic data structures, which are much 
 more common than is typically understood. [Bacon and Rajan 2001]

Weak pointers are sufficient to break cycles. You need a model, but you should have one anyway. In cases where you build temporary data structures you can just avoid RC altogether and just use a pool, then free the entire pool.
 5. Counter must be the same size as the pointer, which can 
 result in significant overhead for small objects.

No? The counter must be able to hold the # of retain() calls.
 6. RC can still pause. When the last head to a large pointer 
 structure is deleted, RC MUST delete each descendant node.

Not really, you can postpone the deletion by pushing it onto a queue. The free it not by node at your own leisure...
 Note that these are paraphrases of the book, not me talking. 
 And these apply equally to ARC and vanilla RC.

You don't use RC alone, you use it with fully owned pointers, region pools, etc. GC is simpler and more uniform, and that's it.
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 17:54:22 -0800, Frustrated <c1514843 drdrb.com> wrote:

 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, gets
 pretty much ignored. The topic is "Smart pointers instead of GC?",
 remember? People here seem to be more interested in diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

Don't be a dick. I get the impression you don't actually read my posts. And I also feel like you're a lot more dogmatic about this than you think I am. I'm absolutely fine with GC in most applications, I really couldn't give any shits if most people want a GC. I'm not dogmatic about it, and I've **honestly** tried to love the GC for years now. What I'm concerned about is that I have _no option_ to use D uninhibited when I need to not have the GC. These are the problems: * GC stalls for long periods time at completely un-predictable moments. * GC stalls become longer *and* more frequent as memory becomes less available, and the working pool becomes larger (what a coincidence). * Memory footprint is unknowable, what if you don't have a virtual memory manager? What if your total memory is measured in megabytes? * It's not possible to know when destruction of an object will happen, which has known workarounds (like in C#) but is also annoying in many cases, and supports the prior point. Conclusion: GC is unfit for embedded systems. One of the most significant remaining and compelling uses for a native systems language. The only realistic path I am aware of is to use ARC, which IS a form of GC, and allows a lot more flexibility in the front-end. GC forces one very particular paradigm upon you. ARC is a GC, but it has some complex properties __which can be addressed in various ways__. Unlike a GC which is entirely inflexible. You're not happy with ARC's cleaning objects up on the spot? Something that many people WANT, but I understand zero cleanup times in the running context is in other occasions a strength of GC; fine, just stick the pointer on a dead list, and free it either later during idle time, or on another thread. On the contrary, I haven't heard any proposal for a GC that would allow it to operate in carefully controlled time-slices, or strictly during idle-time. Cycles are a problem with ARC? True, how much effort are you willing to spend to mitigate the problem? None: run a secondary GC in the background to collect cycles (yes, there is still a GC, but it has much less work to do). Some: Disable background GC, manually require user specified weak references and stuff. Note: A user-preferred combination of the 2 could severely mitigate the workload of the background GC if it is still desired to handle some complex situations, or user errors. Are there any other disadvantages to ARC? I don't know of them if there are. Is far as I can tell, an ARC collector could provide identical convenience as the existing GC for anyone that simply doesn't care. It would also seem that it could provide significantly more options and control for those that do. I am _yet to hear anyone present a realistic path forwards using any form of GC_, so what else do I have to go with? Until I know of any other path forward, I'll stand behind the only one I can see. You're just repeating "I don't care about something that a significant subset of D developers do care about, and I don't think any changes should be made to support them". As far as I know, a switch to ARC could be done in a way that 'regular' users don't lose anything, or even notice... why is that so offensive?

I am not trying to be a dick. But I do feel like a small number of people are trying to gang up on me for daring to point out that the solution they've proposed solution might have bigger problems for other people than they care to admit.

The problem is that you are selfish. You won't let anyone in the sand box to play with their toys... it's your sand box, even though it's not. It's your toys, even though it's not. You pitch a fit when someone comes into the sand box because you don't want them their. They might mess up the sand. Why can't you get it through your thick skull that just because people are suggesting alternative methods does not mean that you can't have your precious AGC? Also, why is it so difficult that just because you have no need for real-time applications that others don't either? Your method of thinking is very arrogant and I wish you the best of luck at the center of the universe... I hear it's a very unfriendly place.

Funny how you conveniently cut the thread immediately prior to paragraph where I say I would be all for supporting ARC side-by-side with the GC. It was only two paragraphs down, I can't imagine that it would've been that hard to include. I can only imagine that you left it off because it makes your name-calling look silly. However I did point out that making ARC the DEFAULT is unlikely to ever happen. Walter has consistently refused to approve much smaller breaking changes in the past. Why would he all-of-a-sudden decide that now is the time for the biggest breaking change in the history of D? I am just pointing out that the creator of D is not likely to approve that pull request while he is still breathing. Please don't shoot the messenger. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 18:14:36 -0800, Ola Fosheim Gr=F8stad  =

<ola.fosheim.grostad+dlang gmail.com> wrote:

 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 1. RC imposes a time overhead on mutators in order to manipulate the =


 counter.

Usually you just do a retain() when your thread attain ownership (at t=

 root of the call tree). If you use embedded counters you most likely  =

 want the data in the cache anyway, but the cacheline is made dirty. So=

 it cost you a write back. That affects reads of small objects more tha=

 writes.

 (The C++ implementation does not use embedded counters.)

 2. Both the counter manipulation and pointer load/store operations MU=


 be atomic to prevent races.

Not with transactional memory. You revert to locking, and you probably=

 want that kind of synchronization anyway if you run multi threaded.

 3. Naive RC turns read ops into store ops to update the count

If we assume naive RC, then we should assume naive GC.

Indeed D is a naive GC, but that's not the point. Naive RC incurs a = penalty that even Naive GC's don't.
 4. No RC can reclaim cyclic data structures, which are much more comm=


 than is typically understood. [Bacon and Rajan 2001]

Weak pointers are sufficient to break cycles. You need a model, but yo=

 should have one anyway.

Yes, they absolutely are. However, supporting them at the automatic leve= l = requires adding keywords to the language. If ARC is made the default in = D = you will automatically have memory leaks where before you did not, and = programmer will have to carefully scan the code by hand to find everythi= ng = that needs to be marked with a 'weak' keyword. It's an extremely = frustrating and error-prone process. The compiler cannot detect = cyclic-refs, it's a halting problem.
 In cases where you build temporary data structures you can just avoid =

 altogether and just use a pool, then free the entire pool.

 5. Counter must be the same size as the pointer, which can result in =


 significant overhead for small objects.

No? The counter must be able to hold the # of retain() calls.

Yes, which can theoretically be as many as a 64-bit integer. In a system= s = language you have to design for what is possible, not merely likely. = Because someday, some nutcase, somewhere is going to overflow the counte= r, = and given that it's hidden by the compiler, will be VERY tricky to figur= e = out. And he will have a perfectly rational reason for doing so.
 6. RC can still pause. When the last head to a large pointer structur=


 is deleted, RC MUST delete each descendant node.

Not really, you can postpone the deletion by pushing it onto a queue. =

 The free it not by node at your own leisure...

The GC Handbook deals with that idea directly: "Unfortunately, this = technique allows large garbage structures to be hidden by smaller ones, = = and hence increases overall space requirements. [Boehm, 2004]" - The = Garbage Collection Handbook
 Note that these are paraphrases of the book, not me talking. And thes=


 apply equally to ARC and vanilla RC.

You don't use RC alone, you use it with fully owned pointers, region =

 pools, etc.

 GC is simpler and more uniform, and that's it.

That may be "it", but it's a pretty big "it". Simplicity and uniformity = = are VERY important. -- = Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 18:14:36 -0800, Ola Fosheim Gr=F8stad  =

<ola.fosheim.grostad+dlang gmail.com> wrote:

 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 1. RC imposes a time overhead on mutators in order to manipulate the =


 counter.

Usually you just do a retain() when your thread attain ownership (at t=

 root of the call tree). If you use embedded counters you most likely  =

 want the data in the cache anyway, but the cacheline is made dirty. So=

 it cost you a write back. That affects reads of small objects more tha=

 writes.

 (The C++ implementation does not use embedded counters.)

There are many more ways to modify the count than a thread.
 2. Both the counter manipulation and pointer load/store operations MU=


 be atomic to prevent races.

Not with transactional memory. You revert to locking, and you probably=

 want that kind of synchronization anyway if you run multi threaded.

Transactional memory incurs a massive perf penalty beyond even what a lo= ck = would. You might be able to do better than a lock with Intel's TSX, but = = that is brand new, available on Haswells only, and even then only the = high-end models. I doubt you'll be able to sell TM-ARC to perf conscienc= e = users as the perf will most likely be much worse than a GC for the next = = decade. -- = Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "woh" <wojw yahoo.com> writes:
  please this GC handbook sound good, can u post it? A queue uses 
more memory than not a queue? Fuk I did not know, I so dumb. Live 
longer more memory used, what///.


  Tuesday, 4 February 2014 at 02:33:50 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 18:14:36 -0800, Ola Fosheim Grøstad 
 <ola.fosheim.grostad+dlang gmail.com> wrote:

 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 1. RC imposes a time overhead on mutators in order to 
 manipulate the counter.

Usually you just do a retain() when your thread attain ownership (at the root of the call tree). If you use embedded counters you most likely want the data in the cache anyway, but the cacheline is made dirty. So it cost you a write back. That affects reads of small objects more than writes. (The C++ implementation does not use embedded counters.)

There are many more ways to modify the count than a thread.
 2. Both the counter manipulation and pointer load/store 
 operations MUST be atomic to prevent races.

Not with transactional memory. You revert to locking, and you probably want that kind of synchronization anyway if you run multi threaded.

Transactional memory incurs a massive perf penalty beyond even what a lock would. You might be able to do better than a lock with Intel's TSX, but that is brand new, available on Haswells only, and even then only the high-end models. I doubt you'll be able to sell TM-ARC to perf conscience users as the perf will most likely be much worse than a GC for the next decade.

Feb 03 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Tuesday, 4 February 2014 at 02:15:21 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:54:22 -0800, Frustrated 
 <c1514843 drdrb.com> wrote:

 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu 
 <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers 
 to D, gets
 pretty much ignored. The topic is "Smart pointers instead 
 of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

Don't be a dick. I get the impression you don't actually read my posts. And I also feel like you're a lot more dogmatic about this than you think I am. I'm absolutely fine with GC in most applications, I really couldn't give any shits if most people want a GC. I'm not dogmatic about it, and I've **honestly** tried to love the GC for years now. What I'm concerned about is that I have _no option_ to use D uninhibited when I need to not have the GC. These are the problems: * GC stalls for long periods time at completely un-predictable moments. * GC stalls become longer *and* more frequent as memory becomes less available, and the working pool becomes larger (what a coincidence). * Memory footprint is unknowable, what if you don't have a virtual memory manager? What if your total memory is measured in megabytes? * It's not possible to know when destruction of an object will happen, which has known workarounds (like in C#) but is also annoying in many cases, and supports the prior point. Conclusion: GC is unfit for embedded systems. One of the most significant remaining and compelling uses for a native systems language. The only realistic path I am aware of is to use ARC, which IS a form of GC, and allows a lot more flexibility in the front-end. GC forces one very particular paradigm upon you. ARC is a GC, but it has some complex properties __which can be addressed in various ways__. Unlike a GC which is entirely inflexible. You're not happy with ARC's cleaning objects up on the spot? Something that many people WANT, but I understand zero cleanup times in the running context is in other occasions a strength of GC; fine, just stick the pointer on a dead list, and free it either later during idle time, or on another thread. On the contrary, I haven't heard any proposal for a GC that would allow it to operate in carefully controlled time-slices, or strictly during idle-time. Cycles are a problem with ARC? True, how much effort are you willing to spend to mitigate the problem? None: run a secondary GC in the background to collect cycles (yes, there is still a GC, but it has much less work to do). Some: Disable background GC, manually require user specified weak references and stuff. Note: A user-preferred combination of the 2 could severely mitigate the workload of the background GC if it is still desired to handle some complex situations, or user errors. Are there any other disadvantages to ARC? I don't know of them if there are. Is far as I can tell, an ARC collector could provide identical convenience as the existing GC for anyone that simply doesn't care. It would also seem that it could provide significantly more options and control for those that do. I am _yet to hear anyone present a realistic path forwards using any form of GC_, so what else do I have to go with? Until I know of any other path forward, I'll stand behind the only one I can see. You're just repeating "I don't care about something that a significant subset of D developers do care about, and I don't think any changes should be made to support them". As far as I know, a switch to ARC could be done in a way that 'regular' users don't lose anything, or even notice... why is that so offensive?

I am not trying to be a dick. But I do feel like a small number of people are trying to gang up on me for daring to point out that the solution they've proposed solution might have bigger problems for other people than they care to admit.

The problem is that you are selfish. You won't let anyone in the sand box to play with their toys... it's your sand box, even though it's not. It's your toys, even though it's not. You pitch a fit when someone comes into the sand box because you don't want them their. They might mess up the sand. Why can't you get it through your thick skull that just because people are suggesting alternative methods does not mean that you can't have your precious AGC? Also, why is it so difficult that just because you have no need for real-time applications that others don't either? Your method of thinking is very arrogant and I wish you the best of luck at the center of the universe... I hear it's a very unfriendly place.

Funny how you conveniently cut the thread immediately prior to paragraph where I say I would be all for supporting ARC side-by-side with the GC. It was only two paragraphs down, I can't imagine that it would've been that hard to include. I can only imagine that you left it off because it makes your name-calling look silly. However I did point out that making ARC the DEFAULT is unlikely to ever happen. Walter has consistently refused to approve much smaller breaking changes in the past. Why would he all-of-a-sudden decide that now is the time for the biggest breaking change in the history of D? I am just pointing out that the creator of D is not likely to approve that pull request while he is still breathing. Please don't shoot the messenger.

It has nothing to do with side by side. Who in their right mind would argue that the AGC is bad in all cases? No one. Who wants to deal with manual memory management when writing programs that do not have real-time requirements? But the fact remains, "D is AGC dependent" and "Some programs perform poorly when used with AGC" means that D is a bad choice for those programs, correct? So, your logic is that we are forced to use D's GC if we want to use D because you think if ARC or MMM is implemented you'll be forced to use it... and, of course, you don't write RT apps so it's not your problem. All we are asking is that D not be GC dependent. This allows one to write RT apps, embedded apps, etc. THIS DOES NOT MEAN you can't use a GC to handle your MM. It just means there are other options, and by your pretentious stance against "us" tells us you don't want D to change. Why not join our team? We want AGC, ARC, MMM, MRC, etc. To do this though D and phobos have to be lifted from the grips of the GC. IT DOESN'T MEAN you'll have to manage the memory of phobo's. You can still use the GC just like you do now without any changes. You'll never know the difference. BUT WE WILL. It will be the perfect world. We get what we want and you get what you want. As it stands you, since you have been arguing points against us, you must not agree with us, which means you do not want us to be happy. Yet we want both of us to be happy. So you must be wrong, unless you know for a fact that we both can't be happy, which is non-sense(cause we can have AGC and ARC side by side). So which is it? Are you with us or against us? Do you want to rid D and Phobos of the hardcoded AGC and allow for other methods of MM? (hopefully you are intelligent enough to understand that this doesn't mean that the AGC will go away or be any different to use) [I'm also not in favor of ridding D of the AGC all together... I don't think anyone is, it's there, it works for the most part for most applications... it is handy and useful in many cases... BUT NOT ALL. So stop pretending it is, and lets solve the problem]
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e013c5eb634bffa04f18c4823
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 12:57, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org
 wrote:

 On 2/3/14, 5:36 PM, Adam Wilson wrote:

 You still haven't dealt with the cyclic reference problem in ARC. There
 is absolutely no way ARC can handle that without programmer input,
 therefore, it is simply not possible to switch D to ARC without adding
 some language support to deal with cyclic-refs. Ergo, it is simply not
 possible to seamlessly switch D to ARC without creating all kinds of
 havoc as people now how memory leaks where they didn't before. In order
 to support ARC the D language will necessarily have to grow/change to
 accommodate it. Apple devs constantly have trouble with cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles.

I agree, I suggested that a few times, and Adam ignored it each time. With this approach, performance/aggressive users would be able to disable the backing GC, and deal with cycles manually, taking responsibility for leaking themselves. All other users would be able to ignore the problem and have it collected by the backing GC, business as usual. The advantage here is *choice*. Users would then have a fully automated system and/or have quite articulate control over it's influence on their app. --089e013c5eb634bffa04f18c4823 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 12:57, Andrei Alexandrescu <span dir=3D"ltr">&lt;<a href=3D"= mailto:SeeWebsiteForEmail erdani.org" target=3D"_blank">SeeWebsiteForEmail = erdani.org</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On 2/3/14, 5:36 PM, Adam W= ilson wrote:<br> </div><div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> You still haven&#39;t dealt with the cyclic reference problem in ARC. There= <br> is absolutely no way ARC can handle that without programmer input,<br> therefore, it is simply not possible to switch D to ARC without adding<br> some language support to deal with cyclic-refs. Ergo, it is simply not<br> possible to seamlessly switch D to ARC without creating all kinds of<br> havoc as people now how memory leaks where they didn&#39;t before. In order= <br> to support ARC the D language will necessarily have to grow/change to<br> accommodate it. Apple devs constantly have trouble with cyclic-refs to<br> this day.<br> </blockquote> <br></div> The stock response: weak pointers. But I think the best solution is to allo= w some form of automatic reference counting backed up by the GC, which will= lift cycles.</blockquote><div><br></div><div>I agree, I suggested that a f= ew times, and Adam ignored it each time.</div> <div>With this approach, performance/aggressive users would be able to disa= ble the backing GC, and deal with cycles manually, taking responsibility fo= r leaking themselves.</div><div>All other users would be able to ignore the= problem and have it collected by the backing GC, business as usual.</div> <div><br></div><div>The advantage here is *choice*. Users would then have a= fully automated system and/or have quite articulate control over it&#39;s = influence on their app.</div></div></div></div> --089e013c5eb634bffa04f18c4823--
Feb 03 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu <turkeyman gmail.com> 
 wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to 
 D, gets
 pretty much ignored. The topic is "Smart pointers instead 
 of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC! Sadly, although written as hyperbole, I feel that the above is fairly close to the actual position of the ARC crowd.

Don't be a dick. I get the impression you don't actually read my posts. And I also feel like you're a lot more dogmatic about this than you think I am. I'm absolutely fine with GC in most applications, I really couldn't give any shits if most people want a GC. I'm not dogmatic about it, and I've **honestly** tried to love the GC for years now. What I'm concerned about is that I have _no option_ to use D uninhibited when I need to not have the GC. These are the problems: * GC stalls for long periods time at completely un-predictable moments. * GC stalls become longer *and* more frequent as memory becomes less available, and the working pool becomes larger (what a coincidence). * Memory footprint is unknowable, what if you don't have a virtual memory manager? What if your total memory is measured in megabytes? * It's not possible to know when destruction of an object will happen, which has known workarounds (like in C#) but is also annoying in many cases, and supports the prior point. Conclusion: GC is unfit for embedded systems. One of the most significant remaining and compelling uses for a native systems language. The only realistic path I am aware of is to use ARC, which IS a form of GC, and allows a lot more flexibility in the front-end. GC forces one very particular paradigm upon you. ARC is a GC, but it has some complex properties __which can be addressed in various ways__. Unlike a GC which is entirely inflexible. You're not happy with ARC's cleaning objects up on the spot? Something that many people WANT, but I understand zero cleanup times in the running context is in other occasions a strength of GC; fine, just stick the pointer on a dead list, and free it either later during idle time, or on another thread. On the contrary, I haven't heard any proposal for a GC that would allow it to operate in carefully controlled time-slices, or strictly during idle-time. Cycles are a problem with ARC? True, how much effort are you willing to spend to mitigate the problem? None: run a secondary GC in the background to collect cycles (yes, there is still a GC, but it has much less work to do). Some: Disable background GC, manually require user specified weak references and stuff. Note: A user-preferred combination of the 2 could severely mitigate the workload of the background GC if it is still desired to handle some complex situations, or user errors. Are there any other disadvantages to ARC? I don't know of them if there are. Is far as I can tell, an ARC collector could provide identical convenience as the existing GC for anyone that simply doesn't care. It would also seem that it could provide significantly more options and control for those that do. I am _yet to hear anyone present a realistic path forwards using any form of GC_, so what else do I have to go with? Until I know of any other path forward, I'll stand behind the only one I can see. You're just repeating "I don't care about something that a significant subset of D developers do care about, and I don't think any changes should be made to support them". As far as I know, a switch to ARC could be done in a way that 'regular' users don't lose anything, or even notice... why is that so offensive?

I am not trying to be a dick. But I do feel like a small number of people are trying to gang up on me for daring to point out that the solution they've proposed solution might have bigger problems for other people than they care to admit. You still haven't dealt with the cyclic reference problem in ARC. There is absolutely no way ARC can handle that without programmer input, therefore, it is simply not possible to switch D to ARC without adding some language support to deal with cyclic-refs. Ergo, it is simply not possible to seamlessly switch D to ARC without creating all kinds of havoc as people now how memory leaks where they didn't before. In order to support ARC the D language will necessarily have to grow/change to accommodate it. Apple devs constantly have trouble with cyclic-refs to this day. I am not against supporting ARC side-by-side with the GC (I'm actually quite for it, I would love the flexibility), but it is unrealistic to make ARC the default option in D as that would subtly break all existing D code, something that Walter has point-blank refused to do in much smaller easier to find+fix cases. You can't grep for a weak-ref. So if that is what you are asking for, then yes, it will never happen in D. Also, I don't think you've fully considered what the perf penalty actually is for a *good* ARC implementation. I just leafed through the P-Code in the GC Handbook for their ARC implementation, it's about 4x longer than what their best P-Code Mark-Sweep implementation is. I would also like to point out that the GC Handbook points out six scientifically confirmed problems with ARC. (See Page 59) 1. RC imposes a time overhead on mutators in order to manipulate the counter. 2. Both the counter manipulation and pointer load/store operations MUST be atomic to prevent races. 3. Naive RC turns read ops into store ops to update the count. 4. No RC can reclaim cyclic data structures, which are much more common than is typically understood. [Bacon and Rajan 2001] 5. Counter must be the same size as the pointer, which can result in significant overhead for small objects. 6. RC can still pause. When the last head to a large pointer structure is deleted, RC MUST delete each descendant node. Note that these are paraphrases of the book, not me talking. And these apply equally to ARC and vanilla RC. Boehm demonstrated in one of his papers (2004) that thread-safe ARC may even lead to longer maximum pause times than a standard Tracing GC.

Most of us know and understand the issues with ARC and that with a GC. Many of us have seen how they play out in systems level development. There is a good reason all serious driver and embedded development is done in C/C++. A language is the compiler+std as one unit. If Phobos depends on the GC, D depends on the GC. If Phobos isn't systems level ready, D isn't systems level ready. I've heard arguments here that you can turn off the GC, but that equates to rewriting functions that already exists in Phobos and not using any third-party library. Why would anyone seriously consider that as an option? Embedded C++ has std:: and third-party libraries where memory is under control? Realistically D as a systems language isn't even at the hobby stage. I understand people are working in this area, as am I. Eagerly testing and trying to contribute where possible. That's the fun part of D, watching and helping it mature into everything it can be. Cheers, ed
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c2019af0fa3704f18c78be
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 13:31, Manu <turkeyman gmail.com> wrote:

 On 4 February 2014 12:57, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 5:36 PM, Adam Wilson wrote:

 You still haven't dealt with the cyclic reference problem in ARC. There
 is absolutely no way ARC can handle that without programmer input,
 therefore, it is simply not possible to switch D to ARC without adding
 some language support to deal with cyclic-refs. Ergo, it is simply not
 possible to seamlessly switch D to ARC without creating all kinds of
 havoc as people now how memory leaks where they didn't before. In order
 to support ARC the D language will necessarily have to grow/change to
 accommodate it. Apple devs constantly have trouble with cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles.

I agree, I suggested that a few times, and Adam ignored it each time. With this approach, performance/aggressive users would be able to disable the backing GC, and deal with cycles manually, taking responsibility for leaking themselves. All other users would be able to ignore the problem and have it collected by the backing GC, business as usual. The advantage here is *choice*. Users would then have a fully automated system and/or have quite articulate control over it's influence on their app.

Oh yeah, and the important detail, that it would apply to phobos and other libraries. The massive generally un-discussed problem here, is that libraries will almost always use whatever is default and/or most convenient. I have no control over the library code I link; if I can't influence the memory collection routine running behind libraries I'm using, there's a good chance I simply can't use that library. I tend to think if we had a GC backed ARC system, then phobos would be required to support 'gc-disabled' usage, ie, handle weak references internally (I don't think this is too much to ask for the standard library), making it useful to the 'gc-disabled' crowd, but the GC would remain collecting cycles and such for all the regular D users, who don't want to worry about such troubles. I think the majority of library authors would then follow the phobos standard of handling weak references manually, which would result in those libraries not being excluded from realtime use. The majority of trivial allocations don't produce cycles; closures, strings, temporary arrays and working sets. These are the allocations that do occur regularly. Major allocations of complex runtime data that may contain cycles don't happen at runtime in realtime apps, but these little trivial/temporary allocations do (and should). Since that sort of transient data doesn't run the risk of cycles, then it will reliably clean up quickly, safely, and immediately under ARC, even with backing GC disabled. --001a11c2019af0fa3704f18c78be Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 13:31, Manu <span dir=3D"ltr">&lt;<a href=3D"mailto:turkeyma= n gmail.com" target=3D"_blank">turkeyman gmail.com</a>&gt;</span> wrote:<br=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1=

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
<div class=3D"h5">On 4 February 2014 12:57, Andrei Alexandrescu <span dir=

ank">SeeWebsiteForEmail erdani.org</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div>On 2/3/14, 5:36 PM, Adam Wilson wrote:<= br> </div><div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor= der-left:1px #ccc solid;padding-left:1ex"> You still haven&#39;t dealt with the cyclic reference problem in ARC. There= <br> is absolutely no way ARC can handle that without programmer input,<br> therefore, it is simply not possible to switch D to ARC without adding<br> some language support to deal with cyclic-refs. Ergo, it is simply not<br> possible to seamlessly switch D to ARC without creating all kinds of<br> havoc as people now how memory leaks where they didn&#39;t before. In order= <br> to support ARC the D language will necessarily have to grow/change to<br> accommodate it. Apple devs constantly have trouble with cyclic-refs to<br> this day.<br> </blockquote> <br></div> The stock response: weak pointers. But I think the best solution is to allo= w some form of automatic reference counting backed up by the GC, which will= lift cycles.</blockquote><div><br></div></div></div><div>I agree, I sugges= ted that a few times, and Adam ignored it each time.</div> <div>With this approach, performance/aggressive users would be able to disa= ble the backing GC, and deal with cycles manually, taking responsibility fo= r leaking themselves.</div><div>All other users would be able to ignore the= problem and have it collected by the backing GC, business as usual.</div> <div><br></div><div>The advantage here is *choice*. Users would then have a= fully automated system and/or have quite articulate control over it&#39;s = influence on their app.</div></div></div></div> </blockquote></div><br></div><div class=3D"gmail_extra">Oh yeah, and the im= portant detail, that it would apply to phobos and other libraries.</div><di= v class=3D"gmail_extra">The massive generally un-discussed problem here, is= that libraries will almost always use whatever is default and/or most conv= enient. I have no control over the library code I link; if I can&#39;t infl= uence the memory collection routine running behind libraries I&#39;m using,= there&#39;s a good chance I simply can&#39;t use that library.</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">I tend to t= hink if we had a GC backed ARC system, then phobos would be required to sup= port &#39;gc-disabled&#39; usage, ie, handle weak references internally (I = don&#39;t think this is too much to ask for the standard library), making i= t useful to the &#39;gc-disabled&#39; crowd, but the GC would remain collec= ting cycles and such for all the regular D users, who don&#39;t want to wor= ry about such troubles. I think the majority of library authors would then = follow the phobos standard of handling weak references manually, which woul= d result in those libraries not being excluded from realtime use.</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">The majorit= y of trivial allocations don&#39;t produce cycles; closures, strings, tempo= rary arrays and working sets. These are the allocations that do occur regul= arly. Major allocations of complex runtime data that may contain cycles don= &#39;t happen at runtime in realtime apps, but these little trivial/tempora= ry allocations do (and should). Since that sort of transient data doesn&#39= ;t run the risk of cycles, then it will reliably clean up quickly, safely, = and immediately under ARC, even with backing GC disabled.</div> </div> --001a11c2019af0fa3704f18c78be--
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c2e064c1e2ae04f18d0f1e
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 14:19, Michel Fortin <michel.fortin michelf.ca> wrote:

 On 2014-02-04 03:45:33 +0000, Manu <turkeyman gmail.com> said:

  The majority of trivial allocations don't produce cycles; closures,
 strings, temporary arrays and working sets.

Oh, no. Beware of closures. That's a frequent problem in Objective-C ARC, even for those who understand ARC well. You have to be very careful of closures creating cycles if you use them as callbacks. For instance, you have a view that has a pointer to the model and sets a callback for the model to call when something change to update the view; that's a cycle and you need to use a weak ref to the view within the closure. Pretty common pattern.

Ah right. Interesting. It sounds like this could be addressed easily though by the API that manages the 'event' handler. If you use a raw delegate, maybe it's a problem, but it sounds like an event-programming construct, and that usually requires an event class that can handle multiple subscribers, as soon as that exists, it's easy to hide the weak reference behind the event api... --001a11c2e064c1e2ae04f18d0f1e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 14:19, Michel Fortin <span dir=3D"ltr">&lt;<a href=3D"mailto= :michel.fortin michelf.ca" target=3D"_blank">michel.fortin michelf.ca</a>&g= t;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On 2014-02-04 03:45:33 +00= 00, Manu &lt;<a href=3D"mailto:turkeyman gmail.com" target=3D"_blank">turke= yman gmail.com</a>&gt; said:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The majority of trivial allocations don&#39;t produce cycles; closures,<br> strings, temporary arrays and working sets.<br> </blockquote> <br></div> Oh, no. Beware of closures. That&#39;s a frequent problem in Objective-C AR= C, even for those who understand ARC well. You have to be very careful of c= losures creating cycles if you use them as callbacks. For instance, you hav= e a view that has a pointer to the model and sets a callback for the model = to call when something change to update the view; that&#39;s a cycle and yo= u need to use a weak ref to the view within the closure. Pretty common patt= ern.</blockquote> <div><br></div><div>Ah right. Interesting.</div><div>It sounds like this co= uld be addressed easily though by the API that manages the &#39;event&#39; = handler. If you use a raw delegate, maybe it&#39;s a problem, but it sounds= like an event-programming construct, and that usually requires an event cl= ass that can handle multiple subscribers, as soon as that exists, it&#39;s = easy to hide the weak reference behind the event api...</div> </div></div></div> --001a11c2e064c1e2ae04f18d0f1e--
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c30946615d0c04f18d164d
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 14:27, Manu <turkeyman gmail.com> wrote:

 On 4 February 2014 14:19, Michel Fortin <michel.fortin michelf.ca> wrote:

 On 2014-02-04 03:45:33 +0000, Manu <turkeyman gmail.com> said:

  The majority of trivial allocations don't produce cycles; closures,
 strings, temporary arrays and working sets.

Oh, no. Beware of closures. That's a frequent problem in Objective-C ARC, even for those who understand ARC well. You have to be very careful of closures creating cycles if you use them as callbacks. For instance, you have a view that has a pointer to the model and sets a callback for the model to call when something change to update the view; that's a cycle and you need to use a weak ref to the view within the closure. Pretty common pattern.

Ah right. Interesting. It sounds like this could be addressed easily though by the API that manages the 'event' handler. If you use a raw delegate, maybe it's a problem, but it sounds like an event-programming construct, and that usually requires an event class that can handle multiple subscribers, as soon as that exists, it's easy to hide the weak reference behind the event api...

I can say that all the closures that I've observed in my D apps so far have been entirely trivial. --001a11c30946615d0c04f18d164d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 14:27, Manu <span dir=3D"ltr">&lt;<a href=3D"mailto:turkeyma= n gmail.com" target=3D"_blank">turkeyman gmail.com</a>&gt;</span> wrote:<br=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1=

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 14:19, Michel Fortin <span dir=3D"ltr">&lt;<a href=3D"mailto= :michel.fortin michelf.ca" target=3D"_blank">michel.fortin michelf.ca</a>&g= t;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div>On 2014-02-04 03:45:33 +0000, Manu &lt;= <a href=3D"mailto:turkeyman gmail.com" target=3D"_blank">turkeyman gmail.co= m</a>&gt; said:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The majority of trivial allocations don&#39;t produce cycles; closures,<br> strings, temporary arrays and working sets.<br> </blockquote> <br></div> Oh, no. Beware of closures. That&#39;s a frequent problem in Objective-C AR= C, even for those who understand ARC well. You have to be very careful of c= losures creating cycles if you use them as callbacks. For instance, you hav= e a view that has a pointer to the model and sets a callback for the model = to call when something change to update the view; that&#39;s a cycle and yo= u need to use a weak ref to the view within the closure. Pretty common patt= ern.</blockquote> <div><br></div><div>Ah right. Interesting.</div><div>It sounds like this co= uld be addressed easily though by the API that manages the &#39;event&#39; = handler. If you use a raw delegate, maybe it&#39;s a problem, but it sounds= like an event-programming construct, and that usually requires an event cl= ass that can handle multiple subscribers, as soon as that exists, it&#39;s = easy to hide the weak reference behind the event api...</div> </div></div></div> </blockquote></div><br></div><div class=3D"gmail_extra">I can say that all = the closures that I&#39;ve observed in my D apps so far have been entirely = trivial.</div></div> --001a11c30946615d0c04f18d164d--
Feb 03 2014
prev sibling next sibling parent Jerry <jlquinn optonline.net> writes:
Manu <turkeyman gmail.com> writes:

 These are the problems:
  * GC stalls for long periods time at completely un-predictable moments.
  * GC stalls become longer *and* more frequent as memory becomes less
 available, and the working pool becomes larger (what a coincidence).
  * Memory footprint is unknowable, what if you don't have a virtual memory
 manager? What if your total memory is measured in megabytes?
  * It's not possible to know when destruction of an object will happen, which
 has known workarounds (like in C#) but is also annoying in many cases, and
 supports the prior point.

 Conclusion:
   GC is unfit for embedded systems. One of the most significant remaining and
 compelling uses for a native systems language.

Hi Manu, Doing a bit of searching, I came across the Metronome GC that IBM came up with. It appears to try to address the problems you raise: http://researcher.watson.ibm.com/researcher/view_project_subpage.php?id=175 Any thoughts? Jerry
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 18:57:00 -0800, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 5:36 PM, Adam Wilson wrote:
 You still haven't dealt with the cyclic reference problem in ARC. There
 is absolutely no way ARC can handle that without programmer input,
 therefore, it is simply not possible to switch D to ARC without adding
 some language support to deal with cyclic-refs. Ergo, it is simply not
 possible to seamlessly switch D to ARC without creating all kinds of
 havoc as people now how memory leaks where they didn't before. In order
 to support ARC the D language will necessarily have to grow/change to
 accommodate it. Apple devs constantly have trouble with cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles. Andrei

The immediate problem that I can see here is you're now paying for TWO GC algorithms. There is no traditional GC without a Mark phase (unless it's a copying collector, which will scare off the Embedded guys), and the mark phase is actually typically the longer portion of the pause. If you have ARC backed up by a GC you'll still have to mark+collect which means the GC still has to track ARC memory and then when a collection is needed, mark and collect. This means that you might reduce the total number of pauses, but you won't eliminate them. That in turn makes it an invalid tool for RT/Embedded purposes. And of course we still have the costs of ARC. Manu still can't rely on pause-free (although ARC isn't either) memory management, and the embedded guys still have to pay the costs in heap size to support the GC. Going the other way, GC is default with ARC support on the side, is not as troublesome from an implementation standpoint because the GC does not have to be taught about the ARC memory. This means that ARC memory is free of being tracked by the GC and the GC has less overall memory to track which makes collection cycles faster. However, I don't think that the RT/Embedded guys will like this either, because it means you are still paying for the GC at some point, and they'll never know for sure if a library they are using is going to GC-allocate (and collect) when they don't expect it. The only way I can see to make the ARC crowd happy is to completely replace the GC entirely, along with the attendant language changes (new keywords, etc) that are probably along the lines of Rust. I strongly believe that the reason we've never seen a GC backed ARC system is because in practice it doesn't completely solve any of the problems with either system but costs quite a bit more than either system on it's own. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0160c35e32a96704f18e03f5
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 15:22, Jerry <jlquinn optonline.net> wrote:

 Manu <turkeyman gmail.com> writes:

 These are the problems:
  * GC stalls for long periods time at completely un-predictable moments.
  * GC stalls become longer *and* more frequent as memory becomes less
 available, and the working pool becomes larger (what a coincidence).
  * Memory footprint is unknowable, what if you don't have a virtual

 manager? What if your total memory is measured in megabytes?
  * It's not possible to know when destruction of an object will happen,

 has known workarounds (like in C#) but is also annoying in many cases,

 supports the prior point.

 Conclusion:
   GC is unfit for embedded systems. One of the most significant

 compelling uses for a native systems language.

Hi Manu, Doing a bit of searching, I came across the Metronome GC that IBM came up with. It appears to try to address the problems you raise: http://researcher.watson.ibm.com/researcher/view_project_subpage.php?id=175 Any thoughts?

Interesting. I'll read :) I know nothing about it, so I can't really comment. And I'm also not an authority on what can/can't be done in terms of GC implementation in the context of D. --089e0160c35e32a96704f18e03f5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 15:22, Jerry <span dir=3D"ltr">&lt;<a href=3D"mailto:jlquinn= optonline.net" target=3D"_blank">jlquinn optonline.net</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> Manu &lt;<a href=3D"mailto:turkeyman gmail.com">turkeyman gmail.com</a>&gt;= writes:<br> <br> &gt; These are the problems:<br> &gt; =C2=A0* GC stalls for long periods time at completely un-predictable m= oments.<br> &gt; =C2=A0* GC stalls become longer *and* more frequent as memory becomes = less<br> &gt; available, and the working pool becomes larger (what a coincidence).<b= r> &gt; =C2=A0* Memory footprint is unknowable, what if you don&#39;t have a v= irtual memory<br> &gt; manager? What if your total memory is measured in megabytes?<br> &gt; =C2=A0* It&#39;s not possible to know when destruction of an object wi= ll happen, which<br> &gt; has known workarounds (like in C#) but is also annoying in many cases,= and<br> &gt; supports the prior point.<br> &gt;<br> &gt; Conclusion:<br> &gt; =C2=A0 GC is unfit for embedded systems. One of the most significant r= emaining and<br> &gt; compelling uses for a native systems language.<br> <br> Hi Manu,<br> <br> Doing a bit of searching, I came across the Metronome GC that IBM came<br> up with. =C2=A0It appears to try to address the problems you raise:<br> <br> <a href=3D"http://researcher.watson.ibm.com/researcher/view_project_subpage= .php?id=3D175" target=3D"_blank">http://researcher.watson.ibm.com/researche= r/view_project_subpage.php?id=3D175</a><br> <br> Any thoughts?<br></blockquote><div><br></div><div>Interesting. I&#39;ll rea= d :)</div><div>I know nothing about it, so I can&#39;t really comment. And = I&#39;m also not an authority on what can/can&#39;t be done in terms of GC = implementation in the context of D.</div> </div></div></div> --089e0160c35e32a96704f18e03f5--
Feb 03 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11330964bc697804f18e8577
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 15:23, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 18:57:00 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

  On 2/3/14, 5:36 PM, Adam Wilson wrote:
 You still haven't dealt with the cyclic reference problem in ARC. There
 is absolutely no way ARC can handle that without programmer input,
 therefore, it is simply not possible to switch D to ARC without adding
 some language support to deal with cyclic-refs. Ergo, it is simply not
 possible to seamlessly switch D to ARC without creating all kinds of
 havoc as people now how memory leaks where they didn't before. In order
 to support ARC the D language will necessarily have to grow/change to
 accommodate it. Apple devs constantly have trouble with cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles. Andrei

algorithms. There is no traditional GC without a Mark phase (unless it's a copying collector, which will scare off the Embedded guys), and the mark phase is actually typically the longer portion of the pause. If you have ARC backed up by a GC you'll still have to mark+collect which means the GC still has to track ARC memory and then when a collection is needed, mark and collect. This means that you might reduce the total number of pauses, but you won't eliminate them. That in turn makes it an invalid tool for RT/Embedded purposes. And of course we still have the costs of ARC. Manu still can't rely on pause-free (although ARC isn't either) memory management, and the embedded guys still have to pay the costs in heap size to support the GC.

So, the way I see this working in general, is that because in the majority case, ARC would release memory immediately freeing up memory regularly, an alloc that would have usually triggered a collect will happen far, far less often. Practically, this means that the mark phase, which you say is the longest phase, would be performed far less often. For me and my kind, I think the typical approach would be to turn off the backing GC, and rely on marking weak references correctly. This satisfies my requirements, and I also lose nothing in terms of facilities in Phobos or other libraries (assuming that those libraries have also marked weak references correctly, which I expect phobos would absolutely be required to do). This serves both worlds nicely, I retain access to libraries since they use the same allocator, the GC remains (and is run less often) for those that want care-free memory management, and for RT/embedded users, they can *practically* disable the GC, and take responsibility for weak references themselves, which I'm happy to do. Going the other way, GC is default with ARC support on the side, is not as
 troublesome from an implementation standpoint because the GC does not have
 to be taught about the ARC memory. This means that ARC memory is free of
 being tracked by the GC and the GC has less overall memory to track which
 makes collection cycles faster. However, I don't think that the RT/Embedded
 guys will like this either, because it means you are still paying for the
 GC at some point, and they'll never know for sure if a library they are
 using is going to GC-allocate (and collect) when they don't expect it.

It also means that phobos and other libraries will use the GC because it's the default. Correct, I don't see this as a valid solution. In fact, I don't see it as a solution at all. Where would implicit allocations like strings, concatenations, closures be allocated? I might as well just use RefCounted, I don't see this offering anything much more than that. The only way I can see to make the ARC crowd happy is to completely replace
 the GC entirely, along with the attendant language changes (new keywords,
 etc) that are probably along the lines of Rust. I strongly believe that the
 reason we've never seen a GC backed ARC system is because in practice it
 doesn't completely solve any of the problems with either system but costs
 quite a bit more than either system on it's own.

Really? [refer to my first paragraph in the reply] It seems to me like ARC in front of a GC would result in the GC running far less collect cycles. And the ARC opposition would be absolved of having to tediously mark weak references. Also, the GC opposition can turn the GC off, and everything will still work (assuming they take care of their cycles). I don't really see the disadvantage here, except that the only-GC-at-all-costs-I-won't-even-consider-ARC crowd would gain a ref-count, but they would also gain the advantage where the GC would run less collect cycles. That would probably balance out. I'm certainly it would be better than what we have, and in theory, everyone would be satisfied. --001a11330964bc697804f18e8577 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 15:23, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> On Mon, 03 Feb 2014 18:57:00 -0800, Andrei Alexandrescu &lt;<a href=3D"mail= to:SeeWebsiteForEmail erdani.org" target=3D"_blank">SeeWebsiteForEmail erda= ni.org</a><u></u>&gt; wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On 2/3/14, 5:36 PM, Adam Wilson wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> You still haven&#39;t dealt with the cyclic reference problem in ARC. There= <br> is absolutely no way ARC can handle that without programmer input,<br> therefore, it is simply not possible to switch D to ARC without adding<br> some language support to deal with cyclic-refs. Ergo, it is simply not<br> possible to seamlessly switch D to ARC without creating all kinds of<br> havoc as people now how memory leaks where they didn&#39;t before. In order= <br> to support ARC the D language will necessarily have to grow/change to<br> accommodate it. Apple devs constantly have trouble with cyclic-refs to<br> this day.<br> </blockquote> <br> The stock response: weak pointers. But I think the best solution is to allo= w some form of automatic reference counting backed up by the GC, which will= lift cycles.<br> <br> Andrei<br> <br> </blockquote> <br> The immediate problem that I can see here is you&#39;re now paying for TWO = GC algorithms. There is no traditional GC without a Mark phase (unless it&#= 39;s a copying collector, which will scare off the Embedded guys), and the = mark phase is actually typically the longer portion of the pause. If you ha= ve ARC backed up by a GC you&#39;ll still have to mark+collect which means = the GC still has to track ARC memory and then when a collection is needed, = mark and collect. This means that you might reduce the total number of paus= es, but you won&#39;t eliminate them. That in turn makes it an invalid tool= for RT/Embedded purposes. And of course we still have the costs of ARC. Ma= nu still can&#39;t rely on pause-free (although ARC isn&#39;t either) memor= y management, and the embedded guys still have to pay the costs in heap siz= e to support the GC.<br> </blockquote><div><br></div><div>So, the way I see this working in general,= is that because in the majority case, ARC would release memory immediately= freeing up memory regularly, an alloc that would have usually triggered a = collect will happen far, far less often.</div> <div>Practically, this means that the mark phase, which you say is the long= est phase, would be performed far less often.</div><div><br></div><div>For = me and my kind, I think the typical approach would be to turn off the backi= ng GC, and rely on marking weak references correctly.</div> <div>This satisfies my requirements, and I also lose nothing in terms of fa= cilities in Phobos or other libraries (assuming that those libraries have a= lso marked weak references correctly, which I expect phobos would absolutel= y be required to do).</div> <div><br></div><div>This serves both worlds nicely, I retain access to libr= aries since they use the same allocator, the GC remains (and is run less of= ten) for those that want care-free memory management, and for RT/embedded u= sers, they can *practically* disable the GC, and take responsibility for we= ak references themselves, which I&#39;m happy to do.</div> <div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Going the other way, GC is default with ARC support on the side, is not as = troublesome from an implementation standpoint because the GC does not have = to be taught about the ARC memory. This means that ARC memory is free of be= ing tracked by the GC and the GC has less overall memory to track which mak= es collection cycles faster. However, I don&#39;t think that the RT/Embedde= d guys will like this either, because it means you are still paying for the= GC at some point, and they&#39;ll never know for sure if a library they ar= e using is going to GC-allocate (and collect) when they don&#39;t expect it= .<br> </blockquote><div><br></div><div>It also means that phobos and other librar= ies will use the GC because it&#39;s the default. Correct, I don&#39;t see = this as a valid solution. In fact, I don&#39;t see it as a solution at all.= </div> <div>Where would implicit allocations like strings, concatenations, closure= s be allocated?</div><div>I might as well just use RefCounted, I don&#39;t = see this offering anything much more than that.</div><div><br></div><blockq= uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"> The only way I can see to make the ARC crowd happy is to completely replace= the GC entirely, along with the attendant language changes (new keywords, = etc) that are probably along the lines of Rust. I strongly believe that the= reason we&#39;ve never seen a GC backed ARC system is because in practice = it doesn&#39;t completely solve any of the problems with either system but = costs quite a bit more than either system on it&#39;s own.</blockquote> <div><br></div><div>Really? [refer to my first paragraph in the reply]</div=
<div>It seems to me like ARC in front of a GC would result in the GC runni=

ing to tediously mark weak references. Also, the GC opposition can turn the= GC off, and everything will still work (assuming they take care of their c= ycles).</div> <div>I don&#39;t really see the disadvantage here, except that the only-GC-= at-all-costs-I-won&#39;t-even-consider-ARC crowd would gain a ref-count, bu= t they would also gain the advantage where the GC would run less collect cy= cles. That would probably balance out.</div> <div><br></div><div>I&#39;m certainly it would be better than what we have,= and in theory, everyone would be satisfied.</div></div></div></div> --001a11330964bc697804f18e8577--
Feb 03 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 01:13:09 UTC, Manu wrote:
 On 4 February 2014 06:52, Adam Wilson <flyboynw gmail.com> 
 wrote:


GC's can
 actually be.

Also, neither languages are systems languages, or practical/appropriate on embedded, or memory limited systems. What is your argument? That D not be a systems language? Additionally, if you are going to beat that drum, you need to prove that either languages GC technology is even possible in the context of D, otherwise it's a red herring.

Well there are quite a few systems with an usable GC it seems, Singularity research OS http://research.microsoft.com/en-us/projects/Singularity/ Netduino hardware boards http://www.netduino.com/netduino/ Cortex M3 boards with Oberon-07 http://www.astrobe.com/boards.htm MirageOS http://www.openmirage.org/ -- Paulo
Feb 03 2014
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Mon, 03 Feb 2014 22:12:18 -0800, Manu <turkeyman gmail.com> wrote:

 On 4 February 2014 15:23, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 18:57:00 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

  On 2/3/14, 5:36 PM, Adam Wilson wrote:
 You still haven't dealt with the cyclic reference problem in ARC.  
 There
 is absolutely no way ARC can handle that without programmer input,
 therefore, it is simply not possible to switch D to ARC without adding
 some language support to deal with cyclic-refs. Ergo, it is simply not
 possible to seamlessly switch D to ARC without creating all kinds of
 havoc as people now how memory leaks where they didn't before. In  
 order
 to support ARC the D language will necessarily have to grow/change to
 accommodate it. Apple devs constantly have trouble with cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles. Andrei

GC algorithms. There is no traditional GC without a Mark phase (unless it's a copying collector, which will scare off the Embedded guys), and the mark phase is actually typically the longer portion of the pause. If you have ARC backed up by a GC you'll still have to mark+collect which means the GC still has to track ARC memory and then when a collection is needed, mark and collect. This means that you might reduce the total number of pauses, but you won't eliminate them. That in turn makes it an invalid tool for RT/Embedded purposes. And of course we still have the costs of ARC. Manu still can't rely on pause-free (although ARC isn't either) memory management, and the embedded guys still have to pay the costs in heap size to support the GC.

So, the way I see this working in general, is that because in the majority case, ARC would release memory immediately freeing up memory regularly, an alloc that would have usually triggered a collect will happen far, far less often. Practically, this means that the mark phase, which you say is the longest phase, would be performed far less often.

Well, if you want the ARC memory to share the heap with the GC the ARC memory will need to be tracked and marked by the GC. Otherwise the GC might try to allocate over the top of ARC memory and vice versa. This means that every time you run a collection you're still marking all ARC+GC memory, that will induce a pause. And the GC will still STW-collect on random allocations, and it will still have to Mark all ARC memory to make sure it's still valid. So yes, there will be fewer pauses, but they will still be there.
 For me and my kind, I think the typical approach would be to turn off the
 backing GC, and rely on marking weak references correctly.
 This satisfies my requirements, and I also lose nothing in terms of
 facilities in Phobos or other libraries (assuming that those libraries  
 have
 also marked weak references correctly, which I expect phobos would
 absolutely be required to do).

 This serves both worlds nicely, I retain access to libraries since they  
 use
 the same allocator, the GC remains (and is run less often) for those that
 want care-free memory management, and for RT/embedded users, they can
 *practically* disable the GC, and take responsibility for weak references
 themselves, which I'm happy to do.


 Going the other way, GC is default with ARC support on the side, is not  
 as
 troublesome from an implementation standpoint because the GC does not  
 have
 to be taught about the ARC memory. This means that ARC memory is free of
 being tracked by the GC and the GC has less overall memory to track  
 which
 makes collection cycles faster. However, I don't think that the  
 RT/Embedded
 guys will like this either, because it means you are still paying for  
 the
 GC at some point, and they'll never know for sure if a library they are
 using is going to GC-allocate (and collect) when they don't expect it.

It also means that phobos and other libraries will use the GC because it's the default. Correct, I don't see this as a valid solution. In fact, I don't see it as a solution at all. Where would implicit allocations like strings, concatenations, closures be allocated? I might as well just use RefCounted, I don't see this offering anything much more than that. The only way I can see to make the ARC crowd happy is to completely replace
 the GC entirely, along with the attendant language changes (new  
 keywords,
 etc) that are probably along the lines of Rust. I strongly believe that  
 the
 reason we've never seen a GC backed ARC system is because in practice it
 doesn't completely solve any of the problems with either system but  
 costs
 quite a bit more than either system on it's own.

Really? [refer to my first paragraph in the reply] It seems to me like ARC in front of a GC would result in the GC running far less collect cycles. And the ARC opposition would be absolved of having to tediously mark weak references. Also, the GC opposition can turn the GC off, and everything will still work (assuming they take care of their cycles). I don't really see the disadvantage here, except that the only-GC-at-all-costs-I-won't-even-consider-ARC crowd would gain a ref-count, but they would also gain the advantage where the GC would run less collect cycles. That would probably balance out. I'm certainly it would be better than what we have, and in theory, everyone would be satisfied.

I'm not convinced. Mostly, because it's not likely going to be good news for the GC crowd. First, now there are two GC algos running unpredictably at different times, so while you *might* experience a perf win in ARC-only mode, we'll probably pay for it in GC-backed ARC mode, because you still have the chance at non-deterministic pause lengths with ARC and you have the GC pauses, and they happen at different times (GC pause on allocate, ARC pause on delete). Each individual pause length *might* be shorter, but there is no guarantee of that, but you end up paying more time on the whole than you would otherwise, remembering that with the GC on, the slow part of the collection has to be performed on all memory, not just the GC memory. So yes you might delete a bit less, but you're marking just as much, and you've still got those pesky ARC pauses to deal with. And in basically everything but games you measure time spent on resource management as a portion of CPU cycles over time, not time spent per frame. That ref-count you hand-wave can actually cost quite a lot. Implementing ARC properly can have some serious perf implications on pointer-ops and count-ops due to the need to make sure that everything is performed atomically. And since this is a compiler thing, you can't say "Don't atomically operate here because I will never do anything that might race." because the compiler has to assume that at some point you will and the compiler cannot detect which mode it needs, or if a library will ruin your day. The only way you could get around this is with yet more syntax hints for the compiler like ' notatomic'. Very quickly ARC starts needing a lot of specialized syntax to make it work efficiently. And that's not good for "Modern Convenience". However, you don't have to perform everything atomically with a GC as the collect phase can always be performed concurrently on a separate thread and in most cases, the mark phase can do stop-the-thread instead of stop-the-world and in some cases, it will never stop anything at all. That can very easily result in pause times that are less than ARC on average. So now I've got a marginal improvement in the speed of ARC over the GC at best, and I still haven't paid for the GC. And if we disable the GC to get the speed back we now require that everyone on the team learns the specialized rules and syntax for cyclic-refs. That might be relatively easy for someone coming from C++, but it will be difficult to teach someone coming from C#/Java, which is statistically the more likely person to come to D. And indeed would be more than enough to stop my company moving to D. I've seen you say more than once that you can't bond with the GC, and believe me I understand, if you search back through the forums, you'll find one of the first things I did when I got here was complain about the GC. But what you're saying is "I can't bond with this horrible GC so we need to throw it out and rebuild the compiler to support ARC." All I am saying is "I can't bond with the horrible GC, so why don't we make a better one, that doesn't ruin responsiveness, because I've seen it done in other places and there is no technical reason D can't do the same, or better." Now that I've started to read the GC Handbook I am starting to suspect that using D, there might be a way to create a completely pause-less GC. Don't hold me too it, I don't know enough yet, but D has some unique capabilities that might just make it possible. -- Adam Wilson GitHub/IRC: LightBender Aurora Project Coordinator
Feb 03 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 02:57:00 UTC, Andrei Alexandrescu 
wrote:
 On 2/3/14, 5:36 PM, Adam Wilson wrote:
 You still haven't dealt with the cyclic reference problem in 
 ARC. There
 is absolutely no way ARC can handle that without programmer 
 input,
 therefore, it is simply not possible to switch D to ARC 
 without adding
 some language support to deal with cyclic-refs. Ergo, it is 
 simply not
 possible to seamlessly switch D to ARC without creating all 
 kinds of
 havoc as people now how memory leaks where they didn't before. 
 In order
 to support ARC the D language will necessarily have to 
 grow/change to
 accommodate it. Apple devs constantly have trouble with 
 cyclic-refs to
 this day.

The stock response: weak pointers. But I think the best solution is to allow some form of automatic reference counting backed up by the GC, which will lift cycles. Andrei

This is the approach taken by Cedar on the Mesa system. "On Adding Garbage Collection and Runtime Types to a Strongly-Typed, Statically-Checked, Concurrent Language" http://www.textfiles.com/bitsavers/pdf/xerox/parc/techReports/CSL-84-7_On_Adding_Garbage_Collection_and_Runtime_Types_to_a_Strongly-Typed_Statically-Checked_Concurrent_Language.pdf -- Paulo
Feb 04 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 00:15:34 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 4 February 2014 at 00:08:28 UTC, Frank Bauer wrote:
 How cool would that be: a language with the memory allocation 
 features of Rust but with stronger syntax and semantics roots 
 in the C / C++ world?

It is really tempting to write a new parser for Rust, isn't it? :-)

Us ML folks don't see a need for it.
Feb 04 2014
prev sibling next sibling parent "Don" <x nospam.com> writes:
On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu <turkeyman gmail.com> 
 wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers 
 to D, gets
 pretty much ignored. The topic is "Smart pointers instead 
 of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC!




 Most of us know and understand the issues with ARC and that 
 with a GC. Many of us have seen how they play out in systems 
 level development. There is a good reason all serious driver 
 and embedded development is done in C/C++.

 A language is the compiler+std as one unit. If Phobos depends 
 on the GC, D depends on the GC. If Phobos isn't systems level 
 ready, D isn't systems level ready. I've heard arguments here 
 that you can turn off the GC, but that equates to rewriting 
 functions that already exists in Phobos and not using any 
 third-party library.

At Sociomantic, that is exactly what we have done. Phobos is almost completely unusable at the present time. I personally don't think that ARC would make much difference. The problem is that *far* too much garbage is being created. And it's completely unnecessary in most cases. To take an extreme example, even a pauseless, perfect GC, wouldn't make std.json acceptable.
 Why would anyone seriously consider that as an option? Embedded 
 C++ has std:: and third-party libraries where memory is under 
 control?

 Realistically D as a systems language isn't even at the hobby 
 stage.

We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.
Feb 04 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 4 February 2014 at 09:59:07 UTC, Don wrote:
 On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu 
 <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers 
 to D, gets
 pretty much ignored. The topic is "Smart pointers instead 
 of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC!




 Most of us know and understand the issues with ARC and that 
 with a GC. Many of us have seen how they play out in systems 
 level development. There is a good reason all serious driver 
 and embedded development is done in C/C++.

 A language is the compiler+std as one unit. If Phobos depends 
 on the GC, D depends on the GC. If Phobos isn't systems level 
 ready, D isn't systems level ready. I've heard arguments here 
 that you can turn off the GC, but that equates to rewriting 
 functions that already exists in Phobos and not using any 
 third-party library.

At Sociomantic, that is exactly what we have done. Phobos is almost completely unusable at the present time. I personally don't think that ARC would make much difference. The problem is that *far* too much garbage is being created. And it's completely unnecessary in most cases. To take an extreme example, even a pauseless, perfect GC, wouldn't make std.json acceptable.
 Why would anyone seriously consider that as an option? 
 Embedded C++ has std:: and third-party libraries where memory 
 is under control?

 Realistically D as a systems language isn't even at the hobby 
 stage.

We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.

OK, I stand corrected! :D Thanks, ed
Feb 04 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 4 February 2014 at 09:59:07 UTC, Don wrote:
 On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu 
 <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers 
 to D, gets
 pretty much ignored. The topic is "Smart pointers instead 
 of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC!




 Most of us know and understand the issues with ARC and that 
 with a GC. Many of us have seen how they play out in systems 
 level development. There is a good reason all serious driver 
 and embedded development is done in C/C++.

 A language is the compiler+std as one unit. If Phobos depends 
 on the GC, D depends on the GC. If Phobos isn't systems level 
 ready, D isn't systems level ready. I've heard arguments here 
 that you can turn off the GC, but that equates to rewriting 
 functions that already exists in Phobos and not using any 
 third-party library.

At Sociomantic, that is exactly what we have done. Phobos is almost completely unusable at the present time. I personally don't think that ARC would make much difference. The problem is that *far* too much garbage is being created. And it's completely unnecessary in most cases. To take an extreme example, even a pauseless, perfect GC, wouldn't make std.json acceptable.
 Why would anyone seriously consider that as an option? 
 Embedded C++ has std:: and third-party libraries where memory 
 is under control?

 Realistically D as a systems language isn't even at the hobby 
 stage.

We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.

Maybe I'm thinking too much embedded, which I admit isn't fair on D and at this stage maybe not a realistic comparison. Our projects are Siemens medical devices so it is 90% embedded, a different level of system perhaps. They too would be on a global scale and I'd love to get D on them :-) Cheers, ed
Feb 04 2014
prev sibling next sibling parent "Don" <x nospam.com> writes:
On Tuesday, 4 February 2014 at 10:32:26 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 09:59:07 UTC, Don wrote:
 On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson 
 wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu 
 <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers 
 to D, gets
 pretty much ignored. The topic is "Smart pointers 
 instead of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC!




 Most of us know and understand the issues with ARC and that 
 with a GC. Many of us have seen how they play out in systems 
 level development. There is a good reason all serious driver 
 and embedded development is done in C/C++.

 A language is the compiler+std as one unit. If Phobos depends 
 on the GC, D depends on the GC. If Phobos isn't systems level 
 ready, D isn't systems level ready. I've heard arguments here 
 that you can turn off the GC, but that equates to rewriting 
 functions that already exists in Phobos and not using any 
 third-party library.

At Sociomantic, that is exactly what we have done. Phobos is almost completely unusable at the present time. I personally don't think that ARC would make much difference. The problem is that *far* too much garbage is being created. And it's completely unnecessary in most cases. To take an extreme example, even a pauseless, perfect GC, wouldn't make std.json acceptable.
 Why would anyone seriously consider that as an option? 
 Embedded C++ has std:: and third-party libraries where memory 
 is under control?

 Realistically D as a systems language isn't even at the hobby 
 stage.

We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.

Maybe I'm thinking too much embedded, which I admit isn't fair on D and at this stage maybe not a realistic comparison.

Yeah, I dunno what "systems language" means really. In practice it seems to mean "competes with C++" and that's how I use it. And C++ had some problems getting into the embedded market. Though even D as "a better C" is a surprisingly nice language.
 Our projects are Siemens medical devices so it is 90% embedded, 
 a different level of system perhaps. They too would be on a 
 global scale and I'd love to get D on them :-)

Yeah. I can imagine that's a tough assignment.
Feb 04 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 11:13:03 UTC, Don wrote:
 On Tuesday, 4 February 2014 at 10:32:26 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 09:59:07 UTC, Don wrote:
 On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 01:36:09 UTC, Adam Wilson 
 wrote:
 On Mon, 03 Feb 2014 17:04:08 -0800, Manu 
 <turkeyman gmail.com> wrote:

 On 4 February 2014 06:21, Adam Wilson <flyboynw gmail.com> 
 wrote:

 On Mon, 03 Feb 2014 12:02:29 -0800, Andrei Alexandrescu <
 SeeWebsiteForEmail erdani.org> wrote:

 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning 
 pointers to D, gets
 pretty much ignored. The topic is "Smart pointers 
 instead of GC?",
 remember? People here seem to be more interested in 
 diverting to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

complete removal of the GC and replacement with compiler generated ARC will be unacceptable to a certain, highly vocal, subset of D users. No arguments can be made to otherwise, regardless of validity. As far as they are concerned the discussion of ARC vs. GC is closed and decided. ARC is the only path forward to the bright and glorious future of D. ARC most efficiently solves all memory management problems ever encountered. Peer-Reviewed Research and the Scientific Method be damned! ALL HAIL ARC!




 Most of us know and understand the issues with ARC and that 
 with a GC. Many of us have seen how they play out in systems 
 level development. There is a good reason all serious driver 
 and embedded development is done in C/C++.

 A language is the compiler+std as one unit. If Phobos 
 depends on the GC, D depends on the GC. If Phobos isn't 
 systems level ready, D isn't systems level ready. I've heard 
 arguments here that you can turn off the GC, but that 
 equates to rewriting functions that already exists in Phobos 
 and not using any third-party library.

At Sociomantic, that is exactly what we have done. Phobos is almost completely unusable at the present time. I personally don't think that ARC would make much difference. The problem is that *far* too much garbage is being created. And it's completely unnecessary in most cases. To take an extreme example, even a pauseless, perfect GC, wouldn't make std.json acceptable.
 Why would anyone seriously consider that as an option? 
 Embedded C++ has std:: and third-party libraries where 
 memory is under control?

 Realistically D as a systems language isn't even at the 
 hobby stage.

We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.

Maybe I'm thinking too much embedded, which I admit isn't fair on D and at this stage maybe not a realistic comparison.

Yeah, I dunno what "systems language" means really.

For me it means you can write an OS with it, even if some tiny parts require the use of Assembly glue. -- Paulo
Feb 04 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Monday, 3 February 2014 at 20:02:30 UTC, Andrei Alexandrescu
wrote:
 On 2/3/14, 6:57 AM, Frank Bauer wrote:
 Anyone asking for the addition of ARC or owning pointers to D, 
 gets
 pretty much ignored. The topic is "Smart pointers instead of 
 GC?",
 remember? People here seem to be more interested in diverting 
 to
 nullable, scope and GC optimization. Telling, indeed.

I thought I made it clear that GC avoidance (which includes considering built-in reference counting) is a major focus of 2014. Andrei

I hope the same mistakes are not repeated of the past: This means that whatever is chosen, removing the hard dependence on any automatic memory management should be avoided. It would be nice to be able have D work in all areas of programming. I think allowing one to turn off all automatic memory management, or even allowing one to choose which allocation method to use per function/class/module get about 99.99 of the use cases. You allow people that want the freedom of now worrying about deallocation to use the AGC and those that need every drop of performance can go the manual route. Maybe such a general approach is difficult to implement? But surely it would be worth it. From this thread, it's obvious that there are plenty of people interested in its success. Getting D runtime off any specific memory allocation scheme would seem to be the priority? Then Phobos? then we party like it's 1999? It would be nice if one could simply write some allocator, drop it into D, and everything work out dandy. e.g., I want to try out a new super fast AGC like metronome GC, I write the code for it, tell D to use it, and then reap the benefits.
Feb 04 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Tuesday, 4 February 2014 at 12:03:31 UTC, Frustrated wrote:
[snip]
 It would be nice if one could simply write some allocator, drop
 it into D, and everything work out dandy. e.g., I want to try 
 out
 a new super fast AGC like metronome GC, I write the code for it,
 tell D to use it, and then reap the benefits.

You can write your own GC now and drop it in, we did as an experiment. If you're talking about a dropping in an allocator that is another matter, an allocator is not a GC. Cheers, ed
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 11:57:25 UTC, Paulo Pinto wrote:
 On Tuesday, 4 February 2014 at 11:13:03 UTC, Don wrote:

 Yeah, I dunno what "systems language" means really.

For me it means you can write an OS with it, even if some tiny parts require the use of Assembly glue.

Pretty close to my definition, but I would add: - you should be able to write a performant OS with it - you should be able to use all core language features when doing so - the mapping to hardware should be obvious - thus the runtime should be transparent/easy to grasp Basically I think a system level language should provide a minimal library and runtime that you can extend to a OS-level library that you use for implementing all services on your OS. I think GC+Phobos should be an extension of such a minimal set up. --- I don't think ARC is needed for D, because I think you use RC only when needed when writing latency sensitive code and use primarily other strategies (unique pointers, embedded objects etc). I do think that interop with C++ is a good strategy. Language level support for C++11 pointer types would be a good addition now that they are being used quite extensively. Interop with C++ is more important than Objective-C. Having a system library core that is GC free would be nice. I think this should be done on the name space level. I.e. having one optimized "core" section that is 100% GC free and one more extensive "std" section that requires GC to be available. I don't think it is realistic to have all libraries being non-GC. It is better to have useful libraries that are somewhat inefficient than not having them due to complexity issue. It is better to have a very easy to use and extensible to XML DOM library than to have a more obfuscated and less extensible XML DOM library without GC. But the XML parser library should be GC free.
Feb 04 2014
prev sibling next sibling parent "Frustrated" <c1514843 drdrb.com> writes:
On Tuesday, 4 February 2014 at 12:18:22 UTC, ed wrote:
 On Tuesday, 4 February 2014 at 12:03:31 UTC, Frustrated wrote:
 [snip]
 It would be nice if one could simply write some allocator, drop
 it into D, and everything work out dandy. e.g., I want to try 
 out
 a new super fast AGC like metronome GC, I write the code for 
 it,
 tell D to use it, and then reap the benefits.

You can write your own GC now and drop it in, we did as an experiment. If you're talking about a dropping in an allocator that is another matter, an allocator is not a GC. Cheers, ed

Yes, you can do a lot with D, even rewrite D runtime to remove GC dependent stuff, or write your own Phobos library. It is not easy and D is not *designed* to do that. AGC is about automatic deallocation so you don't have to *free* the memory you allocate. To do this you have to allocate through the GC so it knows when you allocate. So a GC is an allocator. Regardless, I would simply like to have, as I'm sure many would, the ability to set how D handles memory. 1. No memory management at all - Fast as possible. No memory is ever free'ed. Similar to how DMD works. 2. Manual memory management - Pretty fast but old school. Predictable but error prone. Different allocating schemes for dealing with efficiency. Some apps require this to be performant. 3. Automatic memory management - Easy to use = Lazy mentality. Safer. Most apps would not need anything more than this. 4. Combination of the above - The hard part. How to make all this stuff plug and play, work well together, easy to use, etc... Right now, D basically only has #3. Some have worked D to use 2. std.allocators goes a long way in the right direction as does std.allocators.gc. All I'm wanting is a unified way to use it all and to get D and phobos off the hard dependence of the GC.
Feb 04 2014
prev sibling next sibling parent "Araq" <rumpf_a web.de> writes:
 I've seen you say more than once that you can't bond with the 
 GC, and believe me I understand, if you search back through the 
 forums, you'll find one of the first things I did when I got 
 here was complain about the GC. But what you're saying is "I 
 can't bond with this horrible GC so we need to throw it out and 
 rebuild the compiler to support ARC." All I am saying is "I 
 can't bond with the horrible GC, so why don't we make a better 
 one, that doesn't ruin responsiveness, because I've seen it 
 done in other places and there is no technical reason D can't 
 do the same, or better."

There are lots of technical reasons why D's GC is just the way it is and why Java and C# can do better. They have been enumerated in this forum a lot of times, you simply chose to ignore them: 1. D doesn't restrict pointers, interior pointers abound especially thanks to the design of slices. Interior pointers are a problem for every high performance GC algorithm I'm aware of. 2. D's design doesn't easily allow for read or write barriers in general. You simply cannot make a GC behave without barriers. 3. Unions cause some (minor) problems. In general the type system doesn't distinguish between GC'ed memory and manually managed memory at all.
 Now that I've started to read the GC Handbook I am starting to 
 suspect that using D, there might be a way to create a 
 completely pause-less GC. Don't hold me too it, I don't know 
 enough yet, but D has some unique capabilities that might just 
 make it possible.

That's just bullshit. The opposite is true: Of the big players D is easily the language that is most hostile to an efficient GC implementation.
Feb 04 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 4 February 2014 at 12:46:58 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 4 February 2014 at 11:57:25 UTC, Paulo Pinto wrote:
 On Tuesday, 4 February 2014 at 11:13:03 UTC, Don wrote:

 Yeah, I dunno what "systems language" means really.

For me it means you can write an OS with it, even if some tiny parts require the use of Assembly glue.

Pretty close to my definition, but I would add: - you should be able to write a performant OS with it - you should be able to use all core language features when doing so - the mapping to hardware should be obvious - thus the runtime should be transparent/easy to grasp Basically I think a system level language should provide a minimal library and runtime that you can extend to a OS-level library that you use for implementing all services on your OS. I think GC+Phobos should be an extension of such a minimal set up. --- I don't think ARC is needed for D, because I think you use RC only when needed when writing latency sensitive code and use primarily other strategies (unique pointers, embedded objects etc). I do think that interop with C++ is a good strategy. Language level support for C++11 pointer types would be a good addition now that they are being used quite extensively. Interop with C++ is more important than Objective-C. Having a system library core that is GC free would be nice. I think this should be done on the name space level. I.e. having one optimized "core" section that is 100% GC free and one more extensive "std" section that requires GC to be available. I don't think it is realistic to have all libraries being non-GC. It is better to have useful libraries that are somewhat inefficient than not having them due to complexity issue. It is better to have a very easy to use and extensible to XML DOM library than to have a more obfuscated and less extensible XML DOM library without GC. But the XML parser library should be GC free.

In Oberon and Modula-3's control over GC tends to be a bit easier than D's current design, because of a few points: - No implicit allocations outside NEW - No closures - You can use untraced pointers in unsafe modules. However it would be nice to know if anyone here has real life experience with Lisp Machines and how was their performance. -- Paulo
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 13:03:24 UTC, Araq wrote:
 1. D doesn't restrict pointers, interior pointers abound 
 especially thanks to the design of slices. Interior pointers 
 are a problem for every high performance GC algorithm I'm aware 
 of.

Interior pointers can always be replaced with a pointer + offset, so in the case of slices it can be resolved at the implementation level at some memory trade off? In the case of internal pointers elsewhere you would have to introduce a new type "pointer+offset", but that would break C-interop.
 2. D's design doesn't easily allow for read or write barriers 
 in general. You simply cannot make a GC behave without barriers.

It is difficult without affecting the C-compatible semantics. There are some interesting options on 64 bit CPUs where you in theory can do compaction to a fresh region and put hardware traps on the old region, rewriting pointers trying to access the old region or converting misses to exceptions on the fly (assuming you can have a barrier/isolate that prevents comparison of old/new pointers). Another obvious solution is thread local GC for thread local memory or segmented GC (collect for only a specific pool, refuse pointers to escape the pool, and put a lock on that pool during GC).
 That's just bullshit. The opposite is true: Of the big players 
 D is easily the language that is most hostile to an efficient 
 GC implementation.

Because C is hostile to GC. So you either forsake C-compatibility or stay inefficient?
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Tuesday, 4 February 2014 at 06:47:03 UTC, Walter Bright wrote:
 On 2/3/2014 7:03 PM, Adam Wilson wrote:
 Note that ObjC has special syntax to handle weak pointers. 
 It's not well
 understood by many.

Sounds like explicitly managed memory is hardly worse. On 2/3/2014 3:13 PM, woh wrote:
  Any system that forces a single way of handling memory as the 
 only viable
 method, be it GC( as D currently does)

This is incorrect. You can use malloc/free in D, as well as write & use your own allocators, and even write your own ref counted types. On 2/3/2014 1:42 PM, Shammah Chancellor wrote:
 It's also probably
 possible to create a drop-in replacement for the GC to do 
 something else.

It certainly is possible. There's nothing magic about the current GC, it's just library code.

Andrei Alexandrescu wrote:
 2. Work on Phobos to see what can be done about avoiding 
 unnecessary allocation. Most likely we'll need to also add a 
  nogc flag.
 ...
 4. Work on the core language and druntime to see how to 
 seamlessly accommodate alternate GC mechanisms such as 
 reference counting.
 ...
 I thought I made it clear that GC avoidance (which includes 
 considering built-in reference counting) is a major focus of 
 2014.

 Andrei

Im totally confused: Walter, do you back what Andrei says or do we have a good cop / bad cop situation here?
Feb 04 2014
prev sibling next sibling parent "Don" <x nospam.com> writes:
On Tuesday, 4 February 2014 at 11:57:25 UTC, Paulo Pinto wrote:
 On Tuesday, 4 February 2014 at 11:13:03 UTC, Don wrote:
 On Tuesday, 4 February 2014 at 10:32:26 UTC, ed wrote:
 Realistically D as a systems language isn't even at the 
 hobby stage.

We're using D as a systems language on a global commercial scale. And no, we don't use malloc/free. We just don't use Phobos.

Maybe I'm thinking too much embedded, which I admit isn't fair on D and at this stage maybe not a realistic comparison.

Yeah, I dunno what "systems language" means really.

For me it means you can write an OS with it, even if some tiny parts require the use of Assembly glue.

I know that's the *origin* of the term, but I don't think that's the *meaning*. I mean, that's a really obscure task. People almost never write a new OS, and languages have claimed to be systems languages without ever having done it. AFAICT what it really means is "like C" in some vaguely-defined sense.
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 13:52:38 UTC, Paulo Pinto wrote:
 - No implicit allocations outside NEW
 - No closures

True, on the OS level, in drivers and in real time call backs you want everything to be explicit.
 - You can use untraced pointers in unsafe modules.

I assume it would be possible to add notrace as an attribute to pointers/references in D so that you can avoid tracing objects that are known to be in the pool of traced objects? The best would be to have "notrace-pointer-types" and programmer specified "trace functions" that are more optimal. E.g. keeping a list of all active Nodes and have your own function for scanning it for outgoing pointers, then keep all pointers to Nodes non-traceable.
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 4 February 2014 at 02:51:21 UTC, Andrei Alexandrescu 
wrote:
   Rust seems to have found a very nice model, and even cpp with
 value/unique/rc/weak is IMO superior to what D currently offers

I think Rust expended too much language complexity on that one issue. Andrei

By the way, while this statement was true for initial design, they have recently moved to much more simple model, replacing most of more complicated pointer types with library solutions. I think those who refer to Rust example are more likely to have in mind that new model and your judgement seems to be based on previous one.
Feb 04 2014
prev sibling next sibling parent "Matthias Einwag" <matthias.einwag googlemail.com> writes:
 By the way, while this statement was true for initial design, 
 they have recently moved to much more simple model, replacing 
 most of more complicated pointer types with library solutions. 
 I think those who refer to Rust example are more likely to have 
 in mind that new model and your judgement seems to be based on 
 previous one.

immutable data in their Gc/Rc structures you end up with things like Rc<RefMut<Type>> or Rc<Cell<Type>> and must exactly understand what each of those does. For me already the library solution in C++ (shared_ptr<Type>) is somewhat annoying to type when you compare that to the current D solution or other managed languages. Dereferencing these types in Rust is also very hard at the moment, but they plan do get it easier. Freely mixing ARC and GC is also not that easy in my opinion: As soon as you have a garbage collected object anywhere in your object hierarchy basically anything beneath that will be also "garbage collected" - in the sense of only deallocated when the GC runs. And if you used RAII semantics in some of your ARC managed objects you might wonder why their resources are not immediatly releasd, but also when the GC runs.
Feb 04 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 03 Feb 2014 15:52:53 -0500, Adam Wilson <flyboynw gmail.com> wrote:

 That said, I firmly believe that wholesale replacement of the GC is  
 throwing the baby out with the bathwater. Effectively, the poor D GC  
 implementation has become an excuse to launch a crusade against all GC's  
 everywhere, never mind that Java and the .NET GC's are consistent  
 examples of just how good GC's can actually be.

AFAIK, Java's and .NET's GCs require precise collection. This is not possible in D (at least not fully). While I think we need to move more towards precise collection, we will never be as good as languages which have that restriction. Some GC options just aren't available. I think ARC is good for many (most?) applications, and GC is good for many as well. We should have an option of both. I still look forward to the day when I can write iPhone apps in D instead of objective-C :) -Steve
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 4 February 2014 at 09:59:07 UTC, Don wrote:
 We're using D as a systems language on a global commercial 
 scale. And no, we don't use malloc/free. We just don't use 
 Phobos.

With all respect I don't consider Sociomantic projects examples of systems-level programming (now that I have had a chance to inspect those closely ;)). Soft real-time performance critical system is a better description.
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 4 February 2014 at 18:19:56 UTC, Matthias Einwag 
wrote:
 In my opinion the new model is even harder. ... <skip>

I also find exact implementation considerably over-engineered but I would never propose to just copy stuff to Rust as-is. What I do propose to is to acknowledge general principle - build base language upon allocation-ignorant primitives that provide ownership semantics and move all advanced memory management to the library. I sincerely believe that `std .allocator` alone as presented by Andrei will give D a huge edge when exploring similar approach.
Feb 04 2014
prev sibling next sibling parent =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
 std.typecons.RefCounted!T

Does this work equally well with T being both a value objects (struct) and reference objects (class)?
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 20:29:18 UTC, John J wrote:
 On 02/01/2014 07:57 PM, Adam Wilson wrote:
 On TIOBE, there are three non-GC languages in the entire
 top 20 (C, C++, Obj-C) the rest feature a non-ARC GC of some 
 form.

Add "Pascal" and "Delphi/Object Pascal" to that.

And Python, which I believe primarily use ARC and then a weird GC of some form to catch cycles.
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 20:55:30 UTC, Ola Fosheim Grøstad 
wrote:
 And Python, which I believe primarily use ARC and then a weird 
 GC of some form to catch cycles.

Oh, and PHP too.
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 4 February 2014 at 20:57:28 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 4 February 2014 at 20:55:30 UTC, Ola Fosheim 
 Grøstad wrote:
 And Python, which I believe primarily use ARC and then a weird 
 GC of some form to catch cycles.

Oh, and PHP too.

Oh, and Perl 5…
Feb 04 2014
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Tuesday, 4 February 2014 at 20:53:50 UTC, Nordlöw wrote:
 std.typecons.RefCounted!T

Does this work equally well with T being both a value objects (struct) and reference objects (class)?

RefCounted doesn't support classes yet but that's just because nobody has taken the time to implement it. There also needs to be a WeakRef added in order to round out the smart pointer support. Once that's done Phobos will at least have parity with C++ (it already has Unique).
Feb 04 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 4 February 2014 at 22:30:39 UTC, Walter Bright wrote:
 you can write them twice  or use a template (except in a 
 virtual context), and in both cases
 you keep the efficiency of not checking for null when the 
 argument is not nullable.

That's just what I wish to avoid. Consider adding more pointer types - the combinatorics quickly explode. Heck, just have two pointer parameters, and you already have 4 cases. I wonder how Rust deals with this.

I think it is pretty much same thing as with using specialised types (i.e. SQLEscapedString) - your API explodes exponentially if you try to write code with methods/functions that support all of them but in practice you shouldn't. Same is here. Some functions should always deal with with nullable types and most - only with non-nullable. Tricky part is writing your code in such way that separation is clear but good language guidlines and examples can solve it. Also non-nullable types should be implicitly cast to nullable parameters so you don't always need to support all cases distinctively.
Feb 04 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 4 February 2014 at 22:30:39 UTC, Walter Bright wrote:
 I wonder how Rust deals with this.

The only time ownership matters is if you are going to store the pointer. It is like the difference between a container and a range. An algorithm doesn't need to know about the specifics of a container. Let's use average for example. We might write it in D: int average(InputRange)(InputRange r) { int count = 0; int sum; while(!r.empty) { count++; sum += r.front; r.popFront(); } return sum / count; } Now, this being a template, D will generate new code for a variety of types... but even if we replaced InputRange with a specific thing, let's call it int[], it is still usable by a variety of containers: int average(int[] r) { /* same impl */ } D has two containers built in that provide this range: int[50] staticArray; int[] dynamicArray = new int[](50); average(staticArray[]); // works average(dynamicArray); // works Pointers also offer this: int* pointer = cast(int*) malloc(50 * int.sizeof); average(pointer[0 .. 50]); Moreover, user-defined types can also provide this range: struct Numbers { int[] opSlice() { return [1,2,3]; } } Numbers numbers; average(numbers[]); // works In theory, we could provide either an inputRangeObject or a slice for linked lists, lazy generators, anything. One function, any kind of input. Of course, we could slice memory from any allocator. Heck, we saw three different allocations right here (with three different types! stack, gc, and malloc) all using the same function, without templating. I'm sure none of this is new to you... and this is basically how the rust thing works too. Our usage of int[] (or the input range) are borrowed pointers. Algorithms are written in their terms. The ownership type only matters when you store it. And turns out, this matters in D as well: struct ManualArray(T) { size_t length; T* data; this(size_t len) { data = malloc(T.sizeof * len); length = len; } ~this() { free(data); } T[] opSlice() { return data[0 .. length]; } disable this(this); // copying this is wrong, don't allow it! } void main() { auto array = ManualArray!int(50); average(array[]); // works, reusing our pointer } But, borrowed comes into play if we store it: int[] globalArray; void foo(int[] array) { globalArray = array; } void bar() { auto array = ManualArray!int(50); foo(array[]); // uh oh } void main() { bar(); globalArray[0] = 10; // crash likely, memory safety violated } Again, I'm sure none of this is new to you, but it illustrates owned vs borrowed: ManualArray is owned. Storing it is safe - it ensures its internal pointer is valid throughout its entire life time. But ManualArray.opSlice returns a borrowed reference. Great for algorithms or any processing that doesn't escape the reference. Anything that would be written in terms of an input range is probably correct with this. However, we stored the borrowed reference, which is a no-no. array went out of scope, freeing the memory, leaving the escaped borrowed reference in an invalid state. Let's say we did want to store it. There's a few options: we could make our own copy or store the pre-made copy. GC!(int[]) globalArray; void foo(GC!(int[]) array) { globalArray = array; } That's sane, the GC owns it and we specified that so storing it is cool. We could also take a RefCounted!(int[]), if that's how we wanted to store it. But let's say we wanted to store it with a different method. There's only two sane options: void foo(int[] array) { globalArray = array.dup; } Take a borrowed reference and make a copy of it. The function foo is in charge of allocating (here, we made a GC managed copy). OR, don't implement that and force the user to decide: void foo(GC!(int[]) array) {...} user: foo(ownedArray[]); // error, cannot implicitly convert int[] to GC!(int[]) int[50] stackArray; foo(stackArray[]); // error, cannot implicitly convert int[] to GC!int[] Now, the user makes the decision. It is going to be stored, the function signature says that up front by asking for a non-borrowed reference. They won't get a surprise crash when the globalArray later accesses stack or freed data. They have to deal with the error. They might not call the function, or they might do the .dup themselves. Either way, memory safety is preserved and inefficiencies are visible. So, a function that stores a reference would only ever come in one or two signatures, regardless of how many: 1) the exact match for the callee's allocation strategy. The callee, knowing what the strategy is, can also be sanely responsible for freeing it. (A struct dtor, for example, knows that its members are malloced and can thus call free) 2) A generic borrowed type, e.g. input range or slice, which it then makes a private copy of it internally. Since these are arguably hidden allocations you might not even like these. Calling .dup (or whatever) at the call sight keeps the allocations visible. So bottom line, you don't duplicate functions for the different types. You borrow references for processing (analogous to implementing algorithms with ranges) and own references for storing... which you need to know about, so only one type makes sense.
Feb 04 2014
prev sibling next sibling parent "woh" <wojw yahoo.com> writes:
On Tuesday, 4 February 2014 at 22:30:39 UTC, Walter Bright wrote:
 On 2/4/2014 4:23 AM, Michel Fortin wrote:
 For the rare cases where you actually want both versions to 
 work,

I think you're making a vast assumption that the case is rare. When I write utility functions, I want them to work on as wide a variety of inputs as possible. Otherwise, they are not very useful.
 you can write them twice  or use a template (except in a 
 virtual context), and in both cases
 you keep the efficiency of not checking for null when the 
 argument is not nullable.

That's just what I wish to avoid. Consider adding more pointer types - the combinatorics quickly explode. Heck, just have two pointer parameters, and you already have 4 cases. I wonder how Rust deals with this.
 In any case, I have yet to understand why  nullable as a 
 storage class would be
 any better. How do you solve that problem with a storage class?

Good question. I don't have an answer offhand.

rust would use a reference, basically a pointer that can come from any smart ptr type, and does not outlive its owner
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Tuesday, 4 February 2014 at 22:30:39 UTC, Walter Bright wrote:
 That's just what I wish to avoid. Consider adding more pointer 
 types - the combinatorics quickly explode. Heck, just have two 
 pointer parameters, and you already have 4 cases.

 I wonder how Rust deals with this.

Rust does not have null pointers. In cases where you want to mark an object of some type as having "no value", or being "null", you use a stdlib type Option<T>, which is basically a tagged union plus some static compiler guarantees: enum Option<T> { Some(T), None } An Option<T> either contains a T (tagged 'Some' then) or nothing (tagged 'None'). If you have a function that takes an Option<T> you can also pass it a (pure, non-nullable) T by specifying Some(myT) as the parameter.
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Regarding different pointer types in Rust as function arguments:

A function that takes a borrowed pointer &T can also be called 
with an owning pointer ~T, an RC pointer Rc<T>, or a GC pointer 
Gc<T>. They all convert neatly to a &T. One function to rule them 
... err .. accomodate all.
Feb 04 2014
prev sibling next sibling parent "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Sunday, 2 February 2014 at 16:55:35 UTC, Andrei Alexandrescu 
wrote:
 1. Add  nullable and provide a -nullable compiler flag to 
 verify it. The attribute is inferred locally and for white-box 
 functions (lambdas, templates), and required as annotation 
 otherwise. References not annotated with  nullable are 
 statically enforced to never be null.

 2. Work on Phobos to see what can be done about avoiding 
 unnecessary allocation. Most likely we'll need to also add a 
  nogc flag.

 3. Work on adding tracing capabilities to allocators and see 
 how to integrate them with the language and Phobos.

 4. Work on the core language and druntime to see how to 
 seamlessly accommodate alternate GC mechanisms such as 
 reference counting.

That looks nice, but one thing is missing: 5. Fix property already :P What is the last status about properties? As far as I understand pulls are in place and might get pick up in 2.066, right?
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Wednesday, 5 February 2014 at 00:50:55 UTC, Walter Bright 
wrote:
 Again, what happens with:

     T identity(T t) { return t; }

 ? I.e. the transference of the argument pointer type to the 
 return type?

As far as I see, in Rust the pointer type is lost on the return value as well, if your function takes a reference (borrowed pointer). But you can do: fn identity(x: &T) -> T { return *T; } fn main() { let a = ~T; let r = ~identity(a); } That is: pass by reference and return a value type. If you then pass an owned pointer ~T, you can directly construct the returned value in heap allocated memory ~identity(a) and assign the resulting owned pointer. The compiler is smart enough to not do unnecessary copies. That way the caller is responsible for the relationship between argument pointer type and result pointer type. With a single function, you can freely mix all combinations of argument and return types. E.g. pass an owned pointer and construct a Gc<T> from the returned result.
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
Sorry,

      return *x;

of course.
Feb 04 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a1136391097648404f19f03a9
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 17:50, Adam Wilson <flyboynw gmail.com> wrote:

 On Mon, 03 Feb 2014 22:12:18 -0800, Manu <turkeyman gmail.com> wrote:

 So, the way I see this working in general, is that because in the majority
 case, ARC would release memory immediately freeing up memory regularly, an
 alloc that would have usually triggered a collect will happen far, far
 less
 often.
 Practically, this means that the mark phase, which you say is the longest
 phase, would be performed far less often.

memory will need to be tracked and marked by the GC. Otherwise the GC might try to allocate over the top of ARC memory and vice versa. This means that every time you run a collection you're still marking all ARC+GC memory, that will induce a pause. And the GC will still STW-collect on random allocations, and it will still have to Mark all ARC memory to make sure it's still valid. So yes, there will be fewer pauses, but they will still be there.

I'm not bothered in the least. At that stage, I will have turned the GC off, and I'll handle weak references myself. The GC crowd are then welcome to go on and continue improving the GC in whatever way they plan to do so. For me and my kind, I think the typical approach would be to turn off the
 backing GC, and rely on marking weak references correctly.
 This satisfies my requirements, and I also lose nothing in terms of
 facilities in Phobos or other libraries (assuming that those libraries
 have
 also marked weak references correctly, which I expect phobos would
 absolutely be required to do).

 This serves both worlds nicely, I retain access to libraries since they
 use
 the same allocator, the GC remains (and is run less often) for those that
 want care-free memory management, and for RT/embedded users, they can
 *practically* disable the GC, and take responsibility for weak references
 themselves, which I'm happy to do.


 Going the other way, GC is default with ARC support on the side, is not as

 troublesome from an implementation standpoint because the GC does not
 have
 to be taught about the ARC memory. This means that ARC memory is free of
 being tracked by the GC and the GC has less overall memory to track which
 makes collection cycles faster. However, I don't think that the
 RT/Embedded
 guys will like this either, because it means you are still paying for the
 GC at some point, and they'll never know for sure if a library they are
 using is going to GC-allocate (and collect) when they don't expect it.

the default. Correct, I don't see this as a valid solution. In fact, I don't see it as a solution at all. Where would implicit allocations like strings, concatenations, closures be allocated? I might as well just use RefCounted, I don't see this offering anything much more than that. The only way I can see to make the ARC crowd happy is to completely replace
 the GC entirely, along with the attendant language changes (new keywords,
 etc) that are probably along the lines of Rust. I strongly believe that
 the
 reason we've never seen a GC backed ARC system is because in practice it
 doesn't completely solve any of the problems with either system but costs
 quite a bit more than either system on it's own.

Really? [refer to my first paragraph in the reply] It seems to me like ARC in front of a GC would result in the GC running far less collect cycles. And the ARC opposition would be absolved of having to tediously mark weak references. Also, the GC opposition can turn the GC off, and everything will still work (assuming they take care of their cycles). I don't really see the disadvantage here, except that the only-GC-at-all-costs-I-won't-even-consider-ARC crowd would gain a ref-count, but they would also gain the advantage where the GC would run less collect cycles. That would probably balance out. I'm certainly it would be better than what we have, and in theory, everyone would be satisfied.

I'm not convinced. Mostly, because it's not likely going to be good news for the GC crowd. First, now there are two GC algos running unpredictably at different times, so while you *might* experience a perf win in ARC-only mode, we'll probably pay for it in GC-backed ARC mode, because you still have the chance at non-deterministic pause lengths with ARC and you have the GC pauses, and they happen at different times (GC pause on allocate, ARC pause on delete).

I don't understand. How is ARC non-deterministic? It seems entirely deterministic to me. And if you want to, you can defer destruction to idle time if you fancy. Sure, the GC may pause from time to time, but you already have that anyway. In this case, it'll run much less often. Each individual pause length *might* be shorter, but there is no guarantee
 of that, but you end up paying more time on the whole than you would
 otherwise, remembering that with the GC on, the slow part of the collection
 has to be performed on all memory, not just the GC memory. So yes you might
 delete a bit less, but you're marking just as much, and you've still got
 those pesky ARC pauses to deal with.

Again, what ARC pauses? You mean object destruction time? Defer destruction if you find cleaning up on the spot to be expensive. GC will always have to scan all memory that is allocated. The fact that it's scanning ARC memory is precisely the point (catch cycles), and no additional cost in scan load, since that memory would all be GC memory anyway if ARC wasn't present. And in basically everything but games you measure time spent on resource
 management as a portion of CPU cycles over time, not time spent per frame.

I suspect that spending less time doing GC scan's will result in a win overall. I have nothing to back that up, but it's a strong suspicion. Object destruction, which you seem to have a problem with under ARC, still happens even with a GC, just at some unknown time. It's not clear to me what the additional ARC cost is (other than the obvious inc and dec)... except it facilitates spending less time doing GC collection, which is probably a significant saving. That ref-count you hand-wave can actually cost quite a lot. Implementing
 ARC properly can have some serious perf implications on pointer-ops and
 count-ops due to the need to make sure that everything is performed
 atomically.

I don't think this is true. There's no need to perform the ref fiddling with atomic operations unless it's shared. Everyone expects additional costs for synchronisation of shared objects. And since this is a compiler thing, you can't say "Don't atomically operate
 here because I will never do anything that might race." because the
 compiler has to assume that at some point you will and the compiler cannot
 detect which mode it needs, or if a library will ruin your day. The only
 way you could get around this is with yet more syntax hints for the
 compiler like ' notatomic'.

Ummm. I think D makes a clear assumption that if something isn't marked shared, that it doesn't have to compile code to protect against races. That's the whole point of making shared an explicit attribute. What you say is true in C++ which can't distinguish the cases, I don't think it applies in D. Very quickly ARC starts needing a lot of specialized syntax to make it work
 efficiently. And that's not good for "Modern Convenience".

Other than a weak attribute, what does it need? I'm not aware of anything else. However, you don't have to perform everything atomically with a GC as the
 collect phase can always be performed concurrently on a separate thread and
 in most cases,

You don't have to perform everything atomically in ARC, and the GC is definitely like that now. the mark phase can do stop-the-thread instead of stop-the-world and in some
 cases, it will never stop anything at all.

If I only have one core? ARC doesn't need to mark at all, that cost is effectively distributed among inc/dec ref's, and I'm confident ++ and -- operations performed only on relevant data and only when it's copied is much cheaper than the GC scanning the whole heap, and applying all that logic to determine what things are pointers that it needs to follow and what not. That can very easily result in pause times that are less than ARC on
 average. So now I've got a marginal improvement in the speed of ARC over
 the GC at best, and I still haven't paid for the GC.

What pause does the ARC produce? Are you paying an ambient cost for the GC? When the ARC is doing it's job, the GC wouldn't run. When too many un-handled cycles add up, the GC might run a scan. If/when the GC does run a scan, it's precisely because you _haven't_ already paid the cost via the ARC; it missed it due to cycle, no cost was paid, no time was lost, it was just collected by the GC instead. I don't see how you can sum the costs of the 2 collection mechanisms in this case. Either the ARC cleans it up, and the GC doesn't. Or the GC cleans it up because the ARC didn't. ARC destruction can easily be deferred, and unlike a mark phase which MUST be executed entirely in one step, it is possible to process deferred ARC object destruction at leisure, using only idle time for instance, and arbitrary lengths of time are easily supported. And if we disable the GC to get the speed back I still don't follow, we never lost anything, we only moved it. we now require that everyone on the team learns the specialized rules and
 syntax for cyclic-refs. That might be relatively easy for someone coming
 from C++, but it will be difficult to teach someone coming from C#/Java,
 which is statistically the more likely person to come to D. And indeed
 would be more than enough to stop my company moving to D.

Don't turn the GC off. Indeed, that would be silly for most applications. You need to clarify how moving some collection cost from the GC to ARC makes it more expensive? I still can't see it. As far as I can see, everything the ARC cleans up is burden lifted from the GC, it could only result in the GC running less often, and ARC is not by nature more expensive than GC. I suspect ARC has a lower net cost, since ++/--, only on relevant things, and only when they're copied, is probably a lot less complicated work than a full mark phase, which touches everything, and follows many pointers, mostly unnecessarily. GC mark phase is quite a large workload, increases proportionally to the working data set, and it's also an absolute dcache disaster. No way inc/dec could compare to that workload, particularly as the heap grows large or nears capacity. I've seen you say more than once that you can't bond with the GC, and
 believe me I understand, if you search back through the forums, you'll find
 one of the first things I did when I got here was complain about the GC.
 But what you're saying is "I can't bond with this horrible GC so we need to
 throw it out and rebuild the compiler to support ARC." All I am saying is
 "I can't bond with the horrible GC, so why don't we make a better one, that
 doesn't ruin responsiveness, because I've seen it done in other places and
 there is no technical reason D can't do the same, or better." Now that I've
 started to read the GC Handbook I am starting to suspect that using D,
 there might be a way to create a completely pause-less GC. Don't hold me
 too it, I don't know enough yet, but D has some unique capabilities that
 might just make it possible.

Well when you know, I'm all ears. Truly, I am. But I can't imagine a GC that will work acceptably in environments such as limited memory, realtime, single core, or various combinations of those. I also get the feeling you haven't thought through the cost distribution of a GC backed ARC solution. Either that, or I haven't done my math correctly, which I'd be happy to have demonstrated where I went wrong. --001a1136391097648404f19f03a9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 17:50, Adam Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:f= lyboynw gmail.com" target=3D"_blank">flyboynw gmail.com</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:s= olid;padding-left:1ex"> <div>On Mon, 03 Feb 2014 22:12:18 -0800, Manu &lt;<a href=3D"mailto:turkeym= an gmail.com" target=3D"_blank">turkeyman gmail.com</a>&gt; wrote:<br> </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:s= olid;padding-left:1ex"><div><br></div><div> So, the way I see this working in general, is that because in the majority<= br> case, ARC would release memory immediately freeing up memory regularly, an<= br> alloc that would have usually triggered a collect will happen far, far less= <br> often.<br> Practically, this means that the mark phase, which you say is the longest<b= r> phase, would be performed far less often.<br> <br> </div></blockquote> <br> Well, if you want the ARC memory to share the heap with the GC the ARC memo= ry will need to be tracked and marked by the GC. Otherwise the GC might try= to allocate over the top of ARC memory and vice versa. This means that eve= ry time you run a collection you&#39;re still marking all ARC+GC memory, th= at will induce a pause. And the GC will still STW-collect on random allocat= ions, and it will still have to Mark all ARC memory to make sure it&#39;s s= till valid. So yes, there will be fewer pauses, but they will still be ther= e.<br> </blockquote><div><br></div><div>I&#39;m not bothered in the least. At that= stage, I will have turned the GC off, and I&#39;ll handle weak references = myself. The GC crowd are then welcome to go on and continue improving the G= C in whatever way they plan to do so.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"><div> For me and my kind, I think the typical approach would be to turn off the<b= r> backing GC, and rely on marking weak references correctly.<br> This satisfies my requirements, and I also lose nothing in terms of<br> facilities in Phobos or other libraries (assuming that those libraries have= <br> also marked weak references correctly, which I expect phobos would<br> absolutely be required to do).<br> <br> This serves both worlds nicely, I retain access to libraries since they use= <br> the same allocator, the GC remains (and is run less often) for those that<b= r> want care-free memory management, and for RT/embedded users, they can<br> *practically* disable the GC, and take responsibility for weak references<b= r> themselves, which I&#39;m happy to do.<br> <br> <br></div><div> Going the other way, GC is default with ARC support on the side, is not as<= br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> troublesome from an implementation standpoint because the GC does not have<= br> to be taught about the ARC memory. This means that ARC memory is free of<br=

r> makes collection cycles faster. However, I don&#39;t think that the RT/Embe= dded<br> guys will like this either, because it means you are still paying for the<b= r> GC at some point, and they&#39;ll never know for sure if a library they are= <br> using is going to GC-allocate (and collect) when they don&#39;t expect it.<= br> <br> </blockquote> <br></div><div> It also means that phobos and other libraries will use the GC because it&#3= 9;s<br> the default. Correct, I don&#39;t see this as a valid solution. In fact, I<= br> don&#39;t see it as a solution at all.<br> Where would implicit allocations like strings, concatenations, closures be<= br> allocated?<br> I might as well just use RefCounted, I don&#39;t see this offering anything= <br> much more than that.<br> <br></div><div> The only way I can see to make the ARC crowd happy is to completely replace= <br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> the GC entirely, along with the attendant language changes (new keywords,<b= r> etc) that are probably along the lines of Rust. I strongly believe that the= <br> reason we&#39;ve never seen a GC backed ARC system is because in practice i= t<br> doesn&#39;t completely solve any of the problems with either system but cos= ts<br> quite a bit more than either system on it&#39;s own.<br> </blockquote> <br> <br></div><div> Really? [refer to my first paragraph in the reply]<br> It seems to me like ARC in front of a GC would result in the GC running far= <br> less collect cycles. And the ARC opposition would be absolved of having to<= br> tediously mark weak references. Also, the GC opposition can turn the GC<br> off, and everything will still work (assuming they take care of their<br> cycles).<br> I don&#39;t really see the disadvantage here, except that the<br> only-GC-at-all-costs-I-won&#39;t-<u></u>even-consider-ARC crowd would gain = a<br> ref-count, but they would also gain the advantage where the GC would run<br=

<br> I&#39;m certainly it would be better than what we have, and in theory, ever= yone<br> would be satisfied.<br> </div></blockquote> <br> I&#39;m not convinced. Mostly, because it&#39;s not likely going to be good= news for the GC crowd. First, now there are two GC algos running unpredict= ably at different times, so while you *might* experience a perf win in ARC-= only mode, we&#39;ll probably pay for it in GC-backed ARC mode, because you= still have the chance at non-deterministic pause lengths with ARC and you = have the GC pauses, and they happen at different times (GC pause on allocat= e, ARC pause on delete).</blockquote> <div><br></div><div>I don&#39;t understand. How is ARC non-deterministic? I= t seems entirely deterministic to me. And if you want to, you can defer des= truction to idle time if you fancy.</div><div>Sure, the GC may pause from t= ime to time, but you already have that anyway. In this case, it&#39;ll run = much less often.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> Each individual pause length *might* be sh= orter, but there is no guarantee of that, but you end up paying more time o= n the whole than you would otherwise, remembering that with the GC on, the = slow part of the collection has to be performed on all memory, not just the= GC memory. So yes you might delete a bit less, but you&#39;re marking just= as much, and you&#39;ve still got those pesky ARC pauses to deal with.</bl= ockquote> <div><br></div><div>Again, what ARC pauses? You mean object destruction tim= e? Defer destruction if you find cleaning up on the spot to be expensive. G= C will always have to scan all memory that is allocated. The fact that it&#= 39;s scanning ARC memory is precisely the point (catch cycles), and no addi= tional cost in scan load, since that memory would all be GC memory anyway i= f ARC wasn&#39;t present.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> And in basically everything but games you = measure time spent on resource management as a portion of CPU cycles over t= ime, not time spent per frame.<br> </blockquote><div><br></div><div>I suspect that spending less time doing GC= scan&#39;s will result in a win overall. I have nothing to back that up, b= ut it&#39;s a strong suspicion. Object destruction, which you seem to have = a problem with under ARC, still happens even with a GC, just at some unknow= n time. It&#39;s not clear to me what the additional ARC cost is (other tha= n the obvious inc and dec)... except it facilitates spending less time doin= g GC collection, which is probably a significant saving.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> That ref-count you hand-wave can actually cost quite a lot. Implementing AR= C properly can have some serious perf implications on pointer-ops and count= -ops due to the need to make sure that everything is performed atomically.<= /blockquote> <div><br></div><div>I don&#39;t think this is true. There&#39;s no need to = perform the ref fiddling with atomic operations unless it&#39;s shared.</di= v><div>Everyone expects additional costs for synchronisation of shared obje= cts.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> And since this is a compiler thing, you ca= n&#39;t say &quot;Don&#39;t atomically operate here because I will never do= anything that might race.&quot; because the compiler has to assume that at= some point you will and the compiler cannot detect which mode it needs, or= if a library will ruin your day. The only way you could get around this is= with yet more syntax hints for the compiler like &#39; notatomic&#39;.<br> </blockquote><div><br></div><div>Ummm. I think D makes a clear assumption t= hat if something isn&#39;t marked shared, that it doesn&#39;t have to compi= le code to protect against races.</div><div>That&#39;s the whole point of m= aking shared an explicit attribute.</div> <div>What you say is true in C++ which can&#39;t distinguish the cases, I d= on&#39;t think it applies in D.</div><div><br></div><blockquote class=3D"gm= ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-l= eft-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> Very quickly ARC starts needing a lot of specialized syntax to make it work= efficiently. And that&#39;s not good for &quot;Modern Convenience&quot;.<b= r></blockquote><div><br></div><div>Other than a weak attribute, what does i= t need? I&#39;m not aware of anything else.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> However, you don&#39;t have to perform everything atomically with a GC as t= he collect phase can always be performed concurrently on a separate thread = and in most cases,</blockquote><div><br></div><div>You don&#39;t have to pe= rform everything atomically in ARC, and the GC is definitely like that now.= </div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> the mark phase can do stop-the-thread inst= ead of stop-the-world and in some cases, it will never stop anything at all= .</blockquote> <div><br></div><div>If I only have one core?</div><div>ARC doesn&#39;t need= to mark at all, that cost is effectively distributed among inc/dec ref&#39= ;s, and I&#39;m confident ++ and -- operations performed only on relevant d= ata and only when it&#39;s copied is much cheaper than the GC scanning the = whole heap, and applying all that logic to determine what things are pointe= rs that it needs to follow and what not.<br> </div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px = 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);bord= er-left-style:solid;padding-left:1ex">That can very easily result in pause = times that are less than ARC on average. So now I&#39;ve got a marginal imp= rovement in the speed of ARC over the GC at best, and I still haven&#39;t p= aid for the GC.<br> </blockquote><div><br></div><div>What pause does the ARC produce?</div><div=
<br></div><div>Are you paying an ambient cost for the GC? When the ARC is =

s add up, the GC might run a scan. If/when the GC does run a scan, it&#39;s= precisely because you _haven&#39;t_ already paid the cost via the ARC; it = missed it due to cycle, no cost was paid, no time was lost, it was just col= lected by the GC instead. I don&#39;t see how you can sum the costs of the = 2 collection mechanisms in this case.</div> <div>Either the ARC cleans it up, and the GC doesn&#39;t. Or the GC cleans = it up because the ARC didn&#39;t.</div><div><br></div><div>ARC destruction = can easily be deferred, and unlike a mark phase which MUST be executed enti= rely in one step, it is possible to process deferred ARC object destruction= at leisure, using only idle time for instance, and arbitrary lengths of ti= me are easily supported.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> And if we disable the GC to get the speed back</blockquote><div><br></div><= div>I still don&#39;t follow, we never lost anything, we only moved it.</di= v><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px = 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-l= eft-style:solid;padding-left:1ex"> we now require that everyone on the team learns the specialized rules and = syntax for cyclic-refs. That might be relatively easy for someone coming fr= om C++, but it will be difficult to teach someone coming from C#/Java, whic= h is statistically the more likely person to come to D. And indeed would be= more than enough to stop my company moving to D.<br> </blockquote><div><br></div><div>Don&#39;t turn the GC off. Indeed, that wo= uld be silly for most applications.</div><div>You need to clarify how movin= g some collection cost from the GC to ARC makes it more expensive? I still = can&#39;t see it. As far as I can see, everything the ARC cleans up is burd= en lifted from the GC, it could only result in the GC running less often, a= nd ARC is not by nature more expensive than GC. I suspect ARC has a lower n= et cost, since ++/--, only on relevant things, and only when they&#39;re co= pied, is probably a lot less complicated work than a full mark phase, which= touches everything, and follows many pointers, mostly unnecessarily. GC ma= rk phase is quite a large workload, increases proportionally to the working= data set, and it&#39;s also an absolute dcache disaster. No way inc/dec co= uld compare to that workload, particularly as the heap grows large or nears= capacity.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef= t-style:solid;padding-left:1ex"> I&#39;ve seen you say more than once that you can&#39;t bond with the GC, a= nd believe me I understand, if you search back through the forums, you&#39;= ll find one of the first things I did when I got here was complain about th= e GC. But what you&#39;re saying is &quot;I can&#39;t bond with this horrib= le GC so we need to throw it out and rebuild the compiler to support ARC.&q= uot; All I am saying is &quot;I can&#39;t bond with the horrible GC, so why= don&#39;t we make a better one, that doesn&#39;t ruin responsiveness, beca= use I&#39;ve seen it done in other places and there is no technical reason = D can&#39;t do the same, or better.&quot; Now that I&#39;ve started to read= the GC Handbook I am starting to suspect that using D, there might be a wa= y to create a completely pause-less GC. Don&#39;t hold me too it, I don&#39= ;t know enough yet, but D has some unique capabilities that might just make= it possible.</blockquote> <div><br></div><div>Well when you know, I&#39;m all ears. Truly, I am. But = I can&#39;t imagine a GC that will work acceptably in environments such as = limited memory, realtime, single core, or various combinations of those.</d= iv> <div>I also get the feeling you haven&#39;t thought through the cost distri= bution of a GC backed ARC solution. Either that, or I haven&#39;t done my m= ath correctly, which I&#39;d be happy to have demonstrated where I went wro= ng.</div> </div></div></div> --001a1136391097648404f19f03a9--
Feb 04 2014
prev sibling next sibling parent "Frank Bauer" <y z.com> writes:
On Wednesday, 5 February 2014 at 01:34:59 UTC, Frank Bauer wrote:
     let a = ~T;
     let r = ~identity(a);

The equivalent in D for a GC pointer could be: a = new T; r = new identity(a); or, while we're at it: a2 = newOwn T; r2 = newOwn identity(a2); a3 = newARC T; r3 = newARC identity(a3); or even: a4 = new T; r4 = newOwn identity(a4); ;)
Feb 04 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a1136a76a7cbaaa04f19f1c36
Content-Type: text/plain; charset=UTF-8

On 4 February 2014 19:59, Don <x nospam.com> wrote:

 On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:

 Most of us know and understand the issues with ARC and that with a GC.
 Many of us have seen how they play out in systems level development. There
 is a good reason all serious driver and embedded development is done in
 C/C++.

 A language is the compiler+std as one unit. If Phobos depends on the GC,
 D depends on the GC. If Phobos isn't systems level ready, D isn't systems
 level ready. I've heard arguments here that you can turn off the GC, but
 that equates to rewriting functions that already exists in Phobos and not
 using any third-party library.

At Sociomantic, that is exactly what we have done. Phobos is almost completely unusable at the present time. I personally don't think that ARC would make much difference. The problem is that *far* too much garbage is being created. And it's completely unnecessary in most cases.

I agree here. A new collector can't distract from the task of reducing the amount of garbage produced in the first place. I've never suggested ARC will make a wild difference in terms of performance in the standard use case, that's not the point (although I do imagine it would be faster). I'm saying that ARC is not fundamentally incompatible with many kinds of workloads, and offers the application a level of flexibility that's not available under a GC alone. It's an enabler for some whole new industries to use D with confidence. --001a1136a76a7cbaaa04f19f1c36 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 4= February 2014 19:59, Don <span dir=3D"ltr">&lt;<a href=3D"mailto:x nospam.= com" target=3D"_blank">x nospam.com</a>&gt;</span> wrote:<br><blockquote cl= ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p= adding-left:1ex"> On Tuesday, 4 February 2014 at 03:43:53 UTC, ed wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><br></blockquote> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> Most of us know and understand the issues with ARC and that with a GC. Many= of us have seen how they play out in systems level development. There is a= good reason all serious driver and embedded development is done in C/C++.<= br> <br> A language is the compiler+std as one unit. If Phobos depends on the GC, D = depends on the GC. If Phobos isn&#39;t systems level ready, D isn&#39;t sys= tems level ready. I&#39;ve heard arguments here that you can turn off the G= C, but that equates to rewriting functions that already exists in Phobos an= d not using any third-party library.<br> </blockquote> <br> At Sociomantic, that is exactly what we have done. Phobos is almost complet= ely unusable at the present time.<br> <br> I personally don&#39;t think that ARC would make much difference. The probl= em is that *far* too much garbage is being created. And it&#39;s completely= unnecessary in most cases.<br></blockquote><div><br></div><div>I agree her= e. A new collector can&#39;t distract from the task of reducing the amount = of garbage produced in the first place.</div> <div><br></div><div>I&#39;ve never suggested ARC will make a wild differenc= e in terms of performance in the standard use case, that&#39;s not the poin= t (although I do imagine it would be faster).</div><div>I&#39;m saying that= ARC is not fundamentally incompatible with many kinds of workloads, and of= fers the application a level of flexibility that&#39;s not available under = a GC alone. It&#39;s an enabler for some whole new industries to use D with= confidence.</div> </div></div></div> --001a1136a76a7cbaaa04f19f1c36--
Feb 04 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 5 February 2014 at 02:00:00 UTC, Manu wrote:
 I'm saying that ARC is not fundamentally incompatible with many 
 kinds of
 workloads, and offers the application a level of flexibility 
 that's not
 available under a GC alone. It's an enabler for some whole new 
 industries
 to use D with confidence.

That's true, ARC will improve latencies. There is a reason for why languages like Python, Perl and Php up till now have been primarily ARC based, but are moving more towards GC as CPU speed has increased. The freeze-the-world time GC provides was unacceptable 15-20 years ago, even for non-realtime stuff. It is also true that ARC+GC will make the GC run very seldom, perhaps never, depending on the thresholds you set up and the patterns you use.
Feb 04 2014
prev sibling next sibling parent "Meta" <jared771 gmail.com> writes:
On Wednesday, 5 February 2014 at 00:50:55 UTC, Walter Bright 
wrote:
 On 2/4/2014 4:07 PM, Frank Bauer wrote:
 Regarding different pointer types in Rust as function 
 arguments:

 A function that takes a borrowed pointer &T can also be called 
 with an owning
 pointer ~T, an RC pointer Rc<T>, or a GC pointer Gc<T>. They 
 all convert neatly
 to a &T. One function to rule them ... err .. accomodate all.

Again, what happens with: T identity(T t) { return t; } ? I.e. the transference of the argument pointer type to the return type?

The compiler already infers safe nothrow pure on template functions. Couldn't it do the same for nullable?
Feb 04 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 5 February 2014 at 01:16:47 UTC, Francesco 
Cattoglio wrote:
 5. Fix  property already :P

yes, we should be able to do this with a minimum amount of pain and it'd be soo nice to have.
Feb 04 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 5 February 2014 at 00:47:27 UTC, Walter Bright 
wrote:
 The difficulty comes when needing to transfer the nullability 
 of the argument to the nullability of the return type. Implicit 
 conversions don't help with that.

 See inout.

You could always do it with a template; T identity(T t) { return t;} will literally work. (I'm pretty sure it would also work with const and immutable; the reason inout exists isn't that it is impossible, it is just to counter template bloat) Is this common enough to warrant an addition, or would templates be good enough when it is needed? inout has a very common real world use case: slicing. inout(char)* strchr(inout(char)* haystack, in char* needle); What's the real world equivalent to strchr with regard to nullability? * typeof(this) inside a class is never nullable. You can't call a method with a null this. So a method that returns this can always just return the regular (not null) type. * this[x..y] is never nullable. Slicing a null reference is illegal, so again, no need to offer the nullable overload. * Returning a field of a nullable type is illegal, dereferencing null to get to the fields is illegal. Whereas a struct might return inout(member_variable), it never makes sense to do inout_nullability(member_variable). I can't think of a single real world case where you'd need this... and if I'm wrong, we can always use a template and/or revisit the problem later.
Feb 04 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 5 February 2014 at 00:49:25 UTC, Walter Bright 
wrote:
 What happens with this:

     T identity(T t) { return t; }

My previous post was with regard to nullability. This one is about ownership. The answer is the same: if you need this, use a template, otherwise, you always deal with a specific type (if you are allocating a new copy or need to store the reference somewhere outside the owner's scope) or a borrowed type: GC!Object obj; static assert(typeof(identity(obj) == GC!Object); and so on, you can test this today, D already has all these options. If you don't have a template, how do you approach this? Well, this is really nothing new for one: class Foo {} Foo foo = new Foo(); Object obj = foo; // legal Object identity(Object obj) { return obj; } foo = identity(foo); // won't compile without an explicit cast When you work with interfaces, some information is lost. That's a pain sometimes, but it is a feature too - the function that works on the interface doesn't need to care about anything else. With class inheritance, we can address this with covariant overrides.... but only of it is a member function (method). So let's just accept that the free function loses data. You can't get a container back from a range in the general case. So, what about the method: class Foo { typeof(this) identity() { return this; } } What is typeof(this)? I'll tell you: Foo! Here's why: *) The internal this pointer is never null, and thus never needs to be nullable, so that's out of consideration (see my last message) *) The internal this pointer does *not* own its memory. Calling "delete this;" is (I think) undefined behavior in D: the class might exist on the stack, or be allocated with malloc, and calling freeing the memory inside a method is likely to lead to a crash when the invariant is called upon returning anyway! Since the object does not own its own memory, it must always assume this is a borrowed reference. Therefore: * Escaping `this` is prohibited. Foo globalFoo; class Foo { // illegal, escapes a reference to a borrowed reference void storeThis() { globalFoo = this; } } Consider that this is wrong if the class is made with std.conv.emplace (or the deprecated scope storage class) on a stack buffer. This is silently wrong today, it can leave a dangling reference. (though since emplace is not safe, we do have that protection against such problems... though since we're talking about avoiding the GC here, emplace is something we probably want anyway) * Returning this is always a borrowed pointer, unless you explicitly make a copy. class Foo { Foo identity() { return this; } } GC!Foo foo = new Foo(); // owned by the Gc Foo foo2 = foo.identity(); // borrowed reference, NOT GC!Foo Therefore, we cannot escape it to a global that way either. If we want that, we have to work with foo directly, bypassing foo.identity. A clone method would create a new owned thing, so it returns something more specific: class Foo { GC!Foo clone() { auto foo = new GC; /* copy methods */ return foo; } } A clone method might also be templated on an allocator, and can thus change the return type as needed by the allocator. This wouldn't be virtual... but it kinda has to be. class Foo { virtual GC!Foo clone() {...} } class Bar : Foo { override RC!Foo clone() {...} } You wouldn't want that anyway, since if you called it through the foo interface, you wouldn't know how to free it (does it need to run a struct dtor? The GC? The caller needs to know.) Gotta pick an allocation method in the base class if you want it to work as a virtual function. So... yeah do that or use a non-virtual template. You can make it work. A clone method typically wouldn't even be inout anyway so meh. * A class is responsible for its member references but not itself. When the class' dtor is called, it needs to call member struct dtors, but does not call free(this) - leave that to the outside allcoator. This is already reality today (the GC calls the dtor before releasing the memory).
Feb 04 2014
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 5 February 2014 at 04:22:21 UTC, Andrei 
Alexandrescu wrote:
 Looks like a major weakness.

Even assuming that's true (something I don't agree with), it doesn't apply to D anyway since we can always template these functions! I'd like to say again that multiple pointer types is something we already have in D: * unique pointer (I think this is ~ in Rust but rust syntax looks like vomit to me) is achieved in D by disabling postblit with a struct and freeing in the dtor (RAII). It CAN offer a release method to transfer ownership. * ref counted pointer is achieved with postblit adding ref, dtor releasing ref. * GC pointer could be as simple as struct GC(T) { trustme T t; alias t this; } (where trustme disables the escape analysis checks I'd like to see implemented. Alternatively, if escape analysis was built in, we would make t private than then offer a getter property that returns scope T.) * borrowed pointer is the regular references we have today, plus escape analysis. We don't have to speculate how these work. Just write functions that use GC!T or RC!T or Unique!T - types we can go ahead and write immediately, they are pretty basic structs. Phobos even has unique and refcounted right now. We're already halfway there!
Feb 04 2014
prev sibling next sibling parent "Matthias Einwag" <matthias.einwag googlemail.com> writes:
 A function that takes a borrowed pointer &T can also be called 
 with an owning pointer ~T, an RC pointer Rc<T>, or a GC pointer 
 Gc<T>. They all convert neatly to a &T. One function to rule 
 them ... err .. accomodate all.

The only type that automatically coerces to a borrowed reference is an owned pointer. For value references you fist have to create a borrowes ref through &val. For Gc and Rc you have to pass something like ptr.borrow().borrow_mut ().borrow()
Feb 04 2014
prev sibling next sibling parent "Matthias Einwag" <matthias.einwag googlemail.com> writes:
 Again, what happens with:

    T identity(T t) { return t; }

 ? I.e. the transference of the argument pointer type to the 
 return type?

As far as I see, in Rust the pointer type is lost on the return value as well, if your function takes a reference (borrowed pointer). But you can do: fn identity(x: &T) -> T { return *T; } fn main() { let a = ~T; let r = ~identity(a); } That is: pass by reference and return a value type. If you then pass an owned pointer ~T, you can directly construct the returned value in heap allocated memory ~identity(a) and assign the resulting owned pointer. The compiler is smart enough to not do unnecessary copies.

This would definetly make r an unbecessary heap copy. Return *x would also only work for POD values, because for all others it would cause a (not allowed) move ibstead of a copy. For a copy you would have to return x.clone(). You can return by reference in rust, but you have to annotate everything with proper lifetimes. Possibly incorrect example: fn identity<'a>(x: &'a T) -> &'a { x } I dont think the comparisons to rust make too much sense because the whole system works completely different. For D the question mainly is how to manage classes (and also interfaces to them) with GC or ARC. Rust at the moment can't manage any interfaces/traits with theit smart pointers.
Feb 05 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 5 February 2014 at 00:47:27 UTC, Walter Bright 
wrote:
 On 2/4/2014 2:39 PM, Dicebot wrote:
 Also non-nullable types should be implicitly cast to nullable 
 parameters so you
 don't always need to support all cases distinctively.

The difficulty comes when needing to transfer the nullability of the argument to the nullability of the return type. Implicit conversions don't help with that. See inout.

Why can't inout be used for that as well? Same for pure & co. In my opinion "inout" should be simply generic placeholder saying "allow only code that works for all possible qualifiers and preserve those in return type". But nature of nullable is such that you rarely want to transfer it. Is can be useful as return type sometimes but when you pass it around and process it you almost always want it to be non-nullable. There is no practical benefit in trying to provide all-allowing API's.
Feb 05 2014
prev sibling next sibling parent "Meta" <jared771 gmail.com> writes:
On Wednesday, 5 February 2014 at 15:56:44 UTC, Dicebot wrote:
 On Wednesday, 5 February 2014 at 00:47:27 UTC, Walter Bright 
 wrote:
 On 2/4/2014 2:39 PM, Dicebot wrote:
 Also non-nullable types should be implicitly cast to nullable 
 parameters so you
 don't always need to support all cases distinctively.

The difficulty comes when needing to transfer the nullability of the argument to the nullability of the return type. Implicit conversions don't help with that. See inout.

Why can't inout be used for that as well? Same for pure & co. In my opinion "inout" should be simply generic placeholder saying "allow only code that works for all possible qualifiers and preserve those in return type". But nature of nullable is such that you rarely want to transfer it. Is can be useful as return type sometimes but when you pass it around and process it you almost always want it to be non-nullable. There is no practical benefit in trying to provide all-allowing API's.

If null becomes an invalid value for object references and pointers, then we can use it to unambiguously mean "item does not exist", and it becomes Just Another Value, so it is useful to transfer in various circumstances.
Feb 05 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 5 February 2014 at 17:09:31 UTC, Meta wrote:
 If null becomes an invalid value for object references and 
 pointers, then we can use it to unambiguously mean "item does 
 not exist", and it becomes Just Another Value, so it is useful 
 to transfer in various circumstances.

Yes, in that cases you transfer it as Nullable!T which is supposed to be single generic way to pass both value and reference types that can possibly be undefined. Judging by typical code I see though, such "does not exist" case is most commonly processed as if (!param) return; // return null; So if `param` becomes non-nullable you simply don't call processing function for null cases.
Feb 05 2014