www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - d future or plans for d3

reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
Hi all,

  I want to ask you about D future, i mean next big iteration of D and 
propose some new feature, agent-based programming. Currently, after 
introducing C++11 i see the only advantages of D over C++11 except 
syntax sugare is  garbage collector and modules.

  I recentrly attended student school (workshop) on multi-agent systems 
(MAS)and self-organizing sysetems. I was really impressed and thought 
that this probably is the silver bullet which Brucks declared to be 
absent. I mean agent-based programming as foundation of self-organzing 
systems. If you are interested you can find a lot of information by 
googling.

  So I would like to get your feedback to introduce new paradigm, 
paradigm of agent programming into D.

  Actually, I'm not deep into MAS, but as far as i know it's just 
autonomous class, i.e. class that has it's own independent context of 
execution that can communicate with other parties (agents) and can 
affect on environment if any (like ant).

  So it would be nice to have this in language core/library.

  There is erlang that already satisfied all requirements (as far as i 
know) of MAS language. So the question is does D need to take this 
paradigm? - Or concentrate on its current paradigms? The only advatnage 
ovder erlang i see is that D propose itself as embedded programming 
language which erlang do not satisfy (am i right?).

  So i need your feedback on the following:
	(i)   do you think that D needs to adsorb agent-programming paradigm
	(ii)  can it benefit D comparing to other modern languages

-- 
BR, Ruslan Mullakhmetov
Dec 17 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/18/2011 01:09 AM, Ruslan Mullakhmetov wrote:
 Hi all,

 I want to ask you about D future, i mean next big iteration of D and
 propose some new feature, agent-based programming. Currently, after
 introducing C++11 i see the only advantages of D over C++11 except
 syntax sugare is garbage collector and modules.

C++11 does not change the relation between D and C++ a lot. Why do you think it does?
 I recentrly attended student school (workshop) on multi-agent systems
 (MAS)and self-organizing sysetems. I was really impressed and thought
 that this probably is the silver bullet which Brucks declared to be
 absent. I mean agent-based programming as foundation of self-organzing
 systems. If you are interested you can find a lot of information by
 googling.

 So I would like to get your feedback to introduce new paradigm, paradigm
 of agent programming into D.

 Actually, I'm not deep into MAS, but as far as i know it's just
 autonomous class, i.e. class that has it's own independent context of
 execution that can communicate with other parties (agents) and can
 affect on environment if any (like ant).

 So it would be nice to have this in language core/library.

 There is erlang that already satisfied all requirements (as far as i
 know) of MAS language. So the question is does D need to take this
 paradigm? - Or concentrate on its current paradigms? The only advatnage
 ovder erlang i see is that D propose itself as embedded programming
 language which erlang do not satisfy (am i right?).

 So i need your feedback on the following:
 (i) do you think that D needs to adsorb agent-programming paradigm

The language does not have to be changed to get that to work.
 (ii) can it benefit D

Yes.
 comparing to other modern languages

IMO that is not a very important question. It is not a contest.
Dec 17 2011
next sibling parent reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
On 2011-12-18 00:56:33 +0000, Timon Gehr said:

 C++11 does not change the relation between D and C++ a lot. Why do you 
 think it does?

Because it incorporates many features D declared to be unique to it over C++ like - thread local variables - explicit concurrency model - type deduction - variadic templates - generalized constant expressions Some comparison is made at http://d-programming-language.org/cpp0x.html
 The language does not have to be changed to get that to work.

The C language doesn't have to be changed to get OOP working. There are libraries written in plain C satisfying all requirements of OOP, e.g. libav. Nevertheless C++ was born. So, what do i propose. To get it explicit in language and working out of the box, like in Erlang. the only benefit over Erlang i currently see that D is much more friendly for newcomers from C-like camp. The another is possibility for embedded programming. I was surprised that there are attempts to use MAS at embedded programming e.g. robotics where different controlers are autonomous and communicate with each others.
 
 comparing to other modern languages

IMO that is not a very important question. It is not a contest.

I thin that this is exactly context. I try to explain. D has reputation of marginal language with no concrete niche. For embedded programming C++ and rather plain C is used. For high level programming Java/C# are good enough. For distributed programming - Erlang. For scientific - Matlab, python, fortran, C. When i tell somebody about D the first question i get except community, maturity and stability is where it can be used. Where it gives true advantage. And i can not answer. So i propose to introduce new paradigm, despite of it is _not_ technical problem. it like ajax where combination of parts gave result greater then sum of parts. One may consider it as marketing trick to attract attention to D.
 On 12/18/2011 01:09 AM, Ruslan Mullakhmetov wrote:
 Hi all,
 
 I want to ask you about D future, i mean next big iteration of D and
 propose some new feature, agent-based programming. Currently, after
 introducing C++11 i see the only advantages of D over C++11 except
 syntax sugare is garbage collector and modules.
 

C++11 does not change the relation between D and C++ a lot. Why do you think it does?
 I recentrly attended student school (workshop) on multi-agent systems
 (MAS)and self-organizing sysetems. I was really impressed and thought
 that this probably is the silver bullet which Brucks declared to be
 absent. I mean agent-based programming as foundation of self-organzing
 systems. If you are interested you can find a lot of information by
 googling.
 
 So I would like to get your feedback to introduce new paradigm, paradigm
 of agent programming into D.
 
 Actually, I'm not deep into MAS, but as far as i know it's just
 autonomous class, i.e. class that has it's own independent context of
 execution that can communicate with other parties (agents) and can
 affect on environment if any (like ant).
 
 So it would be nice to have this in language core/library.
 
 There is erlang that already satisfied all requirements (as far as i
 know) of MAS language. So the question is does D need to take this
 paradigm? - Or concentrate on its current paradigms? The only advatnage
 ovder erlang i see is that D propose itself as embedded programming
 language which erlang do not satisfy (am i right?).
 
 So i need your feedback on the following:
 (i) do you think that D needs to adsorb agent-programming paradigm

The language does not have to be changed to get that to work.
 (ii) can it benefit D

Yes.
 comparing to other modern languages

IMO that is not a very important question. It is not a contest.

-- BR, Ruslan Mullakhmetov
Dec 18 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/18/2011 11:51 AM, Ruslan Mullakhmetov wrote:
 On 2011-12-18 00:56:33 +0000, Timon Gehr said:

 C++11 does not change the relation between D and C++ a lot. Why do you
 think it does?

Because it incorporates many features D declared to be unique to it

It does not, except for the most trivial stuff.
 over
 C++ like
 - thread local variables

D never declared that to be unique over C++. In D, thread local is the *default*. C++ does not have that, and it cannot have that.
 - explicit concurrency model

?
 - type deduction

C++11 cannot deduce function return value types. Except for lambdas. C++11 cannot pass template lambda functions as template parameters.
 - variadic templates

Yes, now you can pass a variable number of things to a C++ template. But what kind of things you can pass is still severely restricted.
 - generalized constant expressions

LOL! D has full CTFE. C++11's generalized constant expressions are a joke in comparison.
 Some comparison is made at http://d-programming-language.org/cpp0x.html

 The language does not have to be changed to get that to work.

The C language doesn't have to be changed to get OOP working. There are libraries written in plain C satisfying all requirements of OOP, e.g. libav. Nevertheless C++ was born.

This analogy is broken. D is expressive enough that there is no point in changing the language to support it. What part of MAS would require language support to be done nicely?
 So, what do i propose. To get it explicit in language and working out of
 the box, like in Erlang. the only benefit over Erlang i currently see
 that D is much more friendly for newcomers from C-like camp. The another
 is possibility for embedded programming. I was surprised that there are
 attempts to use MAS at embedded programming e.g. robotics where
 different controlers are autonomous and communicate with each others.

 comparing to other modern languages

IMO that is not a very important question. It is not a contest.

I thin that this is exactly context. I try to explain. D has reputation of marginal language

D pushes the margin. :o)
 with no concrete niche.

It does not need a 'niche' and that is a good thing. D is general purpose. You can throw it at any task and expect good results. That is why MAS does not need explicit language support.
Dec 18 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 3:07 PM, Timon Gehr wrote:
 On 12/18/2011 11:51 AM, Ruslan Mullakhmetov wrote:
 On 2011-12-18 00:56:33 +0000, Timon Gehr said:

 C++11 does not change the relation between D and C++ a lot. Why do you
 think it does?

Because it incorporates many features D declared to be unique to it

It does not, except for the most trivial stuff.

I think it's best to not derail the argument into a D vs. C++ thing. Ruslan, what language and what library features do you envision should be added, and what would be their most likely applications? Thanks, Andrei
Dec 18 2011
parent Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
I think there is no need in language changes. Everythink can be 
implemented via library.

What is needed:
  - base agent class
  - base behaviors
  - runtime that provide ability to run independent agents even in 
single-thread mode. Number of agents can be greater then number of 
treads. so scheduler is needed.
  - communication platform to delver messages between agents with 
possibility of broadcasting, multicasting, unicasting (normal).
  - service discovery mechanism allowing to register/deregister agents 
and query for agents with specific roles.

I will prepare more detail answer to the weekend when i get free.

As for applications, i see embedded market: communicating with each 
other phones via bluetooth/wi-fi/GSM/CDMA, robotics and others. At 
least it my hopes.

On 2011-12-18 22:23:28 +0000, Andrei Alexandrescu said:

 On 12/18/11 3:07 PM, Timon Gehr wrote:
 On 12/18/2011 11:51 AM, Ruslan Mullakhmetov wrote:
 On 2011-12-18 00:56:33 +0000, Timon Gehr said:
 
 C++11 does not change the relation between D and C++ a lot. Why do you
 think it does?

Because it incorporates many features D declared to be unique to it

It does not, except for the most trivial stuff.

I think it's best to not derail the argument into a D vs. C++ thing. Ruslan, what language and what library features do you envision should be added, and what would be their most likely applications? Thanks, Andrei

-- BR, Ruslan Mullakhmetov
Dec 18 2011
prev sibling parent Trass3r <un known.com> writes:
Am 18.12.2011, 11:51 Uhr, schrieb Ruslan Mullakhmetov <tiabaldu gmail.com>:

 On 2011-12-18 00:56:33 +0000, Timon Gehr said:

 C++11 does not change the relation between D and C++ a lot. Why do you  
 think it does?

Because it incorporates many features D declared to be unique to it over C++ like - thread local variables

D turned things around by making thread-local the default!
    - type deduction

    - generalized constant expressions

compiler could do that much better. CTFE ftw.
Dec 18 2011
prev sibling next sibling parent reply a <a a.com> writes:
Ruslan Mullakhmetov Wrote:

 Currently, after 
 introducing C++11 i see the only advantages of D over C++11 except 
 syntax sugare is  garbage collector and modules.

So you are saying that sane templates, range based standard library and concurrency improvements (thread local variables, immutable, message passing) are all just syntactic sugar?
Dec 17 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/18/2011 02:42 AM, a wrote:
 Ruslan Mullakhmetov Wrote:

 Currently, after
 introducing C++11 i see the only advantages of D over C++11 except
 syntax sugare is  garbage collector and modules.

So you are saying that sane templates, range based standard library and concurrency improvements (thread local variables, immutable, message passing) are all just syntactic sugar?

And you didn't even mention CTFE and code generation yet.
Dec 17 2011
next sibling parent Olivier Pisano <olivier.pisano laposte.net> writes:
Le 18/12/11 02:53, Timon Gehr a écrit :
 On 12/18/2011 02:42 AM, a wrote:
 Ruslan Mullakhmetov Wrote:

 Currently, after
 introducing C++11 i see the only advantages of D over C++11 except
 syntax sugare is garbage collector and modules.

So you are saying that sane templates, range based standard library and concurrency improvements (thread local variables, immutable, message passing) are all just syntactic sugar?

And you didn't even mention CTFE and code generation yet.

Arrays manipulation !
Dec 18 2011
prev sibling next sibling parent reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
  I do not want to make a flame over D vs C++11.

  I answered else thread on some differences of C++11 vs D, but i agree 
with you that i was a little bit in a hurry.

  After rechecking i remembered constraints, static compilation (static 
if), contract programming, functional programming. perhaps i skipped 
something.

  For those features you mentioned:
    - templates are know quite sane, but D still outperforms it. The 
question is how many people would benefit from this difference.
    - thread local variables: thread local storage (though not default)
    - message passing implemented via libraries: boost.task?
    - immutables are only needed for functional programming which in 
turn necessary for parallelizing / distributed applications / formal 
verification. I don't know, does D (compiler) utilize it?

  Once again, i'm not trying to make a holywar. I'm D lover myself. But 
a lot of people do not consider this benefits of D enough to shift to 
it as for my opinion and experience.


On 2011-12-18 01:53:14 +0000, Timon Gehr said:

 On 12/18/2011 02:42 AM, a wrote:
 Ruslan Mullakhmetov Wrote:
 
 Currently, after
 introducing C++11 i see the only advantages of D over C++11 except
 syntax sugare is  garbage collector and modules.

So you are saying that sane templates, range based standard library and concurrency improvements (thread local variables, immutable, message passing) are all just syntactic sugar?

And you didn't even mention CTFE and code generation yet.

-- BR, Ruslan Mullakhmetov
Dec 18 2011
parent reply Somedude <lovelydear mailmetrash.com> writes:
Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
  I do not want to make a flame over D vs C++11.
 
 ...
  Once again, i'm not trying to make a holywar. I'm D lover myself. But a
 lot of people do not consider this benefits of D enough to shift to it
 as for my opinion and experience.
 
 

These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare.
Dec 18 2011
next sibling parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <xtzgzorex gmail.com> writes:
On 18-12-2011 12:45, Somedude wrote:
 Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
   I do not want to make a flame over D vs C++11.

 ...
   Once again, i'm not trying to make a holywar. I'm D lover myself. But a
 lot of people do not consider this benefits of D enough to shift to it
 as for my opinion and experience.

These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare.

On the other hand, Erlang's runtime is built around message-passing. I *extremely strongly doubt* that you can get similar performance out of message-passing in D, for two reasons: 1) The Erlang runtime is written in highly optimized C. 2) Erlang uses better garbage collection strategies than D. On point 1: Of course, you can write a message-passing implementation in D that's basically just glorified C, but that defeats the point of using D in the first place IMHO. On point 2: This is highly unlikely to change. It seems (from reading the NG's past discussions) that there is virtually no interest from the core devs of D to do The Right Thing to make precise garbage collection possible, sadly (and neither to make thread-local GC possible, and other such techniques (Erlang runs a separate GC per Erlang process, very successfully)). Yes, Erlang's emulator is not good in raw performance. It was never geared for this (that's why it has easy interfacing with C for performance-sensitive code). However, things like HiPE improve the situation. All languages have their place, and Erlang is still going strong in the world of massive concurrency. I doubt D will be able to challenge this until its GC issues have been resolved (and even then, D is nowhere near as convenient, lacking fault tolerance and location transparency; sometimes these traits are *much* more desirable than raw performance when dealing with concurrency on such high levels). - Alex
Dec 18 2011
next sibling parent reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
As far as I understand Erlang performance is reached by horizontal 
scheduling (i'm not sure i used exact word) by which i mean that if you 
have lack of performance you can buy new hardware nodes (servers) and 
without any changes in code and even recompiling you can distribute 
workload.

That is very good approach in situation when processor stopped getting 
faster but become more parallelized and cheaper.

GC is just a mater of implementation. In presence of resources 
implement good GC algorithm offers no difficulty.

 All languages have their place

sure. and many people doubt of place of D. It would be nice if such good lang will find it's place. On 2011-12-18 12:19:43 +0000, Alex Rønne Petersen said:
 On 18-12-2011 12:45, Somedude wrote:
 Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
 I do not want to make a flame over D vs C++11.
 
 ...
 Once again, i'm not trying to make a holywar. I'm D lover myself. But a
 lot of people do not consider this benefits of D enough to shift to it
 as for my opinion and experience.
 
 

These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare.

On the other hand, Erlang's runtime is built around message-passing. I *extremely strongly doubt* that you can get similar performance out of message-passing in D, for two reasons: 1) The Erlang runtime is written in highly optimized C. 2) Erlang uses better garbage collection strategies than D. On point 1: Of course, you can write a message-passing implementation in D that's basically just glorified C, but that defeats the point of using D in the first place IMHO. On point 2: This is highly unlikely to change. It seems (from reading the NG's past discussions) that there is virtually no interest from the core devs of D to do The Right Thing to make precise garbage collection possible, sadly (and neither to make thread-local GC possible, and other such techniques (Erlang runs a separate GC per Erlang process, very successfully)). Yes, Erlang's emulator is not good in raw performance. It was never geared for this (that's why it has easy interfacing with C for performance-sensitive code). However, things like HiPE improve the situation. All languages have their place, and Erlang is still going strong in the world of massive concurrency. I doubt D will be able to challenge this until its GC issues have been resolved (and even then, D is nowhere near as convenient, lacking fault tolerance and location transparency; sometimes these traits are *much* more desirable than raw performance when dealing with concurrency on such high levels). - Alex

-- BR, Ruslan Mullakhmetov
Dec 18 2011
parent reply Somedude <lovelydear mailmetrash.com> writes:
Le 18/12/2011 15:07, Ruslan Mullakhmetov a écrit :
 
 GC is just a mater of implementation. In presence of resources implement
 good GC algorithm offers no difficulty.
 

Oh really ? Then please make us a favor and write one for D. Also I'm sure the C++ guys will be pleased to hear that it's such an easy task.
Dec 18 2011
parent reply =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <xtzgzorex gmail.com> writes:
On 18-12-2011 15:40, Somedude wrote:
 Le 18/12/2011 15:07, Ruslan Mullakhmetov a écrit :
 GC is just a mater of implementation. In presence of resources implement
 good GC algorithm offers no difficulty.

Oh really ? Then please make us a favor and write one for D. Also I'm sure the C++ guys will be pleased to hear that it's such an easy task.

Yeah, unfortunately, we can't just keep saying "it's an implementation issue". It's very much a real problem; D programmers are *avoiding* the GC because it just isn't comparable to, say, the .NET and Java GCs in performance at all. - Alex
Dec 19 2011
parent reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
On 2011-12-19 11:52:25 +0000, Alex Rønne Petersen said:

 On 18-12-2011 15:40, Somedude wrote:
 Le 18/12/2011 15:07, Ruslan Mullakhmetov a écrit :
 
 GC is just a mater of implementation. In presence of resources implement
 good GC algorithm offers no difficulty.
 

Oh really ? Then please make us a favor and write one for D. Also I'm sure the C++ guys will be pleased to hear that it's such an easy task.

Yeah, unfortunately, we can't just keep saying "it's an implementation issue". It's very much a real problem; D programmers are *avoiding* the GC because it just isn't comparable to, say, the .NET and Java GCs in performance at all. - Alex

Thanks for you explanation. I'm quite far away from GC but where is the problem compare to Java and .NET? Resources (people), specific language features making hard to implement GC or something else? When i said that this is just a matter of implementation i followed the idea that it's already implemented in say Java, C#, Erlang wich GC was declared to be good enough in this topic. -- BR, Ruslan Mullakhmetov
Dec 19 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 07:01 AM, Ruslan Mullakhmetov wrote:
 On 2011-12-19 11:52:25 +0000, Alex Rønne Petersen said:

 On 18-12-2011 15:40, Somedude wrote:
 Le 18/12/2011 15:07, Ruslan Mullakhmetov a écrit :
 GC is just a mater of implementation. In presence of resources
 implement
 good GC algorithm offers no difficulty.

Oh really ? Then please make us a favor and write one for D. Also I'm sure the C++ guys will be pleased to hear that it's such an easy task.

Yeah, unfortunately, we can't just keep saying "it's an implementation issue". It's very much a real problem; D programmers are *avoiding* the GC because it just isn't comparable to, say, the .NET and Java GCs in performance at all. - Alex

Thanks for you explanation. I'm quite far away from GC but where is the problem compare to Java and .NET? Resources (people), specific language features making hard to implement GC or something else? When i said that this is just a matter of implementation i followed the idea that it's already implemented in say Java, C#, Erlang wich GC was declared to be good enough in this topic.

The difference is that those languages are entirely type safe.
Dec 20 2011
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, December 20, 2011 11:21:41 Froglegs wrote:
   I've only recently tried D out, but what I'd like to see..
 
 -GC being truly optional
 -being able to specify if the standard library should use GC or
 not, perhaps with allocators

Some aspects of D will _always_ require a GC or they won't work. Array concatenation would be a prime example. I believe that delegates are another major example. I think that scoped delegates avoid the problem, but any that require closures do not. Other things be done but become risky - e.g. slicing arrays (the GC normally owns the memory such that all dynamic arrays are slices and none of them own their memory, so slicing manually managed memory gets dicey). There are definitely portions of the standard library that can and should be useable without the GC, but because some aspects of the language require, some portions of the standard library will always require it. In general, I don't think that it's reasonable to expect to use D without a GC. You can really minimize how much you use it, and if you're really, really careful and avoid some of D's nicer features, you might be able to avoid it entirely, but realistically, if you're using D, you're going to be using the GC at least some. In general, the trick is going to be allowing custom allocators where it makes sense (e.g. containers) and being smart about how you design your program (e.g. using structs instead of classes if you don't need the extra abilities of a class - such as polymorphism). So, if you're smart, you can be very efficient with memory usage in D, but unless you really want to hamstring your usage of D, avoiding the GC entirely just doesn't work. - Jonathan M Davis
Dec 20 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 11:48 AM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 11:21:41 Froglegs wrote:
    I've only recently tried D out, but what I'd like to see..

 -GC being truly optional
 -being able to specify if the standard library should use GC or
 not, perhaps with allocators

Some aspects of D will _always_ require a GC or they won't work.

This is not a huge problem. They just wont work if the programmer chooses not to use the GC. There is nothing absolutely essential to writing working D programs that requires the GC.
 Array
 concatenation would be a prime example. I believe that delegates are another
 major example. I think that scoped delegates avoid the problem, but any that
 require closures do not. Other things be done but become risky - e.g. slicing
 arrays (the GC normally owns the memory such that all dynamic arrays are
 slices and none of them own their memory, so slicing manually managed memory
 gets dicey).

 There are definitely portions of the standard library that can and should be
 useable without the GC, but because some aspects of the language require, some
 portions of the standard library will always require it. In general, I don't
 think that it's reasonable to expect to use D without a GC. You can really
 minimize how much you use it, and if you're really, really careful and avoid
 some of D's nicer features, you might be able to avoid it entirely, but
 realistically, if you're using D, you're going to be using the GC at least
 some.

 In general, the trick is going to be allowing custom allocators where it makes
 sense (e.g. containers) and being smart about how you design your program
 (e.g. using structs instead of classes if you don't need the extra abilities
 of a class - such as polymorphism). So, if you're smart, you can be very
 efficient with memory usage in D, but unless you really want to hamstring your
 usage of D, avoiding the GC entirely just doesn't work.

Note that you can use manual memory management for classes.
Dec 20 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 11:21 AM, Froglegs wrote:
 I've only recently tried D out, but what I'd like to see..

 -GC being truly optional
 -being able to specify if the standard library should use GC or not,
 perhaps with allocators
 -if D is going to have a GC, then a precise, compacting one would be
 cool, but not if it gets in the way of making the GC optional


 One thing I'm not sure about, D classes are virtual by default, but if
 you mark all functions as final does the class still contain a VFP or
 any other cruft?

The class will still have a vptr. The vtable will contain only the type info.
 Also why are class functions virtual by default? My experience in C++ is
 that I rarely use virtual, so I don't really understand why that is the
 default.

In C++, class and struct are essentially the same. In D, you use classes if you want polymorphism and structs if you don't need it. structs cannot participate in inheritance hierarchies or contain virtual functions. If you don't need virtual functions, you should probably use structs instead of classes. (you are not doing OOP anyway.) structs don't contain a vptr. If you want to use extensible with final methods, class C{final: /* method declarations */} will do the job. But it is not the most common case, so it should not be the default. (IIRC C# takes a different stance on this, they have final methods by default to force the programmer to make dynamic binding explicit. It has both benefits and drawbacks.)
Dec 20 2011
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Timon Gehr:

 If you don't need virtual functions, you should probably use 
 structs instead of classes. (you are not doing OOP anyway.)

I don't agree with both that statements. Bye, bearophile
Dec 20 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 11:57 AM, bearophile wrote:
 Timon Gehr:

 If you don't need virtual functions, you should probably use
 structs instead of classes. (you are not doing OOP anyway.)

I don't agree with both that statements. Bye, bearophile

1. He does not want type info. Structs don't have type info. He does not want virtual functions. Structs don't support virtual functions. Ergo he should use mostly structs. Please defend your disagreement. 2. Dynamic binding is a core concept of OOP. A language that does not support dynamic binding does not support OOP. A program that does not use dynamic binding is not object oriented. What is to disagree with?
Dec 20 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 05:36 PM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 15:49:34 Timon Gehr wrote:
 2. Dynamic binding is a core concept of OOP. A language that does not
 support dynamic binding does not support OOP. A program that does not
 use dynamic binding is not object oriented. What is to disagree with?

I don't agree with that either. You don't need polymorphism for OOP. It's quite restricted without it, but you can still program with objects even if you're restricted to something like D's structs, so you're still doing OOP. - Jonathan M Davis

No. That is glorified procedural style. 'Objects' as in 'OOP' carry data and _behavior_, structs don't (except if you give them some function pointers, but that is just implementing poor man's polymorphism.) Having some kind of dynamic execution model is a requirement for OOP. There are no two ways about it.
Dec 20 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 06:41 PM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 17:58:30 Timon Gehr wrote:
 On 12/20/2011 05:36 PM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 15:49:34 Timon Gehr wrote:
 2. Dynamic binding is a core concept of OOP. A language that does not
 support dynamic binding does not support OOP. A program that does not
 use dynamic binding is not object oriented. What is to disagree with?

I don't agree with that either. You don't need polymorphism for OOP. It's quite restricted without it, but you can still program with objects even if you're restricted to something like D's structs, so you're still doing OOP. - Jonathan M Davis

No. That is glorified procedural style. 'Objects' as in 'OOP' carry data and _behavior_, structs don't (except if you give them some function pointers, but that is just implementing poor man's polymorphism.) Having some kind of dynamic execution model is a requirement for OOP. There are no two ways about it.

Well, I completely disagree. The core of OOP is encapsulating the data within an object

Yes, encapsulation is another core concept of OOP.
 and having functions associated with the object itself which operate
 on that data.

Correct. The functions have to be associated with _the object itself_. Case closed.
 It's about encapsulation and tying the functions to the type.

Static typing is not an OOP concept.
 Polymorphism is a nice bonus, but it's not required.

It is not a bonus. It is part of what OOP is about.
 I'd say that any language which is really trying to do OOP should definitely
 have polymorphism or it's going to have pretty sucky OOP, but it can still
 have OOP.

In case you define OOP differently from all relevant textbooks, based only on the encapsulation aspect, yes.
Dec 20 2011
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/11 11:41 AM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 17:58:30 Timon Gehr wrote:
 On 12/20/2011 05:36 PM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 15:49:34 Timon Gehr wrote:
 2. Dynamic binding is a core concept of OOP. A language that does not
 support dynamic binding does not support OOP. A program that does not
 use dynamic binding is not object oriented. What is to disagree with?

I don't agree with that either. You don't need polymorphism for OOP. It's quite restricted without it, but you can still program with objects even if you're restricted to something like D's structs, so you're still doing OOP. - Jonathan M Davis

No. That is glorified procedural style. 'Objects' as in 'OOP' carry data and _behavior_, structs don't (except if you give them some function pointers, but that is just implementing poor man's polymorphism.) Having some kind of dynamic execution model is a requirement for OOP. There are no two ways about it.

Well, I completely disagree. The core of OOP is encapsulating the data within an object and having functions associated with the object itself which operate on that data. It's about encapsulation and tying the functions to the type. Polymorphism is a nice bonus, but it's not required.

I think the model you're discussing is often called "object-based". Andrei
Dec 20 2011
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Timon Gehr:

 What is to disagree with?

Sorry for my precedent answer, please ignore it, sometimes I have a too much big mouth. I think discussions about definitions are not so interesting. Bye, bearophile
Dec 20 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, December 20, 2011 15:49:34 Timon Gehr wrote:
 2. Dynamic binding is a core concept of OOP. A language that does not
 support dynamic binding does not support OOP. A program that does not
 use dynamic binding is not object oriented. What is to disagree with?

I don't agree with that either. You don't need polymorphism for OOP. It's quite restricted without it, but you can still program with objects even if you're restricted to something like D's structs, so you're still doing OOP. - Jonathan M Davis
Dec 20 2011
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/11 5:17 AM, Froglegs wrote:
 The array concatenation requiring GC I get, but why does a delegate
 require it?

 This link says D allocates closures on the heap

 http://en.wikipedia.org/wiki/Anonymous_function#D

 I don't really get why, C++ lambda works well(aside from broken lack of
 template lambda's) and do not require heap usage, even binding it to
 std::function can generally avoid it if it doesn't exceed the SBO size

Well another way of putting it is std::function MUST do heap allocation if environment size exceeds the SBO size. It's the same thing here. Just like C++ lambdas, D lambdas and local functions don't need heap allocation unless they escape their scope. Andrei
Dec 20 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/11 5:56 AM, Froglegs wrote:
 C++ "closures" do not allow you to maintain a reference to the context
 after the function containing said context returns. Instead, C++
 allows you to choose between copying the variables into the lambda
 instance, or referencing them (the references may not "escape"). The
 compiler may or may not enforce correct uses of reference captures. In
 contrast, D's approach is both intuitive (does not copy variables) and
 safe (conservatively allocates on the heap), with the downside of
 requiring the context to be garbage-collected.

Ah, makes sense now, thanks. Still it seems like a case of "you pay for what you don't use", and seems like a real downer for adopting D since you loose the ability to use lambda's without having the GC shoved down your throat(wouldn't be so bad if the D GC was known for performance, but everything I've read indicates it is quite slow).

D's pass-down lambdas do not need memory allocation. As far as I remember none of std.algorithm's use of lambda allocates memory. Andrei
Dec 20 2011
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 20/12/2011 14:08, Andrei Alexandrescu a écrit :
 On 12/20/11 5:56 AM, Froglegs wrote:
 C++ "closures" do not allow you to maintain a reference to the context
 after the function containing said context returns. Instead, C++
 allows you to choose between copying the variables into the lambda
 instance, or referencing them (the references may not "escape"). The
 compiler may or may not enforce correct uses of reference captures. In
 contrast, D's approach is both intuitive (does not copy variables) and
 safe (conservatively allocates on the heap), with the downside of
 requiring the context to be garbage-collected.

Ah, makes sense now, thanks. Still it seems like a case of "you pay for what you don't use", and seems like a real downer for adopting D since you loose the ability to use lambda's without having the GC shoved down your throat(wouldn't be so bad if the D GC was known for performance, but everything I've read indicates it is quite slow).

D's pass-down lambdas do not need memory allocation. As far as I remember none of std.algorithm's use of lambda allocates memory. Andrei

Is the compiler able to ensure that and do not allocate on the heap ?
Dec 20 2011
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/20/11 7:41 AM, deadalnix wrote:
 D's pass-down lambdas do not need memory allocation. As far as I
 remember none of std.algorithm's use of lambda allocates memory.

 Andrei

Is the compiler able to ensure that and do not allocate on the heap ?

Yes, to the best of my knowledge it's pretty much cut and dried. Andrei
Dec 20 2011
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/3/12 11:34 AM, d coder wrote:
     D's pass-down lambdas do not need memory allocation. As far as I
     remember none of std.algorithm's use of lambda allocates memory.



 On that front Andrei, did you get a chance to discuss my request to make
 std.algorithm more usable with Walter?

 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=140807
 <http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=140807>

 - Regards

It's on my mind, but I don't have a good proposal in mind. For better or worse, it seems to not appear on many people's radar, which pushes its priority down. I do think it should be addressed. Any ideas, please share. Thanks, Andrei
Jan 03 2012
prev sibling next sibling parent Somedude <lovelydear mailmetrash.com> writes:
Le 18/12/2011 13:19, Alex Rønne Petersen a écrit :
 On the other hand, Erlang's runtime is built around message-passing. >

of message-passing in D As I said inmy previous post, the kind of performance that Erlang has is useful for specific applications, but not really for a general purpose language. For instance, it's very well suited for writing the server side software of a MMORPG like World of Warcraft, but it's pretty much useless for writing the client.
Dec 18 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 6:19 AM, Alex Rønne Petersen wrote:
 On 18-12-2011 12:45, Somedude wrote:
 Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
 I do not want to make a flame over D vs C++11.

 ...
 Once again, i'm not trying to make a holywar. I'm D lover myself. But a
 lot of people do not consider this benefits of D enough to shift to it
 as for my opinion and experience.

These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare.

On the other hand, Erlang's runtime is built around message-passing. I *extremely strongly doubt* that you can get similar performance out of message-passing in D, for two reasons: 1) The Erlang runtime is written in highly optimized C. 2) Erlang uses better garbage collection strategies than D. On point 1: Of course, you can write a message-passing implementation in D that's basically just glorified C, but that defeats the point of using D in the first place IMHO.

I'm not so sure. That seems to presuppose that use of any non-C features of D exacts a performance overhead, something I'd disagree with.
 On point 2: This is highly unlikely to change. It seems (from reading
 the NG's past discussions) that there is virtually no interest from the
 core devs of D to do The Right Thing to make precise garbage collection
 possible, sadly (and neither to make thread-local GC possible, and other
 such techniques (Erlang runs a separate GC per Erlang process, very
 successfully)).

Walter and I are both interested in making the GC precise.
 Yes, Erlang's emulator is not good in raw performance. It was never
 geared for this (that's why it has easy interfacing with C for
 performance-sensitive code). However, things like HiPE improve the
 situation.

 All languages have their place, and Erlang is still going strong in the
 world of massive concurrency.

That may be the case, perhaps outside Ericsson. I know a few people at Ericsson, and according to them the use of Erlang there is waning. No new projects are started in Erlang and some production software is rewritten in C++.
 I doubt D will be able to challenge this
 until its GC issues have been resolved (and even then, D is nowhere near
 as convenient, lacking fault tolerance and location transparency;
 sometimes these traits are *much* more desirable than raw performance
 when dealing with concurrency on such high levels).

We need to improve the GC, that's a given. Then the question remains whether fault tolerance and location transparency are language features or library/environment/deployment attributes. Andrei
Dec 18 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 1:46 PM, jdrewsen wrote:
 On Sunday, 18 December 2011 at 17:19:33 UTC, Andrei Alexandrescu wrote:
 On 12/18/11 6:19 AM, Alex Rønne Petersen wrote:
 On 18-12-2011 12:45, Somedude wrote:
 Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
 I do not want to make a flame over D vs C++11.




<snip>
 Walter and I are both interested in making the GC precise.

Wouldn't support for precise GC collection mean larger executable because type information is needed at runtime for precise collection? If so, could extra info be used to improve runtime reflection as well?

That is an interesting opportunity. At any rate, I am 100% convinced precise GC is the only way to go, and I think I've convinced Walter to a good extent as well. Andrei
Dec 18 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/18/2011 11:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

It is an unilateral improvement if both options are kept open. I don't see a reason to cease support for the current GC model. Furthermore, a generational GC performs much better than a simple mark-sweep GC.
Dec 18 2011
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 5:11 PM, Vladimir Panteleev wrote:
 Also, D can't have a completely precise
 GC as long as it has unions and can pass managed memory to C code.

Also casts pointer <-> integrals. These arguments don't blunt the desirability of precise GC. Andrei
Dec 18 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 12:16 AM, Andrei Alexandrescu wrote:
 On 12/18/11 5:11 PM, Vladimir Panteleev wrote:
 Also, D can't have a completely precise
 GC as long as it has unions and can pass managed memory to C code.

Also casts pointer <-> integrals.

Casting a pointer to GC memory to an integral results in undefined behavior: http://www.d-programming-language.org/garbage.html
 These arguments don't blunt the desirability of precise GC.


 Andrei

Dec 18 2011
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 12:11 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:02:17 UTC, Timon Gehr wrote:
 It is an unilateral improvement if both options are kept open. I don't
 see a reason to cease support for the current GC model.

I believe that currently the plan does not include providing this choice.

That would strike me as odd. Andrei/Walter?
 Furthermore, a generational GC performs much better than a simple
 mark-sweep GC.

Unless you change the way references work, generational and "precise" aspects of a GC are orthogonal.

You are right. I have had in mind a generational GC that uses a copying collector for the nursery as this is what most state-of-the-art VM GCs do.
 Also, D can't have a completely precise
 GC as long as it has unions and can pass managed memory to C code.

We can change the way unions are layed out. The compiler can mark GC memory passed to a C function as non-movable, or passing GC memory to a C function can be made illegal if the GC is enabled.
Dec 18 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 5:18 PM, Timon Gehr wrote:
 On 12/19/2011 12:11 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:02:17 UTC, Timon Gehr wrote:
 It is an unilateral improvement if both options are kept open. I don't
 see a reason to cease support for the current GC model.

I believe that currently the plan does not include providing this choice.

That would strike me as odd. Andrei/Walter?

The plan is in too early a stage to even discuss that choice.
 Furthermore, a generational GC performs much better than a simple
 mark-sweep GC.

Unless you change the way references work, generational and "precise" aspects of a GC are orthogonal.

You are right. I have had in mind a generational GC that uses a copying collector for the nursery as this is what most state-of-the-art VM GCs do.
 Also, D can't have a completely precise
 GC as long as it has unions and can pass managed memory to C code.

We can change the way unions are layed out. The compiler can mark GC memory passed to a C function as non-movable, or passing GC memory to a C function can be made illegal if the GC is enabled.

Unions will be conservative. The golden standard is that SafeD can't use them or anything that forces conservative approaches. Andrei
Dec 18 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
 You are right. I have had in mind a generational GC that uses a
 copying collector for the nursery as this is what most
 state-of-the-art VM GCs do.

 ...

 We can change the way unions are layed out. The compiler can mark GC
 memory passed to a C function as non-movable, or passing GC memory to
 a C function can be made illegal if the GC is enabled.

These changes are too invasive for the language at this point, I believe. We need to work with what we have.

I disagree. Code that relies on other semantics would just have to use conservative GC.
Dec 18 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 12:45 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:31:03 UTC, Timon Gehr wrote:
 On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
 You are right. I have had in mind a generational GC that uses a
 copying collector for the nursery as this is what most
 state-of-the-art VM GCs do.

believe. We need to work with what we have.

I disagree. Code that relies on other semantics would just have to use conservative GC.

Please elaborate on how you would hypothetically change D to accustom such changes. I am having trouble imagining such an implementation that would not have a considerable impact on D's performance.

If you have an union union X{ int x; int* y; } The compiler would just lay out x and y sequentially instead of at the same memory location. Alternatively, it could add a tag to each union. In case of passing GC memory to C functions, I would prefer to just disallow the C code to capture the reference, and to disable GC while the C function runs. (disabling and enabling GC is just a matter of modifying a counter somewhere, that should not visibly impact performance.) If C code was to capture a reference to the memory then it would probably take ownership anyway, which would necessitate to allocate the memory with malloc even with conservative GC. I am not sure what to do about void[] though.
Dec 18 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 01:00 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:55:17 UTC, Timon Gehr wrote:
 On 12/19/2011 12:45 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:31:03 UTC, Timon Gehr wrote:
 On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
 You are right. I have had in mind a generational GC that uses a
 copying collector for the nursery as this is what most
 state-of-the-art VM GCs do.

believe. We need to work with what we have.

I disagree. Code that relies on other semantics would just have to use conservative GC.

Please elaborate on how you would hypothetically change D to accustom such changes. I am having trouble imagining such an implementation that would not have a considerable impact on D's performance.

If you have an union union X{ int x; int* y; } The compiler would just lay out x and y sequentially instead of at the same memory location. Alternatively, it could add a tag to each union. In case of passing GC memory to C functions, I would prefer to just disallow the C code to capture the reference, and to disable GC while the C function runs. (disabling and enabling GC is just a matter of modifying a counter somewhere, that should not visibly impact performance.) If C code was to capture a reference to the memory then it would probably take ownership anyway, which would necessitate to allocate the memory with malloc even with conservative GC. I am not sure what to do about void[] though.

OK... but what about the "generational GC that uses a copying collector for the nursery"?

I am not sure I get the question. The D specification already allows moving GC implementations. What is the issue you are thinking about?
Dec 18 2011
prev sibling next sibling parent reply a <a a.com> writes:
 In case of passing GC memory to C functions, I would prefer to just 
 disallow the C code to capture the reference, and to disable GC while 
 the C function runs.

Wouldn't this be a problem with non concurrent GC? If you have multiple threads that spend most of the time in C function calls (for example performing IO), garbage collection would be impossible most of the time. In the extreme case garbage would never be collected. Couldn't we just require that the programmer keeps a reference to the memory passed to C in a D variable? AFAIK python does it that way, but I don't use python enough to know if it causes serious problems there.
Dec 18 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 05:23 AM, a wrote:
 In case of passing GC memory to C functions, I would prefer to just
 disallow the C code to capture the reference, and to disable GC while
 the C function runs.

Wouldn't this be a problem with non concurrent GC? If you have multiple threads that spend most of the time in C function calls (for example performing IO), garbage collection would be impossible most of the time. In the extreme case garbage would never be collected.

Garbage collection is triggered by allocations. The C function does not perform GC allocations. While the memory is allocated, the GC is enabled. With non-concurrent GC there is certainly no issue, but the GC would have to be disabled and re-enabled on by thread basis.
 Couldn't we just require that the programmer keeps a reference to the memory
passed to C in a D variable? AFAIK python does it that way, but
 I don't use python enough to know if it causes serious problems there.

The issue is that a copying GC must be able to move the memory. It cannot do that if there are references it does not know about, because it has to update all references to the moved memory. If you just keep around a reference to the memory, your reference will be updated, but the reference from the C code will be left dangling.
Dec 19 2011
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 01:00 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:55:17 UTC, Timon Gehr wrote:
 On 12/19/2011 12:45 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:31:03 UTC, Timon Gehr wrote:
 On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
 You are right. I have had in mind a generational GC that uses a
 copying collector for the nursery as this is what most
 state-of-the-art VM GCs do.

believe. We need to work with what we have.

I disagree. Code that relies on other semantics would just have to use conservative GC.

Please elaborate on how you would hypothetically change D to accustom such changes. I am having trouble imagining such an implementation that would not have a considerable impact on D's performance.

If you have an union union X{ int x; int* y; } The compiler would just lay out x and y sequentially instead of at the same memory location. Alternatively, it could add a tag to each union. In case of passing GC memory to C functions, I would prefer to just disallow the C code to capture the reference, and to disable GC while the C function runs. (disabling and enabling GC is just a matter of modifying a counter somewhere, that should not visibly impact performance.) If C code was to capture a reference to the memory then it would probably take ownership anyway, which would necessitate to allocate the memory with malloc even with conservative GC. I am not sure what to do about void[] though.

OK... but what about the "generational GC that uses a copying collector for the nursery"?

I think we should attempt an implementation, find out where the main performance issues lie and then find a suitable set of trade-offs.
Dec 19 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-12-19 00:19, Adam Wilson wrote:
 On Sun, 18 Dec 2011 15:02:17 -0800, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 12/18/2011 11:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

It is an unilateral improvement if both options are kept open. I don't see a reason to cease support for the current GC model. Furthermore, a generational GC performs much better than a simple mark-sweep GC.

It seems to that we are really dancing around the potential solution. A pluggable GC interface that allowed the developer to choose the right GC for the task, or no GC at all. Imagine if all the developer had to do is set a compiler switch and the compiler automatically linked in the correct GC for the job. D could ship with a default GC then others could write different GC's based on different paradigms or their own needs. It would be a piece of work to get the interfaces right, but definitely worth it in the long run. Theoretically this would also give the developer the ability to link in multiple collectors and switch between them during program execution, provided the working set data was stored in a common format; although I have never heard of a use case for such a capability. I imagine that it could be useful if your application needs a low power state and you wanted to use a more power-efficient GC over a more precise one. I'm just theorizing here though. I'm sure their are technical snags that would prevent this from happening. I think supporting a NoGC environment is a good idea in the long run as their are cases to be made for manual memory management, but it also shouldn't be our first goal. GC's are were the value is.

It's already possible to plug different GC's at link time. -- /Jacob Carlborg
Dec 18 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 4:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

I think we can do a lot toward improving the footprint and performance of a precise GC while benefitting of its innate advantages. Andrei
Dec 18 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/11 5:22 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:13:03 UTC, Andrei Alexandrescu wrote:
 On 12/18/11 4:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

I think we can do a lot toward improving the footprint and performance of a precise GC while benefitting of its innate advantages.

Still, a more conservative GC will always outperform a more precise one in scanning speed.

I'm not sure. I seem to recall discussions with pathological cases when large regions of memory were scanned for no good reason.
 Without knowing the price, it would be unwise to jump
 into it without even considering the possibility of leaving a choice.

Sure.
 I am not against the idea, but I believe that more research is needed
 before rash decisions are taken. If the performance penalty turns out to
 be insignificant, then choice would be pointless. And if there will be a
 considerable performance gap, the "burden" of choice (compiler
 switch/boolean runtime setting + maintenance costs) could be worth it.

I ordered the GC book :o). Andrei
Dec 18 2011
next sibling parent Somedude <lovelydear mailmetrash.com> writes:
Le 19/12/2011 00:28, Andrei Alexandrescu a écrit :
 I ordered the GC book :o).
 
 
 Andrei

Also, at the risk of being redundant... http://www.scribd.com/doc/26102695/Garbage-Collection-in-JVM http://www.scribd.com/doc/59150636/C4-Continuously-Concurrent-Compacting-Collector
Dec 18 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/19/2011 07:50 PM, Vladimir Panteleev wrote:
 On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
 According to this wikipedia page
 http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC
 that is used by D, with some minor modifications of course.

I'm not sure if that's true... I believe that they both use the same basic idea, but AFAIK the D garbage collector is a D port of a C rewrite which was originally written for something else. The D GC has been optimized a lot since its first versions.

It would probably be interesting to test the mostly concurrent generational Boehm GC with D. I'd expect it to perform a lot better than the simple mark and sweep GC we have in druntime.
Dec 19 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/20/2011 08:33 PM, Robert Jacques wrote:
 On Mon, 19 Dec 2011 10:54:22 -0800, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 12/19/2011 07:50 PM, Vladimir Panteleev wrote:
 On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
 According to this wikipedia page
 http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC
 that is used by D, with some minor modifications of course.

I'm not sure if that's true... I believe that they both use the same basic idea, but AFAIK the D garbage collector is a D port of a C rewrite which was originally written for something else. The D GC has been optimized a lot since its first versions.

It would probably be interesting to test the mostly concurrent generational Boehm GC with D. I'd expect it to perform a lot better than the simple mark and sweep GC we have in druntime.

The Boehm GC isn't concurrent nor generational.

http://www.hpl.hp.com/personal/Hans_Boehm/gc/ 'The collector uses a mark-sweep algorithm. It provides incremental and generational collection under operating systems which provide the right kind of virtual memory support. [...]' http://www.hpl.hp.com/personal/Hans_Boehm/gc/gcdescr.html 'Generational Collection and Dirty Bits We basically use the concurrent and generational GC algorithm described in "Mostly Parallel Garbage Collection", by Boehm, Demers, and Shenker.'
Dec 20 2011
prev sibling next sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Yet, most high performance network elements from Ericson are running Erlang.
Able to process millions of phone calls per second.

What is D doing in real world?

This is the type of questions the community has to provide to naysayers.

--
Paulo

Somedude Wrote:

 Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
  I do not want to make a flame over D vs C++11.
 
 ...
  Once again, i'm not trying to make a holywar. I'm D lover myself. But a
 lot of people do not consider this benefits of D enough to shift to it
 as for my opinion and experience.
 
 

These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare.

Dec 18 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 20 Dec 2011 12:11:27 -0800, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 12/20/2011 08:33 PM, Robert Jacques wrote:
 On Mon, 19 Dec 2011 10:54:22 -0800, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 12/19/2011 07:50 PM, Vladimir Panteleev wrote:
 On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
 According to this wikipedia page
 http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC
 that is used by D, with some minor modifications of course.

I'm not sure if that's true... I believe that they both use the same basic idea, but AFAIK the D garbage collector is a D port of a C rewrite which was originally written for something else. The D GC has been optimized a lot since its first versions.

It would probably be interesting to test the mostly concurrent generational Boehm GC with D. I'd expect it to perform a lot better than the simple mark and sweep GC we have in druntime.

The Boehm GC isn't concurrent nor generational.

http://www.hpl.hp.com/personal/Hans_Boehm/gc/ 'The collector uses a mark-sweep algorithm. It provides incremental and generational collection under operating systems which provide the right kind of virtual memory support. [...]' http://www.hpl.hp.com/personal/Hans_Boehm/gc/gcdescr.html 'Generational Collection and Dirty Bits We basically use the concurrent and generational GC algorithm described in "Mostly Parallel Garbage Collection", by Boehm, Demers, and Shenker.'

Yes and no. Notice the very big conditional on those statements. The approach used in these papers is the similar to that of CDGC and have similar operating system limitations. Specifically, they don't work on Windows. But thank you for expanding my knowledge of GCs, as I thought that the virtual memory tricks came after Boehm.
Dec 20 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Mon, 19 Dec 2011 10:54:22 -0800, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 12/19/2011 07:50 PM, Vladimir Panteleev wrote:
 On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
 According to this wikipedia page
 http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC
 that is used by D, with some minor modifications of course.

I'm not sure if that's true... I believe that they both use the same basic idea, but AFAIK the D garbage collector is a D port of a C rewrite which was originally written for something else. The D GC has been optimized a lot since its first versions.

It would probably be interesting to test the mostly concurrent generational Boehm GC with D. I'd expect it to perform a lot better than the simple mark and sweep GC we have in druntime.

The Boehm GC isn't concurrent nor generational.
Dec 20 2011
prev sibling next sibling parent Andrew Wiley <wiley.andrew.j gmail.com> writes:
On Sun, Dec 18, 2011 at 6:19 AM, Alex R=F8nne Petersen
<xtzgzorex gmail.com> wrote:
 On 18-12-2011 12:45, Somedude wrote:
 Le 18/12/2011 12:13, Ruslan Mullakhmetov a =E9crit :
 =A0I do not want to make a flame over D vs C++11.

 ...
 =A0Once again, i'm not trying to make a holywar. I'm D lover myself. Bu=



 lot of people do not consider this benefits of D enough to shift to it
 as for my opinion and experience.

These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare.

On the other hand, Erlang's runtime is built around message-passing. I *extremely strongly doubt* that you can get similar performance out of message-passing in D, for two reasons: 1) The Erlang runtime is written in highly optimized C. 2) Erlang uses better garbage collection strategies than D. On point 1: Of course, you can write a message-passing implementation in =

 that's basically just glorified C, but that defeats the point of using D =

 the first place IMHO.

 On point 2: This is highly unlikely to change. It seems (from reading the
 NG's past discussions) that there is virtually no interest from the core
 devs of D to do The Right Thing to make precise garbage collection possib=

 sadly (and neither to make thread-local GC possible, and other such
 techniques (Erlang runs a separate GC per Erlang process, very
 successfully)).

 Yes, Erlang's emulator is not good in raw performance. It was never geare=

 for this (that's why it has easy interfacing with C for
 performance-sensitive code). However, things like HiPE improve the
 situation.

 All languages have their place, and Erlang is still going strong in the
 world of massive concurrency. I doubt D will be able to challenge this un=

 its GC issues have been resolved (and even then, D is nowhere near as
 convenient, lacking fault tolerance and location transparency; sometimes
 these traits are *much* more desirable than raw performance when dealing
 with concurrency on such high levels).

So I'm currently working on a message passing library in D that should scale better than std.concurrency currently does, and I went ahead and implemented this benchmark in it: http://www.krazykoding.com/2011/07/scala-actor-v-erlang-genserver.html Now, this benchmark doesn't really measure parallelism, but it does measure message passing speed. Some results: Erlang (OTP): 7.253 seconds, 413,602.903 per sec Erlang (receive): 5.486 seconds, 546,891.677 sec My Library: 3.021 seconds, 993,048.659 per sec Now, my library isn't complete enough for Github yet, and that will probably get worse before it's completed, but it does show that D can compete. This library supports closure-based actors (this benchmark was run on them), which gets us to Erlang's level in terms of how many actors can run. The other thing D has going here is that unlike Scala (which is known for its message passing implementations), we can statically guarantee the safety of data passed between threads and avoid all heap allocation if a message can be passed as a struct that isn't too large.
Dec 18 2011
prev sibling next sibling parent "jdrewsen" <jdrewsen nospam.com> writes:
On Sunday, 18 December 2011 at 17:19:33 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/11 6:19 AM, Alex Rønne Petersen wrote:
 On 18-12-2011 12:45, Somedude wrote:
 Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit :
 I do not want to make a flame over D vs C++11.




<snip>
 Walter and I are both interested in making the GC precise.

Wouldn't support for precise GC collection mean larger executable because type information is needed at runtime for precise collection? If so, could extra info be used to improve runtime reflection as well? /Jonas
Dec 18 2011
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu 
wrote:
 That is an interesting opportunity. At any rate, I am 100% 
 convinced precise GC is the only way to go, and I think I've 
 convinced Walter to a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.
Dec 18 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Vladimir Panteleev:

 Sacrificing something (performance, executable size) for 
 something else is not an unilateral improvement.

I presume they will add a very easy way (like a compilation switch? No need to modify druntime, or to recompile DMD, or to write your own GC) to switch back to less precise regime, or generally to avoid paying for things you don't want to pay. LDC compiler has pragmas to avoid specific typeinfos and moduleinfos, but it is a non portable thing and it's kind of low level. Bye, bearophile
Dec 18 2011
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:02:17 UTC, Timon Gehr wrote:
 It is an unilateral improvement if both options are kept open. 
 I don't see a reason to cease support for the current GC model.

I believe that currently the plan does not include providing this choice.
 Furthermore, a generational GC performs much better than a 
 simple mark-sweep GC.

Unless you change the way references work, generational and "precise" aspects of a GC are orthogonal. Also, D can't have a completely precise GC as long as it has unions and can pass managed memory to C code.
Dec 18 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Vladimir Panteleev:

 Also, D can't have a completely precise GC as long as it has unions

Despite C/D unions are untagged, in many cases there is some kind of manually managed tag (maybe stored elsewhere), or the programmer often knows (despite being sometimes wrong) what type is present inside the union at a given time. So I have suggested a standard optional method for D unions, named onGC, that is called by the GC and informs the GC what type is present inside the union. So if you keep a union that is able to contain a size_t and a pointer, the onGC is able to tell the GC if in a given moment the union contains a pointer to follow in its search for alive objects, or an integral number to ignore. Bye, bearophile
Dec 18 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 18 Dec 2011 15:02:17 -0800, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 12/18/2011 11:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

It is an unilateral improvement if both options are kept open. I don't see a reason to cease support for the current GC model. Furthermore, a generational GC performs much better than a simple mark-sweep GC.

It seems to that we are really dancing around the potential solution. A pluggable GC interface that allowed the developer to choose the right GC for the task, or no GC at all. Imagine if all the developer had to do is set a compiler switch and the compiler automatically linked in the correct GC for the job. D could ship with a default GC then others could write different GC's based on different paradigms or their own needs. It would be a piece of work to get the interfaces right, but definitely worth it in the long run. Theoretically this would also give the developer the ability to link in multiple collectors and switch between them during program execution, provided the working set data was stored in a common format; although I have never heard of a use case for such a capability. I imagine that it could be useful if your application needs a low power state and you wanted to use a more power-efficient GC over a more precise one. I'm just theorizing here though. I'm sure their are technical snags that would prevent this from happening. I think supporting a NoGC environment is a good idea in the long run as their are cases to be made for manual memory management, but it also shouldn't be our first goal. GC's are were the value is. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:13:03 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/11 4:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei 
 Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% 
 convinced
 precise GC is the only way to go, and I think I've convinced 
 Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

I think we can do a lot toward improving the footprint and performance of a precise GC while benefitting of its innate advantages.

Still, a more conservative GC will always outperform a more precise one in scanning speed. Without knowing the price, it would be unwise to jump into it without even considering the possibility of leaving a choice. I am not against the idea, but I believe that more research is needed before rash decisions are taken. If the performance penalty turns out to be insignificant, then choice would be pointless. And if there will be a considerable performance gap, the "burden" of choice (compiler switch/boolean runtime setting + maintenance costs) could be worth it.
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
 You are right. I have had in mind a generational GC that uses a 
 copying collector for the nursery as this is what most 
 state-of-the-art VM GCs do.

 ...

 We can change the way unions are layed out. The compiler can 
 mark GC memory passed to a C function as non-movable, or 
 passing GC memory to a C function can be made illegal if the GC 
 is enabled.

These changes are too invasive for the language at this point, I believe. We need to work with what we have.
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:31:03 UTC, Timon Gehr wrote:
 On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr wrote:
 You are right. I have had in mind a generational GC that uses 
 a
 copying collector for the nursery as this is what most
 state-of-the-art VM GCs do.

I believe. We need to work with what we have.

I disagree. Code that relies on other semantics would just have to use conservative GC.

Please elaborate on how you would hypothetically change D to accustom such changes. I am having trouble imagining such an implementation that would not have a considerable impact on D's performance.
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:28:16 UTC, Andrei Alexandrescu 
wrote:
 I'm not sure. I seem to recall discussions with pathological 
 cases when large regions of memory were scanned for no good 
 reason.

We need to factor the frequency of such cases, and their impact. Also, a more precise GC isn't the only solution to such problems - one alternative would be readily-available tools to diagnose memory problems. For example, Diamond (the toolkit I've written for D1) can display a graphical memory map with scannable areas being highlighted. It also allows you to easily find the pointer chain that prevents your object from being collected.
 I ordered the GC book :o).

A lot of GC research seems to be tied to VMs. D is a bit of an exception...
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:55:17 UTC, Timon Gehr wrote:
 On 12/19/2011 12:45 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:31:03 UTC, Timon Gehr wrote:
 On 12/19/2011 12:24 AM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:18:22 UTC, Timon Gehr 
 wrote:
 You are right. I have had in mind a generational GC that 
 uses a
 copying collector for the nursery as this is what most
 state-of-the-art VM GCs do.

point, I believe. We need to work with what we have.

I disagree. Code that relies on other semantics would just have to use conservative GC.

Please elaborate on how you would hypothetically change D to accustom such changes. I am having trouble imagining such an implementation that would not have a considerable impact on D's performance.

If you have an union union X{ int x; int* y; } The compiler would just lay out x and y sequentially instead of at the same memory location. Alternatively, it could add a tag to each union. In case of passing GC memory to C functions, I would prefer to just disallow the C code to capture the reference, and to disable GC while the C function runs. (disabling and enabling GC is just a matter of modifying a counter somewhere, that should not visibly impact performance.) If C code was to capture a reference to the memory then it would probably take ownership anyway, which would necessitate to allocate the memory with malloc even with conservative GC. I am not sure what to do about void[] though.

OK... but what about the "generational GC that uses a copying collector for the nursery"?
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 19 December 2011 at 00:07:47 UTC, Timon Gehr wrote:
 OK... but what about the "generational GC that uses a copying 
 collector
 for the nursery"?

I am not sure I get the question. The D specification already allows moving GC implementations. What is the issue you are thinking about?

How you would implement one with reasonable performance.
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 18 December 2011 at 23:19:08 UTC, Adam Wilson wrote:
 It seems to that we are really dancing around the potential 
 solution. A pluggable GC interface that allowed the developer 
 to choose the right GC for the task, or no GC at all. Imagine 
 if all the developer had to do is set a compiler switch and the 
 compiler automatically linked in the correct GC for the job. D 
 could ship with a default GC then others could write different 
 GC's based on different paradigms or their own needs. It would 
 be a piece of work to get the interfaces right, but definitely 
 worth it in the long run.

This pretty much already describes the current situation. This is the interface: https://github.com/D-Programming-Language/druntime/blob/master/src/core/memory.d#L19 You can install your own GC as a "proxy" (this is used for sharing the heap with shared objects): https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L33
 Theoretically this would also give the developer the ability to 
 link in multiple collectors and switch between them during 
 program execution, provided the working set data was stored in 
 a common format; although I have never heard of a use case for 
 such a capability.

It would probably be more practical to allow one GC to be configurable at runtime.
 I think supporting a NoGC environment is a good idea in the 
 long run as their are cases to be made for manual memory 
 management, but it also shouldn't be our first goal. GC's are 
 were the value is.

I believe there have been several attempts to start a no-GC runtime+stdlib (including one of mine), but they never got off the ground.
Dec 18 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 18 Dec 2011 15:55:18 -0800, Vladimir Panteleev  
<vladimir thecybershadow.net> wrote:

 On Sunday, 18 December 2011 at 23:28:16 UTC, Andrei Alexandrescu wrote:
 I'm not sure. I seem to recall discussions with pathological cases when  
 large regions of memory were scanned for no good reason.

We need to factor the frequency of such cases, and their impact. Also, a more precise GC isn't the only solution to such problems - one alternative would be readily-available tools to diagnose memory problems. For example, Diamond (the toolkit I've written for D1) can display a graphical memory map with scannable areas being highlighted. It also allows you to easily find the pointer chain that prevents your object from being collected.
 I ordered the GC book :o).

A lot of GC research seems to be tied to VMs. D is a bit of an exception...

Well, maybe we can do the research then, I love research projects myself and D would be a good vehicle to display the viability of native GC's, especially if we had different implementations to experiment with. Just because we aren't a university doesn't mean we can't do research. It seems to me that no one else is interested... -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
 It seems to me that no one else is interested...

Not true :)
Dec 18 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 18 Dec 2011 23:19:41 -0800, Vladimir Panteleev  
<vladimir thecybershadow.net> wrote:

 On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
 It seems to me that no one else is interested...

Not true :)

Hehe, I mean't outside the D community. I can think of five people off the top of my head inside the D community who are interested in it, myself excluded. ;-P -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 18 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 19 December 2011 at 07:27:14 UTC, Adam Wilson wrote:
 On Sun, 18 Dec 2011 23:19:41 -0800, Vladimir Panteleev 
 <vladimir thecybershadow.net> wrote:

 On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
 It seems to me that no one else is interested...

Not true :)

Hehe, I mean't outside the D community. I can think of five people off the top of my head inside the D community who are interested in it, myself excluded. ;-P

Ah. D'oh. Yeah, I think the thing closest to D is the Boehm GC (which has a good amount of research associated with it, I believe).
Dec 18 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 18 Dec 2011 23:43:04 -0800, Jacob Carlborg <doob me.com> wrote:

 On 2011-12-19 00:19, Adam Wilson wrote:
 On Sun, 18 Dec 2011 15:02:17 -0800, Timon Gehr <timon.gehr gmx.ch>  
 wrote:

 On 12/18/2011 11:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu  
 wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter  
 to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

It is an unilateral improvement if both options are kept open. I don't see a reason to cease support for the current GC model. Furthermore, a generational GC performs much better than a simple mark-sweep GC.

It seems to that we are really dancing around the potential solution. A pluggable GC interface that allowed the developer to choose the right GC for the task, or no GC at all. Imagine if all the developer had to do is set a compiler switch and the compiler automatically linked in the correct GC for the job. D could ship with a default GC then others could write different GC's based on different paradigms or their own needs. It would be a piece of work to get the interfaces right, but definitely worth it in the long run. Theoretically this would also give the developer the ability to link in multiple collectors and switch between them during program execution, provided the working set data was stored in a common format; although I have never heard of a use case for such a capability. I imagine that it could be useful if your application needs a low power state and you wanted to use a more power-efficient GC over a more precise one. I'm just theorizing here though. I'm sure their are technical snags that would prevent this from happening. I think supporting a NoGC environment is a good idea in the long run as their are cases to be made for manual memory management, but it also shouldn't be our first goal. GC's are were the value is.

It's already possible to plug different GC's at link time.

Cool, I didn't know that. Thanks for the heads up, i'm still working hard at understanding D's rather large set of capabilities. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 19 2011
prev sibling next sibling parent "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 18 Dec 2011 23:29:38 -0800, Vladimir Panteleev  
<vladimir thecybershadow.net> wrote:

 On Monday, 19 December 2011 at 07:27:14 UTC, Adam Wilson wrote:
 On Sun, 18 Dec 2011 23:19:41 -0800, Vladimir Panteleev  
 <vladimir thecybershadow.net> wrote:

 On Monday, 19 December 2011 at 07:11:10 UTC, Adam Wilson wrote:
 It seems to me that no one else is interested...

Not true :)

Hehe, I mean't outside the D community. I can think of five people off the top of my head inside the D community who are interested in it, myself excluded. ;-P

Ah. D'oh. Yeah, I think the thing closest to D is the Boehm GC (which has a good amount of research associated with it, I believe).

Ahh yes, Boehm. According to this wikipedia page http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also the GC that is used by D, with some minor modifications of course. If this is really the case then we really would be pushing the boundaries of GC research. My finger has been hovering over the buy link for the GC book for days ... I'm getting closer. I may not be much of compiler guy, but memory management and GC's have been a subject near and dear to my heart for some years now. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Dec 19 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 19 December 2011 at 08:28:52 UTC, Adam Wilson wrote:
 According to this wikipedia page 
 http://en.wikipedia.org/wiki/Boehm_garbage_collector it is also 
 the GC that is used by D, with some minor modifications of 
 course.

I'm not sure if that's true... I believe that they both use the same basic idea, but AFAIK the D garbage collector is a D port of a C rewrite which was originally written for something else. The D GC has been optimized a lot since its first versions.
Dec 19 2011
prev sibling next sibling parent reply "Froglegs" <lugtug gmail.com> writes:
  I've only recently tried D out, but what I'd like to see..

-GC being truly optional
-being able to specify if the standard library should use GC or 
not, perhaps with allocators
-if D is going to have a GC, then a precise, compacting one would 
be cool, but not if it gets in the way of making the GC optional


  One thing I'm not sure about, D classes are virtual by default, 
but if you mark all functions as final does the class still 
contain a VFP or any other cruft?
  Also why are class functions virtual by default? My experience 
in C++ is that I rarely use virtual, so I don't really understand 
why that is the default.
Dec 20 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Froglegs:

   One thing I'm not sure about, D classes are virtual by default, 
 but if you mark all functions as final does the class still 
 contain a VFP or any other cruft?

Even D final classes, that do not have virtual methods, have a pointer to virtual table. It's used to know what class the instance is (for reflection too and for the destructor).
   Also why are class functions virtual by default? My experience 
 in C++ is that I rarely use virtual, so I don't really understand 
 why that is the default.

Maybe because D OO design copies Java OO design a lot. But even in C# methods are not virtual on default. Bye, bearophile
Dec 20 2011
prev sibling next sibling parent reply "Froglegs" <lugtug gmail.com> writes:
 The class will still have a vptr. The vtable will contain only 
 the type info.

No way to disable type info(like in most C++ compilers you can disable RTTI)? I get that GC might want it, but if I disable GC why would I want type info? I saw that D is planning to make the standard containers into classes with final methods, why do this instead of using structs if it bloats each instance of the container?
 Some aspects of D will _always_ require a GC or they won't 
 work. Array concatenation would be a prime example. I believe 
 that delegates are another major example. I think that scoped 
 delegates avoid the problem, but any that require closures do 
 not. Other things be done but become risky - e.g. slicing 
 arrays (the GC normally owns the memory such that all dynamic 
 arrays are slices and none of them own their memory, so slicing 
 manually managed memory gets dicey).

The array concatenation requiring GC I get, but why does a delegate require it? This link says D allocates closures on the heap http://en.wikipedia.org/wiki/Anonymous_function#D I don't really get why, C++ lambda works well(aside from broken lack of template lambda's) and do not require heap usage, even binding it to std::function can generally avoid it if it doesn't exceed the SBO size
Dec 20 2011
parent jerro <nomail nomail.com> writes:
 The array concatenation requiring GC I get, but why does a 
 delegate require it?

If you really want a stack allocated delegate, you could use something like: import std.stdio, std.traits; struct DelegateWrapper(alias fun, Args...) { Args args; private auto f(ParameterTypeTuple!fun[Args.length..$] otherArgs) { return fun(args, otherArgs); } auto dg() { return &f; } } auto delegateWrapper(alias fun, A...)(A a) { return DelegateWrapper!(fun,A)(a); } void main() { static test (int a, int b, int c) { writeln(a, b, c); } auto b = delegateWrapper!(test)(1,2); b.dg()(3); }
Dec 20 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 20 December 2011 at 11:17:32 UTC, Froglegs wrote:
 The class will still have a vptr. The vtable will contain only 
 the type info.

No way to disable type info(like in most C++ compilers you can disable RTTI)? I get that GC might want it, but if I disable GC why would I want type info?

It's used for casts and other language/library features depending on typeid. There's also the standard Object virtual methods: toString (used e.g. when passing objects to writeln), toHash, opEquals, opCmp (used in associative arrays, array sorting).
Dec 20 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 20 December 2011 at 11:17:32 UTC, Froglegs wrote:
 The array concatenation requiring GC I get, but why does a 
 delegate require it?

 This link says D allocates closures on the heap

 http://en.wikipedia.org/wiki/Anonymous_function#D

 I don't really get why, C++ lambda works well(aside from broken 
 lack of template lambda's) and do not require heap usage, even 
 binding it to std::function can generally avoid it if it 
 doesn't exceed the  SBO size

C++ "closures" do not allow you to maintain a reference to the context after the function containing said context returns. Instead, C++ allows you to choose between copying the variables into the lambda instance, or referencing them (the references may not "escape"). The compiler may or may not enforce correct uses of reference captures. In contrast, D's approach is both intuitive (does not copy variables) and safe (conservatively allocates on the heap), with the downside of requiring the context to be garbage-collected.
Dec 20 2011
prev sibling next sibling parent reply "Froglegs" <lugtug gmail.com> writes:
 C++ "closures" do not allow you to maintain a reference to the 
 context after the function containing said context returns. 
 Instead, C++ allows you to choose between copying the variables 
 into the lambda instance, or referencing them (the references 
 may not "escape"). The compiler may or may not enforce correct 
 uses of reference captures. In contrast, D's approach is both 
 intuitive (does not copy variables) and safe (conservatively 
 allocates on the heap), with the downside of requiring the 
 context to be garbage-collected.

Ah, makes sense now, thanks. Still it seems like a case of "you pay for what you don't use", and seems like a real downer for adopting D since you loose the ability to use lambda's without having the GC shoved down your throat(wouldn't be so bad if the D GC was known for performance, but everything I've read indicates it is quite slow).
Dec 20 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Froglegs:

 Still it seems like a case of "you pay for what you don't use",

That's a design rule for C++, but D is a bit different :-) Often in D there are ways to not pay what you don't use, but you have to ask for them. If you don't ask for those ways, you usually need to pay a little, and you get a program that's safer or more easy to debug. So for D the rule is more like "Safety on default, and unsafe (and cheap) on request".
 and seems like a real downer for adopting D since you loose the 
 ability to use lambda's without having the GC shoved down your 
 throat

In theory in D there is a way to use (at the usage site) static delegates, that don't allocate a closure on the heap. I don't know if now this feature is fully correctly implemented, but if not it's planned. Bye, bearophile
Dec 20 2011
prev sibling next sibling parent "Froglegs" <lugtug gmail.com> writes:
 D's pass-down lambdas do not need memory allocation. As far as 
 I remember none of std.algorithm's use of lambda allocates 
 memory.

 Andrei

Oh cool, I like that
Dec 20 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Sun, 18 Dec 2011 15:29:23 -0800, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 Unions will be conservative. The golden standard is that SafeD can't use
 them or anything that forces conservative approaches.


 Andrei

Is there a strong rational for a conservative approach to unions? Why not simply set the GC pointer flag bit every time a union is assigned to? Given that there are fully precise GC implementations for C, why should D aim for something less?
Dec 20 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Sun, 18 Dec 2011 15:28:16 -0800, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 On 12/18/11 5:22 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 23:13:03 UTC, Andrei Alexandrescu wrote:
 On 12/18/11 4:53 PM, Vladimir Panteleev wrote:
 On Sunday, 18 December 2011 at 20:32:18 UTC, Andrei Alexandrescu wrote:
 That is an interesting opportunity. At any rate, I am 100% convinced
 precise GC is the only way to go, and I think I've convinced Walter to
 a good extent as well.

Sacrificing something (performance, executable size) for something else is not an unilateral improvement.

I think we can do a lot toward improving the footprint and performance of a precise GC while benefitting of its innate advantages.

Still, a more conservative GC will always outperform a more precise one in scanning speed.

I'm not sure. I seem to recall discussions with pathological cases when large regions of memory were scanned for no good reason.

Scanning speed is proportional to the size of the live heap, which will always be larger for conservative collectors. So while conservative collectors are faster per byte, they have to scan more bytes. There's been a bunch of research into precise GCs for C, as graduate students love hard problems. There are several solutions out there currently; the one I stumbled upon is called Magpie. The associated thesis has some pretty in depth performance analyses. There are also some follow up papers from later students and more real world tests of precise vs conservative vs manual.
Dec 20 2011
prev sibling parent d coder <dlang.coder gmail.com> writes:
--90e6ba6155222b3dcc04b5a31e6f
Content-Type: text/plain; charset=ISO-8859-1

 D's pass-down lambdas do not need memory allocation. As far as I remember
 none of std.algorithm's use of lambda allocates memory.

On that front Andrei, did you get a chance to discuss my request to make std.algorithm more usable with Walter? http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=140807 - Regards --90e6ba6155222b3dcc04b5a31e6f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margi= n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><br></d= iv></div> D&#39;s pass-down lambdas do not need memory allocation. As far as I rememb= er none of std.algorithm&#39;s use of lambda allocates memory.<span><font c= olor=3D"#888888"><br><br></font></span></blockquote><div><br></div><div> <br></div><div>On that front Andrei, did you get a chance to discuss my req= uest to make std.algorithm more usable with Walter?</div><div> <br></div><div>=A0<a href=3D"http://www.digitalmars.com/webnews/newsgroups.= php?art_group=3Ddigitalmars.D&amp;article_id=3D140807">http://www.digitalma= rs.com/webnews/newsgroups.php?art_group=3Ddigitalmars.D&amp;article_id=3D14= 0807</a></div> <div><br></div><div>- Regards</div><div><br></div></div> --90e6ba6155222b3dcc04b5a31e6f--
Jan 03 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, December 18, 2011 04:09:21 Ruslan Mullakhmetov wrote:
   I want to ask you about D future

It will be years before we seriously start looking at D3, and while there are ideas for what we might like to do with it, it's far too early to say what's likely to happen with it. D2 needs to be fully stabilized and be in general use for a while before we really look at expanding it into D3. We really need to work at making D2 a success before we worry about where we're going next. Also, many of the best changes to D3 won't be evident until D2 has been used enough for its problems to become evident. D1 and D2 manage to improve on C++'s problems as well as they do precisely because we know what they are. We don't really know what D2's primary problems are yet, and that will take time. Also, while D3 may be years off, it could be that after D2 has stabilized more, we'll add new features that are backwards compatible. So, just because D3 is years off does not necessarily mean that D2 is static. In addition, many things can be done in libraries without needing to add anything to the language, so what you can do with D2 will continue to improve, even if the language itself doesn't change much. Regardless, the main focus right now is in stabilizing dmd and fleshing out the standard library, not in creating a new version of D with new features. It's too early for that. - Jonathan M Davis
Dec 17 2011
parent Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
 It will be years before we seriously start looking at D3


Absolutely. Just wanted to share with my thoughts. When i wrote to NG i do not intend to say "Let do it right know", instead wanted to say "When D3 would be considered..." Any good movement as i hope D is ought to have some global view and global goals. I think, new paradigm would benefit D. Also as already mentioned else thread this can be used like marketing trick. On 2011-12-18 02:26:51 +0000, Jonathan M Davis said:
 On Sunday, December 18, 2011 04:09:21 Ruslan Mullakhmetov wrote:
 I want to ask you about D future

It will be years before we seriously start looking at D3, and while there are ideas for what we might like to do with it, it's far too early to say what's likely to happen with it. D2 needs to be fully stabilized and be in general use for a while before we really look at expanding it into D3. We really need to work at making D2 a success before we worry about where we're going next. Also, many of the best changes to D3 won't be evident until D2 has been used enough for its problems to become evident. D1 and D2 manage to improve on C++'s problems as well as they do precisely because we know what they are. We don't really know what D2's primary problems are yet, and that will take time. Also, while D3 may be years off, it could be that after D2 has stabilized more, we'll add new features that are backwards compatible. So, just because D3 is years off does not necessarily mean that D2 is static. In addition, many things can be done in libraries without needing to add anything to the language, so what you can do with D2 will continue to improve, even if the language itself doesn't change much. Regardless, the main focus right now is in stabilizing dmd and fleshing out the standard library, not in creating a new version of D with new features. It's too early for that. - Jonathan M Davis

-- BR, Ruslan Mullakhmetov
Dec 18 2011
prev sibling next sibling parent Trass3r <un known.com> writes:
Am 18.12.2011, 01:09 Uhr, schrieb Ruslan Mullakhmetov <tiabaldu gmail.com>:

 Hi all,

   I want to ask you about D future, i mean next big iteration of D and  
 propose some new feature, agent-based programming. Currently, after  
 introducing C++11 i see the only advantages of D over C++11 except  
 syntax sugare is  garbage collector and modules.

Sorry, but you clearly haven't used D for anything serious. You still have plenty of things like static if, compile-time reflection and CTFE which is basically _impossible_ in C++. And even if you consider C++11, it's just awful compared to D. I constantly wonder why they had to invent such a horrible syntax. That has nothing to do with syntax sugar. D just got it straight.
Dec 18 2011
prev sibling next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
On Sun, 18 Dec 2011 04:09:21 +0400, Ruslan Mullakhmetov wrote:

   I want to ask you about D future, i mean next big iteration of D and
 propose some new feature, agent-based programming. Currently, after
 introducing C++11 i see the only advantages of D over C++11 except
 syntax sugare is  garbage collector and modules.

I do not think D (as a language) should be modified to support agent- based programming. A Phobos module (or package) would do. I seriously do not see what language changes we need for agent-based programming. :)
Dec 20 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, December 20, 2011 17:58:30 Timon Gehr wrote:
 On 12/20/2011 05:36 PM, Jonathan M Davis wrote:
 On Tuesday, December 20, 2011 15:49:34 Timon Gehr wrote:
 2. Dynamic binding is a core concept of OOP. A language that does not
 support dynamic binding does not support OOP. A program that does not
 use dynamic binding is not object oriented. What is to disagree with?

I don't agree with that either. You don't need polymorphism for OOP. It's quite restricted without it, but you can still program with objects even if you're restricted to something like D's structs, so you're still doing OOP. - Jonathan M Davis

No. That is glorified procedural style. 'Objects' as in 'OOP' carry data and _behavior_, structs don't (except if you give them some function pointers, but that is just implementing poor man's polymorphism.) Having some kind of dynamic execution model is a requirement for OOP. There are no two ways about it.

Well, I completely disagree. The core of OOP is encapsulating the data within an object and having functions associated with the object itself which operate on that data. It's about encapsulation and tying the functions to the type. Polymorphism is a nice bonus, but it's not required. I'd say that any language which is really trying to do OOP should definitely have polymorphism or it's going to have pretty sucky OOP, but it can still have OOP. - Jonathan M Davis
Dec 20 2011