www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Concurrency.

reply Debdata <debdatta.basu91 gmail.com> writes:
Hi,

I have been evaluating D for the past week, and have some concerns regarding
the No Default sharing rule. I have been reading the D book by Andrei as
well. This particular feature has been very confusing to me as I come from the
c++ world.

I agree that message passing and resource hiding are a great way to go for a
lot of cases, but there are an equally large (Larger?) number of cases
that would benefit from global sharing. Especially when threading for
performance rather than convenience. The cutting edge of high performance
threading is really the task stealing approach. See intel's TBB, cilk, etc.
Implementing things like that would become unnecessarily verbose as most
things will be shared and we have to keep tagging.

Unless I am missing something. :D

-Debdatta Basu
Nov 27 2011
next sibling parent reply Jude <10equals2 gmail.com> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/27/2011 11:51 PM, Debdata wrote:
 Hi,
 
 I have been evaluating D for the past week, and have some concerns
 regarding the No Default sharing rule. I have been reading the D
 book by Andrei as well. This particular feature has been very
 confusing to me as I come from the c++ world.
 
 I agree that message passing and resource hiding are a great way to
 go for a lot of cases, but there are an equally large (Larger?)
 number of cases that would benefit from global sharing. Especially
 when threading for performance rather than convenience. The cutting
 edge of high performance threading is really the task stealing
 approach. See intel's TBB, cilk, etc. Implementing things like that
 would become unnecessarily verbose as most things will be shared
 and we have to keep tagging.
 
 Unless I am missing something. :D
 
 -Debdatta Basu

Just wanted to make sure you know, there is a keyword to force global storage: __gshared Also D has std.parallelism, which seems to implement what you are talking about... Would this not be considered [the same as|close enough to] task stealing? wikipedia seems to think so... TBB is a collection of components for parallel programming: Basic algorithms: parallel_for, parallel_reduce, parallel_scan void main() { // Parallel reduce can be combined with std.algorithm.map to interesting // effect. The following example (thanks to Russel Winder) calculates // pi by quadrature using std.algorithm.map and TaskPool.reduce. // getTerm is evaluated in parallel as needed by TaskPool.reduce. // // Timings on an Athlon 64 X2 dual core machine: // // TaskPool.reduce: 12.170 s // std.algorithm.reduce: 24.065 s immutable n = 1_000_000_000; immutable delta = 1.0 / n; real getTerm(int i) { immutable x = ( i - 0.5 ) * delta; return delta / ( 1.0 + x * x ); } immutable pi = 4.0 * taskPool.reduce!"a + b"( std.algorithm.map!getTerm(iota(n))); } But I must admit that I really don't have much experience in this regard and am more interested in hearing your responses. =P -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJO0yqrAAoJENcHIWLyQiSljswIAOQUG/jxDD6XjplV8vhhxkWQ plxpAVZWv1Oi9TBpK6QB9PMP9qka+BGAPMmEFyoKv4EozcJHxYda3gfqZm+jKdEb j2rcE/5ZSWvgcr2NcPiwhYIraYGGb4TqzhhHcN8mbebze3/Lnjf1tWpT9X9QEDLG eSe/wNMagYHTYAzv1Q1ubbYzawyeyNPclPDDbEEdTVUsJhl02QxKFUxJTX0M2N6a mHhd6DxOf9Pss9iZfjmegLe3Sz5/xxhZfwrYEisTirvRpElNOvzc2JttznFa7wEe wVgUVQ844IHvcg06ecY8XFVbIbPUJMia5J6JGofcr4JmLbEDcXBydH33ffa4rAI= =APH6 -----END PGP SIGNATURE-----
Nov 27 2011
parent reply Debdatta <debdatta.basu91 gmail.com> writes:
Jude,

Yes, it is similar, but not exactly the same. The difference is in  the way
tasks are managed. There is no global task queue, and each thread has
its own. Each thread operates on its own task queue, and each task can
recursively add tasks to the owning thread's head. The head exclusively
belongs to the thread, hence no locking is required. When a thread is out of
tasks, it steals a task from the tail of another queue in a round robin
fashion. In this case, locking is required. :D It has substantially lower
overhead than global task pools.

This allows us to efficiently map recursive algorithms with non uniform work
loads to the task queuing scheme.

But that's not important to the current discussion, as the language mechanisms
that allow us to implement the current task pool can also be used for
task stealing based approaches.

What I would like to know, is how does the no default sharing idiom apply to
that? I have seen the examples already, and its not particularly clear
to me how this could be done without explicitly adding and managing  shared
tags.

- Debdatta
Nov 27 2011
parent reply Jude <10equals2 gmail.com> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/28/2011 01:05 AM, Debdatta wrote:
 Jude,
 
 Yes, it is similar, but not exactly the same. The difference is in
 the way tasks are managed. There is no global task queue, and each
 thread has its own. Each thread operates on its own task queue, and
 each task can recursively add tasks to the owning thread's head.
 The head exclusively belongs to the thread, hence no locking is
 required. When a thread is out of tasks, it steals a task from the
 tail of another queue in a round robin fashion. In this case,
 locking is required. :D It has substantially lower overhead than
 global task pools.
 
 This allows us to efficiently map recursive algorithms with non
 uniform work loads to the task queuing scheme.
 
 But that's not important to the current discussion, as the language
 mechanisms that allow us to implement the current task pool can
 also be used for task stealing based approaches.
 
 What I would like to know, is how does the no default sharing idiom
 apply to that? I have seen the examples already, and its not
 particularly clear to me how this could be done without explicitly
 adding and managing  shared tags.
 

Debdatta, Nice response, very informative. Now I'm intrigued. - -Jude -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJO0zXzAAoJENcHIWLyQiSlIjgH/jcRcxoikWeKp80ab6iNROua d8ByGCe+kOJIUmx0zToqEbaJAQ+D8+1W6uAUPp9Hu+946LYtDDs+eh5jdVcUBcCj blRGUFfR1gstzVDjJlYocDc2AjKSbo4mlti+XEGCTPb2rWm0ykDEcobpKf2LJx5A G5gbB1fRKtFcYUOeWapZc94bYlqOege+KH0DaQ90Li7wzu7SGlcACaQ3VW7D8AyM zKJBpODJw7qy5WKQLVsCH5pQbfwfQ1J1JV8tY91+cpnjrv+A713EsGaM3P0QfQxa fFVHE/eVlrhtHKUxgVrQ5nW3GEATs/SB40jOM+JmZWcA8b6QSHgiVyurpslSl30= =Ic1l -----END PGP SIGNATURE-----
Nov 27 2011
next sibling parent reply Debdatta <debdatta.basu91 gmail.com> writes:
Jude,

Glad to hear that you found my post informative. Anyway, from what I
understand, shared is infectious. However, lets say we have the following:

class A
{
    .......

};
shared class B
{
    constructor()
    {
       myA = new A;
    }
A myA;
}

This will create a problem as the myA is shared, whereas A is not. Hence, we
have to tag A to be shared as well. This will lead to most things
marked shared and will cause confusion. Not to mention that default sharing is
the convention most programmers have been born with, and thats hard
to change in a person. :D

However, in default shared languages, hiding types is easy. For instance, if I
wanted a thread with a completely local context, I would do this:

class myLocalClass : Thread
{
public:
//whatever I want accessible by other classes go here.

void run();

private:

//whatever I want to make specific to my thread will be located here.

}

This seems to be a more elegant solution to the problem. Any thoughts?

-Debdatta
Nov 28 2011
parent reply Debdatta <debdatta.basu91 gmail.com> writes:
Let me get this straight. Instances are shared... and marking a class shared
marks all its members shared? If what you said were true, it would be
trivial to instantiate a class as both shared and unshared. Without any
duplication.

class A
{
///
}

shared A sharedA = new A;
A unsharedA = new A;

I don't can you elaborate your last post?

Thanks.
Debdatta
Nov 28 2011
next sibling parent reply Debdatta <debdatta.basu91 gmail.com> writes:
I get it now. Totally. This is like a mechanism to enable shared-correctness.
Just like const correctness in c++. :D

This does allow for some nice static runtime checks to ensure that the code is
free of races. But I somehow still like the deafult is shared idiom.
To be fair i never use const correctness features in c++ either. :P

In my (limited) experience, involving mostly threading for performance, sharing
is the norm.( Take a look at .NET's task parallel library, or
intel's TBB.)

the point i am trying to make is that restricting access to thread specific
data is far simpler than keeping track of what is shared. this is
because shared data is usually all over the place when you want to thread for
performance. However, thread local data resides in one place( the
thread class mostly).

Look at the example I gave a few posts back:

class MyThread : Thread
{

public:

void run;


private:

//any data i put here can only be seen by the current thread.

}

Keeping this in mind, whats the rationale for no default sharing?

-Debdatta Basu
Nov 28 2011
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-11-28 11:48:17 +0000, Debdatta <debdatta.basu91 gmail.com> said:

 I get it now. Totally. This is like a mechanism to enable 
 shared-correctness. Just like const correctness in c++. :D
 
 This does allow for some nice static runtime checks to ensure that the 
 code is free of races. But I somehow still like the deafult is shared 
 idiom.
 To be fair i never use const correctness features in c++ either. :P
 
 In my (limited) experience, involving mostly threading for performance, 
 sharing is the norm.( Take a look at .NET's task parallel library, or
 intel's TBB.)
 
 the point i am trying to make is that restricting access to thread 
 specific data is far simpler than keeping track of what is shared. this 
 is
 because shared data is usually all over the place when you want to 
 thread for performance. However, thread local data resides in one 
 place( the
 thread class mostly).
 
 Look at the example I gave a few posts back:
 
 class MyThread : Thread
 {
 
 public:
 
 void run;
 
 
 private:
 
 //any data i put here can only be seen by the current thread.
 
 }
 
 Keeping this in mind, whats the rationale for no default sharing?

Actually, the private data of your thread object could be used by other threads if the MyThread object is accessible from another thread and someone calls a method on it. Or you might have a MyThread object instance messing with the variables of another instance. Hopefully the methods of the MyThread object would be careful about not sharing the private data with other threads, but that's no guaranty. Also, you'll likely call external functions from your thread's main function: how do you know they are thread-safe, that they aren't using mutable global variables? Or that they aren't storing their input data elsewhere? With the no default sharing rule, they'll always be thread-safe (unless they circumvent the type system), both in their global state and with the data you provide to them. Not having default sharing is about providing guaranties, it's about making thread-safety checkable by the compiler. The implementation we have is not perfect (it hampers a couple of patterns), but hopefully that can be improved. Also take note of this: Private isn't the same in D as in C++. Private means module-private: D won't prevent other code from accessing MyThread's private members as long as this other code is in the same module. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 28 2011
parent reply Debdatta <debdatta.basu91 gmail.com> writes:
 Michael,

Thanks for clearing that up. Your post was very informative. :)

 Actually, the private data of your thread object could be used by other
 threads if the MyThread object is accessible from another thread and
 someone calls a method on it.


Absolutely! However I presume that any self respecting programmer will not manipulate shared data without planning it, or at least thinking about it. :D Any self respecting coder would know about memory boundaries, and not arbitrarily modify shared data.
Not having default sharing is about providing guaranties, it's about
making thread-safety checkable by the compiler.


No piece of code can absolutely guarantee threading errors. When you start a thread you are most likely doing it because you want to manipulate shared data. If you wanted complete memory isolation, with the only means of communication being messages, you would create processes, not threads. I understand that this is an over generalization, and the current no default sharing idiom provides less isolation than processes, yo get the drift. :D Threads within a process, by definition(in this case the os defines them), operate within a shared memory space. I don't understand the need to add an extra level of complexity in the code, as well as the compiler, by abstracting something thats inherant to the os. Its like const correctness. You can write more concise, perfectly working programs without it. and all it gives you is an extra check on whether a function will modify something, which I, as a programmer, already know. You may interpret my comments as me not wanting to give up control to the compiler. In that sense, even a GC is abstracting something thats inherant to the os. :D But the GC gives a huge return on investment, and I don't see how the no default sharing rule does. -Debdatta Basu.
Nov 28 2011
next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
 
Not having default sharing is about providing guaranties, it's about
making thread-safety checkable by the compiler.


No piece of code can absolutely guarantee threading errors. When you start

Threading errors and thread-safety checks are different things... It is pretty much possible to do thread-safety checks perhaps not all, but compiler tries what it can do to "shout" back at developer when an error is made. :)
Nov 28 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/28/2011 01:57 PM, Debdatta wrote:
  Michael,

 Thanks for clearing that up. Your post was very informative. :)

 Actually, the private data of your thread object could be used by other
 threads if the MyThread object is accessible from another thread and
 someone calls a method on it.


Absolutely! However I presume that any self respecting programmer will not manipulate shared data without planning it, or at least thinking about it. :D Any self respecting coder would know about memory boundaries, and not arbitrarily modify shared data.

Any self respecting coder only writes correct code. That is why there are never software bugs.
 Not having default sharing is about providing guaranties, it's about
 making thread-safety checkable by the compiler.



Nope, it is mostly about providing safe, efficient and convenient _defaults_ (that is why it is called no sharing _by default_). In C#/Java you have those three options: 1. Your variables are shared, but not that much, because you have no clue when or if at all or in what order changes to a variable in one thread get propagated to other threads. This is the default, of course! 2. Your variables are shared, and the language guarantees sequential consistency of the memory location across threads. To get that you need to annotate your variables with volatile. That makes sense because variables that are not intended to be shared would suffer from a severe and pointless performance hit if they would have to behave the same. Now, well then those should not be shared at all, right? 3. Thread Local Storage. This is the most obscure of the three. In D you have those three options: 1. Your variables are thread local. This guarantees some nice properties, like code that was intended to run in a single-threaded environment when it was written will behave nicely in a multi-threaded environment too, because every thread gets a copy of all global data. This is a sane default. 2. Your variables are shared and the language guarantees sequential consistency (not implemented in DMD yet afaik). This is the 'shared' type modifier. shared is transitive (as opposed to volatile in C#/Java), but if you feel like it, you can just cast away shared. (preferably you should know that it is safe, but as I understand that is the kind of stuff you enjoy a lot.) 3. Your variables are shared, but there is no sequential consistency. This is __gshared. If you have a closer look, you will see that there is a 1 to 1 feature correspondence there. Nothing to complain about.
 No piece of code can absolutely guarantee threading errors. When you start a
thread you are most likely doing it because you want to manipulate
 shared data. If you wanted complete memory isolation, with the only means of
communication being messages, you would create processes, not threads.
 I understand that this is an over generalization, and the current no default
sharing idiom provides less isolation than processes, yo get the drift.
 :D

 Threads within a process, by definition(in this case the os defines them),
operate within a shared memory space.

As soon as those threads run on different cores with separate deeply nested caching hierarchies, that is just an illusion established by message passing on the hardware level.
 I don't understand the need to add
 an extra level of complexity in the code, as well as the compiler, by
abstracting something thats inherant to the os.

There is no extra level of complexity. On the contrary. TLS by default is a very easy concept and it accurately reflects the ideal mental model for the hardware. The only thing that gets added is a tiny bit of explicitness.
 Its like const correctness. You can write more concise, perfectly working
programs without it. and all it gives you is an extra check on whether a
 function will modify something, which I, as a programmer, already know.

Nope, it does not do that. In C++, const correctness means that you cannot call a non-const method on a const reference and that const methods cannot assign to fields of the 'this' reference (unless they are marked as 'mutable', that is). And that is it. D const can provide some guarantees, especially in pure function signatures that enable some compiler optimizations. There is also 'immutable', that gives much stronger guarantees than const. Furthermore shared is not like const correctness because it (should) actually affect the generated code.
 You may interpret my comments as me not wanting to give up control to the
 compiler. In that sense, even a GC is abstracting something thats inherant to
the os. :D But the GC gives a huge return on investment, and I don't
 see how the no default sharing rule does.

Nobody is giving up control here. You can use type casts to make the compiler shut up, or use __gshared data.
Nov 28 2011
next sibling parent reply Debdatta <debdatta.basu91 gmail.com> writes:
1. Your variables are thread local. This guarantees some nice
properties, like code that was intended to run in a single-threaded
environment when it was written will behave nicely in a multi-threaded
environment too, because every thread gets a copy of all global data.
This is a sane default.


Now I understand your reasoning a lot better. :D this is one case where D's defaults will come very handy. Its very different from what I have seen so far, so maybe its a culture shock.
There is one to one feature correspondence.


Of course. That is obvious.:D It just seemed counter intuitive to have every variable I declare to be thread local. Will experiment with it some more and see if I can get used to the concept. You seem to have very good reasons to prefer it, and I hope I can see them too. :D -Debdatta
Nov 28 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/28/2011 07:41 PM, Debdatta wrote:
 1. Your variables are thread local. This guarantees some nice
 properties, like code that was intended to run in a single-threaded
 environment when it was written will behave nicely in a multi-threaded
 environment too, because every thread gets a copy of all global data.
 This is a sane default.


Now I understand your reasoning a lot better. :D this is one case where D's defaults will come very handy. Its very different from what I have seen so far, so maybe its a culture shock.

Probably that is the case. =)
 There is one to one feature correspondence.


Of course. That is obvious.:D It just seemed counter intuitive to have every variable I declare to be thread local. Will experiment with it some more and see if I can get used to the concept. You seem to have very good reasons to prefer it, and I hope I can see them too. :D -Debdatta

Nov 28 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, November 28, 2011 18:41:46 Debdatta wrote:
There is one to one feature correspondence.


Of course. That is obvious.:D It just seemed counter intuitive to have every variable I declare to be thread local. Will experiment with it some more and see if I can get used to the concept. You seem to have very good reasons to prefer it, and I hope I can see them too. :D

It's only counter-intuitive because most languages share everything by default. Well-written multi-threaded code already separates everything such that threads communicate only at key points in the program, so while it's atypical to make much thread-local in other languages, good programs in other lanuages generally wouldn't care if most of their variables became thread- local. D just takes it to the next level by actually making them thread-local and enforcing it with the type system. So now, each of those points where threads are supposed to communicate either use message passing or shared, and the rest of the program is protected from having its pieces stomp on each other unless you go out of your way to make it happen. Ultimately, D's basic approach makes more sense IMHO, but it's just not what has typically been done in other languages, so it requires mental shift, but since most good programs essentially already follow D's model in other languages (just without usually actually making the data thread-local), ultimately, it's not really all that big a shift. I've had to deal with programs where any thread can be anywhere in the code, and it's horrific. D's allows such an approach, but it makes it a lot harder, so the default approach ends up being much more sane. - Jonathan M Davis
Nov 28 2011
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/30/11 5:04 PM, Sean Kelly wrote:
 On Nov 28, 2011, at 3:48 AM, Debdatta wrote:
 In my (limited) experience, involving mostly threading for
 performance, sharing is the norm.( Take a look at .NET's task
 parallel library, or intel's TBB.)

So is complexity and the propensity for deadlocks ;-)

In fact there's a bit of confusion somewhere; in fork-join parallelism (.NET TPL, TBB) there is in fact very little sharing; individual threads work on disjoint data sets, and the join operation simply establishes checkpoints. Andrei
Nov 30 2011
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 30, 2011, at 5:41 PM, Andrei Alexandrescu wrote:

 On 11/30/11 5:04 PM, Sean Kelly wrote:
 On Nov 28, 2011, at 3:48 AM, Debdatta wrote:
=20
 In my (limited) experience, involving mostly threading for
 performance, sharing is the norm.( Take a look at .NET's task
 parallel library, or intel's TBB.)

So is complexity and the propensity for deadlocks ;-)

In fact there's a bit of confusion somewhere; in fork-join parallelism =

work on disjoint data sets, and the join operation simply establishes = checkpoints. I suppose I should have read up a bit on TBB first. I'd assumed it was = more of a classic multithreading library. My mistake.=
Dec 02 2011
prev sibling next sibling parent Brad Roberts <braddr slice-2.puremagic.com> writes:
On Mon, 28 Nov 2011, Debdatta wrote:

1. Your variables are thread local. This guarantees some nice
properties, like code that was intended to run in a single-threaded
environment when it was written will behave nicely in a multi-threaded
environment too, because every thread gets a copy of all global data.
This is a sane default.


Now I understand your reasoning a lot better. :D this is one case where D's defaults will come very handy. Its very different from what I have seen so far, so maybe its a culture shock.
There is one to one feature correspondence.


Of course. That is obvious.:D It just seemed counter intuitive to have every variable I declare to be thread local. Will experiment with it some more and see if I can get used to the concept. You seem to have very good reasons to prefer it, and I hope I can see them too. :D -Debdatta

If _most_ of your variables are thread local, then I think you might well be doing something wrong. Since that means that you're using a lot of what would be global variables in C/C++. In my experience, MOST variables are function scoped, and thus stack variables. My 2 cents, Brad
Nov 28 2011
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/01/2011 02:06 AM, Sean Kelly wrote:
 On Nov 28, 2011, at 2:54 AM, Jonathan M Davis wrote:

 On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class shared
 marks all its members shared? If what you said were true, it would be
 trivial to instantiate a class as both shared and unshared. Without any
 duplication.

 class A
 {
 ///
 }

 shared A sharedA = new A;
 A unsharedA = new A;

 I don't can you elaborate your last post?

Instantiating it isn't the hard part. You probably need a constructor which is shared, but that's not all that big a deal. The problem is that every member function that you want to call with a shared instance needs to be marked as shared (just like a member function which is called with a const instance must be marked as const). And while a const member function may work with both mutable and immutable instances, a shared member function will _only_ work with shared instances.

This seems wrong to me. If a function is callable in a shared scenario, why would it not be callable in an unshared scenario? At worst depending on how shared were implemented the call would be slower than an unshared version but still entirely correct.

Because of escaping. Assume what you suggest would work: shared A imaglobalsharedvariable; class A { int x; void foo()shared{ imaglobalsharedvariable = this; } void goo()shared{x = 1;} } void main() { auto imalocalunsharedvariable = new A; imalocalunsharedvariable.foo(); assert(imalocalunsharedvariable is imaglobalsharedvariable); // oops } If we had a proper implementation of the 'scope' storage class it would probably work by annotating void goo() scope shared{...}
Dec 01 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 28, 2011 08:48:20 Debdatta wrote:
 Jude,
 
 Glad to hear that you found my post informative. Anyway, from what I
 understand, shared is infectious. However, lets say we have the following:
 
 class A
 {
     .......
 
 };
 shared class B
 {
     constructor()
     {
        myA = new A;
     }
 A myA;
 }
 
 This will create a problem as the myA is shared, whereas A is not. Hence, we
 have to tag A to be shared as well. This will lead to most things marked
 shared and will cause confusion. Not to mention that default sharing is the
 convention most programmers have been born with, and thats hard to change
 in a person. :D
 
 However, in default shared languages, hiding types is easy. For instance, if
 I wanted a thread with a completely local context, I would do this:
 
 class myLocalClass : Thread
 {
 public:
 //whatever I want accessible by other classes go here.
 
 void run();
 
 private:
 
 //whatever I want to make specific to my thread will be located here.
 
 }
 
 This seems to be a more elegant solution to the problem. Any thoughts?

I would point out that there's no such thing as a shared class. An _instance_ of a class can be shared, but not the class itself. It's the member functions which are shared or not. Marking the class as shared just marks _everything_ in the class as shared. It's possible to have class which can be instantiated as both shared and unshared. However, it leads to a fair bit of code duplication (or the use of template or string mixins to avoid code duplication), since every function that you want to be able to call with both a shared and non-shared instance must be duplicated (one function being marked shared, the other not). However, the idea is that you'll generally design a class to be used as either shared or non-shared, and that can have large ramifications on how stuff works (e.g. whether a class is synchronized or whether mutexes or synchronized blocks are used), so the implementations are often _expected_ to be different. Overall, it's expected that the use of shared will be relatively rare, and the language design reflects that, so it's likely to to harder to use in cases where you want to use shared heavily. There have been several recent discussions on various issues with shared. So, you might want to look through the archives. - Jonathan M Davis
Nov 28 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class shared
 marks all its members shared? If what you said were true, it would be
 trivial to instantiate a class as both shared and unshared. Without any
 duplication.
 
 class A
 {
 ///
 }
 
 shared A sharedA = new A;
 A unsharedA = new A;
 
 I don't can you elaborate your last post?

Instantiating it isn't the hard part. You probably need a constructor which is shared, but that's not all that big a deal. The problem is that every member function that you want to call with a shared instance needs to be marked as shared (just like a member function which is called with a const instance must be marked as const). And while a const member function may work with both mutable and immutable instances, a shared member function will _only_ work with shared instances. So, end up having to duplicate any functions that you want to be able to call with both shared and unshared instances. Now, there's a decent chance that the shared versions need to be different anyway, because they may need to deal with locking in some manner or the like, and the judicious use of template mixins and string mixins can avoid much duplication, but you still end up having to duplicate a fair bit, so trying to use a class in both a shared and unshared context can get annoying. - Jonathan M Davis
Nov 28 2011
prev sibling next sibling parent "Marco Leise" <Marco.Leise gmx.de> writes:
Am 28.11.2011, 11:54 Uhr, schrieb Jonathan M Davis <jmdavisProg gmx.com>:

 On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class  
 shared
 marks all its members shared? If what you said were true, it would be
 trivial to instantiate a class as both shared and unshared. Without any
 duplication.

 class A
 {
 ///
 }

 shared A sharedA = new A;
 A unsharedA = new A;


maybe it is: shared A sharedA = new shared(A); I think I read something like that in the past.
Nov 28 2011
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 28/11/2011 06:51, Debdata a écrit :
 Hi,

 I have been evaluating D for the past week, and have some concerns regarding
the No Default sharing rule. I have been reading the D book by Andrei as
 well. This particular feature has been very confusing to me as I come from the
c++ world.

 I agree that message passing and resource hiding are a great way to go for a
lot of cases, but there are an equally large (Larger?) number of cases
 that would benefit from global sharing. Especially when threading for
performance rather than convenience. The cutting edge of high performance
 threading is really the task stealing approach. See intel's TBB, cilk, etc.
Implementing things like that would become unnecessarily verbose as most
 things will be shared and we have to keep tagging.

 Unless I am missing something. :D

 -Debdatta Basu

What you think is an advantage is actually a source of errors very difficult to spot. See that document : http://www.hpl.hp.com/personal/Hans_Boehm/misc_slides/pldi05_threads.pdf
Nov 28 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 27, 2011, at 9:51 PM, Debdata wrote:
=20
 I agree that message passing and resource hiding are a great way to go =

cases
 that would benefit from global sharing. Especially when threading for =

performance
 threading is really the task stealing approach. See intel's TBB, cilk, =

most
 things will be shared and we have to keep tagging.

This is more the domain of std.parallelism and it's quite useful, but = each have their strengths. For one, message-based concurrency can scale = horizontally across machines if done correctly (see Erlang, for = instance), which can be a huge win. Also, I find it to be a much easier = model to develop for. The work-stealing and similar models expose some = of the details of threading and concurrency by necessity, which means = that the programmer has to think about this and craft the code = appropriately. I think this model will be progressively less common as = time goes on.=
Nov 30 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 28, 2011, at 3:48 AM, Debdatta wrote:
=20
 In my (limited) experience, involving mostly threading for =

library, or
 intel's TBB.)

So is complexity and the propensity for deadlocks ;-)=
Nov 30 2011
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 28, 2011, at 2:54 AM, Jonathan M Davis wrote:

 On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class =


 marks all its members shared? If what you said were true, it would be
 trivial to instantiate a class as both shared and unshared. Without =


 duplication.
=20
 class A
 {
 ///
 }
=20
 shared A sharedA =3D new A;
 A unsharedA =3D new A;
=20
 I don't can you elaborate your last post?

Instantiating it isn't the hard part. You probably need a constructor =

 shared, but that's not all that big a deal. The problem is that every =

 function that you want to call with a shared instance needs to be =

 shared (just like a member function which is called with a const =

 be marked as const). And while a const member function may work with =

 mutable and immutable instances, a shared member function will _only_ =

 with shared instances.

This seems wrong to me. If a function is callable in a shared scenario, = why would it not be callable in an unshared scenario? At worst = depending on how shared were implemented the call would be slower than = an unshared version but still entirely correct.=
Nov 30 2011
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, November 30, 2011 17:06:22 Sean Kelly wrote:
 On Nov 28, 2011, at 2:54 AM, Jonathan M Davis wrote:
 On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class
 shared marks all its members shared? If what you said were true, it
 would be trivial to instantiate a class as both shared and unshared.
 Without any duplication.
 
 class A
 {
 ///
 }
 
 shared A sharedA = new A;
 A unsharedA = new A;
 
 I don't can you elaborate your last post?

Instantiating it isn't the hard part. You probably need a constructor which is shared, but that's not all that big a deal. The problem is that every member function that you want to call with a shared instance needs to be marked as shared (just like a member function which is called with a const instance must be marked as const). And while a const member function may work with both mutable and immutable instances, a shared member function will _only_ work with shared instances.

This seems wrong to me. If a function is callable in a shared scenario, why would it not be callable in an unshared scenario? At worst depending on how shared were implemented the call would be slower than an unshared version but still entirely correct.

While it may be theoretically fine, as I understand it, that's how the type system operates at this point. However, I use shared quite rarely, so some of its finer points are likely to escape me. I definitely have to agree, however, that at this point, using shared heavily seems to be quite problematic due to stuff like seemingly unnecessary code duplication which is currently necessary. - Jonathan M Davis
Nov 30 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12/01/2011 03:29 AM, Jonathan M Davis wrote:
 On Wednesday, November 30, 2011 17:06:22 Sean Kelly wrote:
 On Nov 28, 2011, at 2:54 AM, Jonathan M Davis wrote:
 On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class
 shared marks all its members shared? If what you said were true, it
 would be trivial to instantiate a class as both shared and unshared.
 Without any duplication.

 class A
 {
 ///
 }

 shared A sharedA = new A;
 A unsharedA = new A;

 I don't can you elaborate your last post?

Instantiating it isn't the hard part. You probably need a constructor which is shared, but that's not all that big a deal. The problem is that every member function that you want to call with a shared instance needs to be marked as shared (just like a member function which is called with a const instance must be marked as const). And while a const member function may work with both mutable and immutable instances, a shared member function will _only_ work with shared instances.

This seems wrong to me. If a function is callable in a shared scenario, why would it not be callable in an unshared scenario? At worst depending on how shared were implemented the call would be slower than an unshared version but still entirely correct.

While it may be theoretically fine, as I understand it, that's how the type system operates at this point. However, I use shared quite rarely, so some of its finer points are likely to escape me. I definitely have to agree, however, that at this point, using shared heavily seems to be quite problematic due to stuff like seemingly unnecessary code duplication which is currently necessary. - Jonathan M Davis

You bring this up very often. Do you write a lot of concurrent code where this actually bothers you? Efficient shared methods need to be different from efficient unshared ones anyway. If it is just about having locked versions of some methods that perform the same operations (except they may randomly dead lock your program if you are not cautious), then, well: even languages that do not have the 'shared' type modifier use synchronized wrapper classes to solve *the same problem*. In D, the boilerplate that constitutes the wrapper class should even be able to be generated by some nifty little template. Or alternatively you just mix in the wrapper methods into the same class. This is impossible to achieve in other languages.
Dec 01 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12/01/2011 04:51 PM, Timon Gehr wrote:
 On 12/01/2011 03:29 AM, Jonathan M Davis wrote:
 On Wednesday, November 30, 2011 17:06:22 Sean Kelly wrote:
 On Nov 28, 2011, at 2:54 AM, Jonathan M Davis wrote:
 On Monday, November 28, 2011 10:28:34 Debdatta wrote:
 Let me get this straight. Instances are shared... and marking a class
 shared marks all its members shared? If what you said were true, it
 would be trivial to instantiate a class as both shared and unshared.
 Without any duplication.

 class A
 {
 ///
 }

 shared A sharedA = new A;
 A unsharedA = new A;

 I don't can you elaborate your last post?

Instantiating it isn't the hard part. You probably need a constructor which is shared, but that's not all that big a deal. The problem is that every member function that you want to call with a shared instance needs to be marked as shared (just like a member function which is called with a const instance must be marked as const). And while a const member function may work with both mutable and immutable instances, a shared member function will _only_ work with shared instances.

This seems wrong to me. If a function is callable in a shared scenario, why would it not be callable in an unshared scenario? At worst depending on how shared were implemented the call would be slower than an unshared version but still entirely correct.

While it may be theoretically fine, as I understand it, that's how the type system operates at this point. However, I use shared quite rarely, so some of its finer points are likely to escape me. I definitely have to agree, however, that at this point, using shared heavily seems to be quite problematic due to stuff like seemingly unnecessary code duplication which is currently necessary. - Jonathan M Davis

You bring this up very often. Do you write a lot of concurrent code where this actually bothers you? Efficient shared methods need to be different from efficient unshared ones anyway. If it is just about having locked versions of some methods that perform the same operations (except they may randomly dead lock your program if you are not cautious), then, well: even languages that do not have the 'shared' type modifier use synchronized wrapper classes to solve *the same problem*. In D, the boilerplate that constitutes the wrapper class should even be able to be generated by some nifty little template. Or alternatively you just mix in the wrapper methods into the same class. This is impossible to achieve in other languages.

(I meant 'other languages that do not support the shared modifier', obviously)
Dec 01 2011
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Dec 1, 2011, at 7:28 AM, Timon Gehr wrote:

 On 12/01/2011 02:06 AM, Sean Kelly wrote:
=20
=20
 This seems wrong to me.  If a function is callable in a shared =


worst depending on how shared were implemented the call would be slower = than an unshared version but still entirely correct.
=20
 Because of escaping.=20

I'd overlooked that. Thanks.=
Dec 02 2011