www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A monitor for every object

reply bearophile <bearophileHUGS lycos.com> writes:
I have found an interesting post by Scott Johnson in this Lambda the Ultimate
thread:
http://lambda-the-ultimate.org/node/724#comment-6621

He says:

9th circle: Concurrent mutable state. The obnoxious practice of mutating shared
state from multiple threads of control, leading into a predictable cycle of
race conditions, deadlocks, and other assorted misbehavior from which there is
no return. And if a correct solution (for synchronization) is found for a given
program, chances are any substantial change to the program will make it
incorrect again. But you won't find it, instead your customer will. Despite
that, reams of code (and TONS of middleware) has been written to try and make
this tractable. And don't get me started on a certain programming language
which starts with "J" that saw fit to make EVERY object have its very own
monitor....<

This is just one quotation, but I have found similar comments four or five other times around the Web. So is the design choice of copying this part of the Java design inside D good? I'd like opinions on this topic. Recently I have suggested an optional nomonitor annotation for D classes (to optionally remove a word from class instances and to reduce class instantiation overhead a bit). Another option is doing the opposite, and defining a withmonitor annotation where you want a class to have a monitor. Bye, bearophile
Feb 04 2011
next sibling parent Mafi <mafi example.org> writes:
Am 04.02.2011 13:26, schrieb bearophile:
 I have found an interesting post by Scott Johnson in this Lambda the Ultimate
thread:
 http://lambda-the-ultimate.org/node/724#comment-6621

 He says:

 9th circle: Concurrent mutable state. The obnoxious practice of mutating
shared state from multiple threads of control, leading into a predictable cycle
of race conditions, deadlocks, and other assorted misbehavior from which there
is no return. And if a correct solution (for synchronization) is found for a
given program, chances are any substantial change to the program will make it
incorrect again. But you won't find it, instead your customer will. Despite
that, reams of code (and TONS of middleware) has been written to try and make
this tractable. And don't get me started on a certain programming language
which starts with "J" that saw fit to make EVERY object have its very own
monitor....<

This is just one quotation, but I have found similar comments four or five other times around the Web. So is the design choice of copying this part of the Java design inside D good? I'd like opinions on this topic. Recently I have suggested an optional nomonitor annotation for D classes (to optionally remove a word from class instances and to reduce class instantiation overhead a bit). Another option is doing the opposite, and defining a withmonitor annotation where you want a class to have a monitor. Bye, bearophile

little behavior change). It called 'pragma'. Exchange nomonitor with pragma(nomonitor) and I'm for your idea. pragma looks and feels better IMO. Mafi PS: cant see any ttributes more :(
Feb 04 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 04 Feb 2011 07:26:22 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:

 I have found an interesting post by Scott Johnson in this Lambda the  
 Ultimate thread:
 http://lambda-the-ultimate.org/node/724#comment-6621

 He says:

 9th circle: Concurrent mutable state. The obnoxious practice of  
 mutating shared state from multiple threads of control, leading into a  
 predictable cycle of race conditions, deadlocks, and other assorted  
 misbehavior from which there is no return. And if a correct solution  
 (for synchronization) is found for a given program, chances are any  
 substantial change to the program will make it incorrect again. But you  
 won't find it, instead your customer will. Despite that, reams of code  
 (and TONS of middleware) has been written to try and make this  
 tractable. And don't get me started on a certain programming language  
 which starts with "J" that saw fit to make EVERY object have its very  
 own monitor....<

This is just one quotation, but I have found similar comments four or five other times around the Web. So is the design choice of copying this part of the Java design inside D good? I'd like opinions on this topic. Recently I have suggested an optional nomonitor annotation for D classes (to optionally remove a word from class instances and to reduce class instantiation overhead a bit). Another option is doing the opposite, and defining a withmonitor annotation where you want a class to have a monitor.

D's monitors are lazily created, so there should be no issue with resource allocation. If you don't ever lock an object instance, it's not going to consume any resources. Most of the time the extra word isn't noticed because the memory size of a class is usually not exactly a power of 2. D also allows you to replace it's monitor with a custom monitor object (i.e. core.sync.Mutex) so you can have more control over the mutex, assign the same mutex to multiple objects, use conditions, etc. It's much more flexible than Java or C# IMO. -Steve
Feb 04 2011
next sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Steven Schveighoffer wrote:
 D's monitors are lazily created, so there should be no issue with
 resource allocation.=20

same object at the same time? Is there a global lock to avoid race conditions in this case? Jerome --=20 mailto:jeberger free.fr http://jeberger.free.fr Jabber: jeberger jabber.fr
Feb 04 2011
parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Robert Jacques wrote:
 On Fri, 04 Feb 2011 17:23:35 -0500, J=C3=A9r=C3=B4me M. Berger <jeberge=

 wrote:
=20
 Steven Schveighoffer wrote:
 D's monitors are lazily created, so there should be no issue with
 resource allocation.

same object at the same time? Is there a global lock to avoid race conditions in this case? Jerome

Only the reference to the mutex is shared, so all you need in an atomic=

This requires an atomic "if (a is null) a =3D b;". I did not know that such a beast existed. Jerome --=20 mailto:jeberger free.fr http://jeberger.free.fr Jabber: jeberger jabber.fr
Feb 05 2011
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Feb 4, 2011, at 3:06 PM, Tomek Sowi=C5=84ski wrote:

 Steven Schveighoffer napisa=C5=82:
=20
 D also allows you to replace it's monitor with a custom monitor =


 (i.e. core.sync.Mutex) so you can have more control over the mutex, =


 the same mutex to multiple objects, use conditions, etc.  It's much =


 flexible than Java or C# IMO.

I didn't know, thx. Where is it documented?

Only in the docs for the Mutex ctor here: http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html By the way, when using Mutex as an object monitor you currently can't = attach dispose event handlers to the object (ie. via std.signals). That = functionality is only supported by the default monitor. I've considered = changing this, but doing so imposes some weird requirements on the = creators of an external object monitor, like Mutex.=
Feb 04 2011
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
Steven Schveighoffer Wrote:

 On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
 wrote:
 
 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:

 Steven Schveighoffer napisał:

 D also allows you to replace it's monitor with a custom monitor object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.

I didn't know, thx. Where is it documented?

Only in the docs for the Mutex ctor here: http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html By the way, when using Mutex as an object monitor you currently can't attach dispose event handlers to the object (ie. via std.signals). That functionality is only supported by the default monitor. I've considered changing this, but doing so imposes some weird requirements on the creators of an external object monitor, like Mutex.

Sean, I could have sworn that mutex can take over multiple objects' monitors. This would be highly desirable in a complex structure where members of an object should use the same mutex for synchronized calls. But looking at the docs once again, it looks like it can only be the monitor for one object (as the target object is accepted only on construction). Is that a limitation we cannot remove?

Now that you mention it, I think TDPL describes a call like setMonitor() that's supposed to do this, and I just forgot to implement it. There's no technical barrier. It's simply a matter of assigning the monitor reference to the correct part of the Mutex instance.
Feb 07 2011
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
Steven Schveighoffer Wrote:

 On Mon, 07 Feb 2011 10:33:29 -0500, Robert Jacques <sandford jhu.edu>  
 wrote:
 Steve, you can always assign to an object's monitor variable manually.  
 But adding this functionality to Mutex's and Object's API would be  
 appreciated.

Sure, Mutex does this already. What I was simply asking is if it will blow up or not :)

It'll work fine, so long as the monitor is a Mutex. If you want to share the default monitor, that will require some work, because the instance is manually allocated. Oh, another issue with using Mutex as an object monitor is that it's allocated on the GC heap, so a synchronized block in the object's dtor could fail. All the usual workarounds apply--just something to be aware of.
Feb 07 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Fri, 04 Feb 2011 17:23:35 -0500, Jrme M. Berger <jeberger free.fr>  
wrote:

 Steven Schveighoffer wrote:
 D's monitors are lazily created, so there should be no issue with
 resource allocation.

same object at the same time? Is there a global lock to avoid race conditions in this case? Jerome

Only the reference to the mutex is shared, so all you need in an atomic op.
Feb 04 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 05 Feb 2011 03:00:31 -0500, Jrme M. Berger <jeberger free.fr>  
wrote:

 Robert Jacques wrote:
 On Fri, 04 Feb 2011 17:23:35 -0500, Jrme M. Berger <jeberger free.fr>
 wrote:

 Steven Schveighoffer wrote:
 D's monitors are lazily created, so there should be no issue with
 resource allocation.

same object at the same time? Is there a global lock to avoid race conditions in this case? Jerome

Only the reference to the mutex is shared, so all you need in an atomic op.

This requires an atomic "if (a is null) a = b;". I did not know that such a beast existed. Jerome

Yes, the beast exists and it's the basis for most lock-free programming as well as lock implementations. It's generally known as compare and swap or CAS (see http://en.wikipedia.org/wiki/Compare_and_swap and core.atomic.cas). There's also another atomic primitive called Load-Link/Store-Conditional, which is available an several non-x86 architectures (ARM, PowerPC, etc) and is generally considered a more powerful primitive than CAS.
Feb 05 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
wrote:

 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:

 Steven Schveighoffer napisał:

 D also allows you to replace it's monitor with a custom monitor object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.

I didn't know, thx. Where is it documented?

Only in the docs for the Mutex ctor here: http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html By the way, when using Mutex as an object monitor you currently can't attach dispose event handlers to the object (ie. via std.signals). That functionality is only supported by the default monitor. I've considered changing this, but doing so imposes some weird requirements on the creators of an external object monitor, like Mutex.

Sean, I could have sworn that mutex can take over multiple objects' monitors. This would be highly desirable in a complex structure where members of an object should use the same mutex for synchronized calls. But looking at the docs once again, it looks like it can only be the monitor for one object (as the target object is accepted only on construction). Is that a limitation we cannot remove? -Steve
Feb 07 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Mon, 07 Feb 2011 08:20:20 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:

 On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
 wrote:

 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:

 Steven Schveighoffer napisał:

 D also allows you to replace it's monitor with a custom monitor object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.

I didn't know, thx. Where is it documented?

Only in the docs for the Mutex ctor here: http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html By the way, when using Mutex as an object monitor you currently can't attach dispose event handlers to the object (ie. via std.signals). That functionality is only supported by the default monitor. I've considered changing this, but doing so imposes some weird requirements on the creators of an external object monitor, like Mutex.

Sean, I could have sworn that mutex can take over multiple objects' monitors. This would be highly desirable in a complex structure where members of an object should use the same mutex for synchronized calls. But looking at the docs once again, it looks like it can only be the monitor for one object (as the target object is accepted only on construction). Is that a limitation we cannot remove? -Steve

Steve, you can always assign to an object's monitor variable manually. But adding this functionality to Mutex's and Object's API would be appreciated.
Feb 07 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 07 Feb 2011 10:33:29 -0500, Robert Jacques <sandford jhu.edu>  
wrote:

 On Mon, 07 Feb 2011 08:20:20 -0500, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:

 On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
 wrote:

 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:

 Steven Schveighoffer napisał:

 D also allows you to replace it's monitor with a custom monitor  
 object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.

I didn't know, thx. Where is it documented?

Only in the docs for the Mutex ctor here: http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html By the way, when using Mutex as an object monitor you currently can't attach dispose event handlers to the object (ie. via std.signals). That functionality is only supported by the default monitor. I've considered changing this, but doing so imposes some weird requirements on the creators of an external object monitor, like Mutex.

Sean, I could have sworn that mutex can take over multiple objects' monitors. This would be highly desirable in a complex structure where members of an object should use the same mutex for synchronized calls. But looking at the docs once again, it looks like it can only be the monitor for one object (as the target object is accepted only on construction). Is that a limitation we cannot remove? -Steve

Steve, you can always assign to an object's monitor variable manually. But adding this functionality to Mutex's and Object's API would be appreciated.

Sure, Mutex does this already. What I was simply asking is if it will blow up or not :) -Steve
Feb 07 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 07 Feb 2011 14:29:37 -0500, Sean Kelly <sean invisibleduck.org>  
wrote:

 Steven Schveighoffer Wrote:

 On Mon, 07 Feb 2011 10:33:29 -0500, Robert Jacques <sandford jhu.edu>
 wrote:
 Steve, you can always assign to an object's monitor variable manually.
 But adding this functionality to Mutex's and Object's API would be
 appreciated.

Sure, Mutex does this already. What I was simply asking is if it will blow up or not :)

It'll work fine, so long as the monitor is a Mutex. If you want to share the default monitor, that will require some work, because the instance is manually allocated.

I think sharing a default monitor is not necessary, we can make do with Mutex.
 Oh, another issue with using Mutex as an object monitor is that it's  
 allocated on the GC heap, so a synchronized block in the object's dtor  
 could fail.  All the usual workarounds apply--just something to be aware  
 of.

It might be a good idea to identify these limitations (the signal one and this one) in the docs if they aren't already... And thanks for looking at this. -Steve
Feb 07 2011
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
bearophile Wrote:

 a certain programming language which starts with "J" that saw fit to make
EVERY object have its very own monitor....<
 So is the design choice of copying this part of the Java design inside D good?
I'd like opinions on this topic.

C# has this design too, but locking a common object directly is not used. C# used to create separate object for locking, the property to access it being called SyncRoot. It's implementation is a usual combination of new object() and CompareExchange. http://msdn.microsoft.com/en-us/library/system.collections.icollection.syncroot.aspx I don't feel the need for locking functionality in Object, this is usually a major design decision for a couple of classes - even in a large project. After all, after this design decision you should also carefully implement the locking in the code using the object, this will take some time, so it can't be a light-minded decision.
Feb 04 2011
parent Kagamin <spam here.lot> writes:
Kagamin Wrote:

 bearophile Wrote:
 
 a certain programming language which starts with "J" that saw fit to make
EVERY object have its very own monitor....<
 So is the design choice of copying this part of the Java design inside D good?
I'd like opinions on this topic.

C# has this design too, but locking a common object directly is not used. C# used to create separate object for locking, the property to access it being called SyncRoot. It's implementation is a usual combination of new object() and CompareExchange.

Ah, I forgot to say, that C# has a sort of guideline to lock the SyncRoot instead of collection itself.
Feb 04 2011
prev sibling next sibling parent Tomek =?UTF-8?B?U293acWEc2tp?= <just ask.me> writes:
Steven Schveighoffer napisa=C5=82:

 D's monitors are lazily created, so there should be no issue with resourc=

 allocation.  If you don't ever lock an object instance, it's not going to=

 consume any resources.  Most of the time the extra word isn't noticed =20
 because the memory size of a class is usually not exactly a power of 2.

Except when you put'em in an array. Could happen.
 D also allows you to replace it's monitor with a custom monitor object =20
 (i.e. core.sync.Mutex) so you can have more control over the mutex, assig=

 the same mutex to multiple objects, use conditions, etc.  It's much more =

 flexible than Java or C# IMO.

I didn't know, thx. Where is it documented? --=20 Tomek
Feb 04 2011
prev sibling next sibling parent Tomek =?UTF-8?B?U293acWEc2tp?= <just ask.me> writes:
Tomek Sowi=C5=84ski napisa=C5=82:

 D's monitors are lazily created, so there should be no issue with resou=


 allocation.  If you don't ever lock an object instance, it's not going =


 consume any resources.  Most of the time the extra word isn't noticed =


 because the memory size of a class is usually not exactly a power of 2.=


=20
 Except when you put'em in an array. Could happen.

Sorry, for some reason I thought the mutex is on the stack. --=20 Tomek
Feb 04 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Fri, 04 Feb 2011 07:26:22 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:
 I have found an interesting post by Scott Johnson in this Lambda the  
 Ultimate thread:
 http://lambda-the-ultimate.org/node/724#comment-6621

 He says:

 9th circle: Concurrent mutable state. The obnoxious practice of  
 mutating shared state from multiple threads of control, leading into a  
 predictable cycle of race conditions, deadlocks, and other assorted  
 misbehavior from which there is no return. And if a correct solution  
 (for synchronization) is found for a given program, chances are any  
 substantial change to the program will make it incorrect again. But you  
 won't find it, instead your customer will. Despite that, reams of code  
 (and TONS of middleware) has been written to try and make this  
 tractable. And don't get me started on a certain programming language  
 which starts with "J" that saw fit to make EVERY object have its very  
 own monitor....<

This is just one quotation, but I have found similar comments four or five other times around the Web. So is the design choice of copying this part of the Java design inside D good? I'd like opinions on this topic. Recently I have suggested an optional nomonitor annotation for D classes (to optionally remove a word from class instances and to reduce class instantiation overhead a bit). Another option is doing the opposite, and defining a withmonitor annotation where you want a class to have a monitor.

Hmm... Well, I'd recommend making nomonitor the default and then only annotate certain classes withmonitor, although I'd prefer a different keyword, say 'shared'. Oh, wait a second. *sigh* Every since it was decided that a class couldn't contain both shared and non-shared methods/fields I've been expecting that the monitor and support for synchronized methods would be removed from Object or at least from Object's spec. But this is likely a high-cost/low-gain optimization and a lot of things regarding shared/immutable classes need to be fixed before it can happen. I do think it should happen, not so much for the word of memory, but in order to prevent objects not designed to be shared from being shared. Which is what the 9th circle is talking about. One of the cool thing about D's monitors, is that by manually setting them, you can protect multiple objects with a single monitor. So, for a set of interwoven objects (i.e. trees, etc) you're not acquiring a new lock every method call and you are not going to have an internal deadlock/race. This is actually the essential runtime feature of many ownership systems. (Although, without an actual ownership-type system you can't elide synchronization entirely)
Feb 04 2011