www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A monitor for every object

reply bearophile <bearophileHUGS lycos.com> writes:
I have found an interesting post by Scott Johnson in this Lambda the Ultimate
thread:
http://lambda-the-ultimate.org/node/724#comment-6621

He says:


9th circle: Concurrent mutable state. The obnoxious practice of mutating shared
state from multiple threads of control, leading into a predictable cycle of
race conditions, deadlocks, and other assorted misbehavior from which there is
no return. And if a correct solution (for synchronization) is found for a given
program, chances are any substantial change to the program will make it
incorrect again. But you won't find it, instead your customer will. Despite
that, reams of code (and TONS of middleware) has been written to try and make
this tractable. And don't get me started on a certain programming language
which starts with "J" that saw fit to make EVERY object have its very own
monitor....<



This is just one quotation, but I have found similar comments four or five
other times around the Web.

So is the design choice of copying this part of the Java design inside D good?
I'd like opinions on this topic.

Recently I have suggested an optional  nomonitor annotation for D classes (to
optionally remove a word from class instances and to reduce class instantiation
overhead a bit). Another option is doing the opposite, and defining a
 withmonitor annotation where you want a class to have a monitor.

Bye,
bearophile
Feb 04 2011
next sibling parent Mafi <mafi example.org> writes:
Am 04.02.2011 13:26, schrieb bearophile:

 I have found an interesting post by Scott Johnson in this Lambda the Ultimate
thread:
 http://lambda-the-ultimate.org/node/724#comment-6621

 He says:


 9th circle: Concurrent mutable state. The obnoxious practice of mutating
shared state from multiple threads of control, leading into a predictable cycle
of race conditions, deadlocks, and other assorted misbehavior from which there
is no return. And if a correct solution (for synchronization) is found for a
given program, chances are any substantial change to the program will make it
incorrect again. But you won't find it, instead your customer will. Despite
that, reams of code (and TONS of middleware) has been written to try and make
this tractable. And don't get me started on a certain programming language
which starts with "J" that saw fit to make EVERY object have its very own
monitor....<



 This is just one quotation, but I have found similar comments four or five
other times around the Web.

 So is the design choice of copying this part of the Java design inside D good?
I'd like opinions on this topic.

 Recently I have suggested an optional  nomonitor annotation for D classes (to
optionally remove a word from class instances and to reduce class instantiation
overhead a bit). Another option is doing the opposite, and defining a
 withmonitor annotation where you want a class to have a monitor.

 Bye,
 bearophile



little behavior change). It called 'pragma'. Exchange  nomonitor with 
pragma(nomonitor) and I'm for your idea. pragma looks and feels better IMO.

Mafi

PS: cant see any  ttributes more :(
Feb 04 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 04 Feb 2011 07:26:22 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:


 I have found an interesting post by Scott Johnson in this Lambda the  
 Ultimate thread:
 http://lambda-the-ultimate.org/node/724#comment-6621

 He says:


 9th circle: Concurrent mutable state. The obnoxious practice of  
 mutating shared state from multiple threads of control, leading into a  
 predictable cycle of race conditions, deadlocks, and other assorted  
 misbehavior from which there is no return. And if a correct solution  
 (for synchronization) is found for a given program, chances are any  
 substantial change to the program will make it incorrect again. But you  
 won't find it, instead your customer will. Despite that, reams of code  
 (and TONS of middleware) has been written to try and make this  
 tractable. And don't get me started on a certain programming language  
 which starts with "J" that saw fit to make EVERY object have its very  
 own monitor....<



 This is just one quotation, but I have found similar comments four or  
 five other times around the Web.

 So is the design choice of copying this part of the Java design inside D  
 good? I'd like opinions on this topic.

 Recently I have suggested an optional  nomonitor annotation for D  
 classes (to optionally remove a word from class instances and to reduce  
 class instantiation overhead a bit). Another option is doing the  
 opposite, and defining a  withmonitor annotation where you want a class  
 to have a monitor.



D's monitors are lazily created, so there should be no issue with resource  
allocation.  If you don't ever lock an object instance, it's not going to  
consume any resources.  Most of the time the extra word isn't noticed  
because the memory size of a class is usually not exactly a power of 2.

D also allows you to replace it's monitor with a custom monitor object  
(i.e. core.sync.Mutex) so you can have more control over the mutex, assign  
the same mutex to multiple objects, use conditions, etc.  It's much more  
flexible than Java or C# IMO.

-Steve
Feb 04 2011
next sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Steven Schveighoffer wrote:

 D's monitors are lazily created, so there should be no issue with
 resource allocation.=20



same object at the same time? Is there a global lock to avoid race
conditions in this case?

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Feb 04 2011
parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Robert Jacques wrote:

 On Fri, 04 Feb 2011 17:23:35 -0500, J=C3=A9r=C3=B4me M. Berger <jeberge=




 wrote:
=20

 Steven Schveighoffer wrote:

 D's monitors are lazily created, so there should be no issue with
 resource allocation.



 same object at the same time? Is there a global lock to avoid race
 conditions in this case?

         Jerome



 Only the reference to the mutex is shared, so all you need in an atomic=




	This requires an atomic "if (a is null) a =3D b;". I did not know
that such a beast existed.

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Feb 05 2011
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Feb 4, 2011, at 3:06 PM, Tomek Sowi=C5=84ski wrote:


 Steven Schveighoffer napisa=C5=82:
=20

 D also allows you to replace it's monitor with a custom monitor =







 (i.e. core.sync.Mutex) so you can have more control over the mutex, =







 the same mutex to multiple objects, use conditions, etc.  It's much =







 flexible than Java or C# IMO.



 I didn't know, thx. Where is it documented?



Only in the docs for the Mutex ctor here:

http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html

By the way, when using Mutex as an object monitor you currently can't =
attach dispose event handlers to the object (ie. via std.signals).  That =
functionality is only supported by the default monitor.  I've considered =
changing this, but doing so imposes some weird requirements on the =
creators of an external object monitor, like Mutex.=
Feb 04 2011
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
Steven Schveighoffer Wrote:


 On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
 wrote:
 

 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:


 Steven Schveighoffer napisał:


 D also allows you to replace it's monitor with a custom monitor object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.



 I didn't know, thx. Where is it documented?



 Only in the docs for the Mutex ctor here:

 http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html

 By the way, when using Mutex as an object monitor you currently can't  
 attach dispose event handlers to the object (ie. via std.signals).  That  
 functionality is only supported by the default monitor.  I've considered  
 changing this, but doing so imposes some weird requirements on the  
 creators of an external object monitor, like Mutex.



 Sean, I could have sworn that mutex can take over multiple objects'  
 monitors.  This would be highly desirable in a complex structure where  
 members of an object should use the same mutex for synchronized calls.
 
 But looking at the docs once again, it looks like it can only be the  
 monitor for one object (as the target object is accepted only on  
 construction).  Is that a limitation we cannot remove?



Now that you mention it, I think TDPL describes a call like setMonitor() that's
supposed to do this, and I just forgot to implement it.  There's no technical
barrier.  It's simply a matter of assigning the monitor reference to the
correct part of the Mutex instance.
Feb 07 2011
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
Steven Schveighoffer Wrote:


 On Mon, 07 Feb 2011 10:33:29 -0500, Robert Jacques <sandford jhu.edu>  
 wrote:

 Steve, you can always assign to an object's monitor variable manually.  
 But adding this functionality to Mutex's and Object's API would be  
 appreciated.



 Sure, Mutex does this already.  What I was simply asking is if it will  
 blow up or not :)



It'll work fine, so long as the monitor is a Mutex.  If you want to share the
default monitor, that will require some work, because the instance is manually
allocated.  Oh, another issue with using Mutex as an object monitor is that
it's allocated on the GC heap, so a synchronized block in the object's dtor
could fail.  All the usual workarounds apply--just something to be aware of.
Feb 07 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Fri, 04 Feb 2011 17:23:35 -0500, Jérôme M. Berger <jeberger free.fr>  
wrote:


 Steven Schveighoffer wrote:

 D's monitors are lazily created, so there should be no issue with
 resource allocation.



 same object at the same time? Is there a global lock to avoid race
 conditions in this case?

 		Jerome



Only the reference to the mutex is shared, so all you need in an atomic op.
Feb 04 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Sat, 05 Feb 2011 03:00:31 -0500, Jérôme M. Berger <jeberger free.fr>  
wrote:


 Robert Jacques wrote:

 On Fri, 04 Feb 2011 17:23:35 -0500, Jérôme M. Berger <jeberger free.fr>
 wrote:


 Steven Schveighoffer wrote:

 D's monitors are lazily created, so there should be no issue with
 resource allocation.



 same object at the same time? Is there a global lock to avoid race
 conditions in this case?

         Jerome



 Only the reference to the mutex is shared, so all you need in an atomic  
 op.



 	This requires an atomic "if (a is null) a = b;". I did not know
 that such a beast existed.

 		Jerome



Yes, the beast exists and it's the basis for most lock-free programming as  
well as lock implementations. It's generally known as compare and swap or  
CAS (see http://en.wikipedia.org/wiki/Compare_and_swap and  
core.atomic.cas). There's also another atomic primitive called  
Load-Link/Store-Conditional, which is available an several non-x86  
architectures (ARM, PowerPC, etc) and is generally considered a more  
powerful primitive than CAS.
Feb 05 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
wrote:


 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:


 Steven Schveighoffer napisał:


 D also allows you to replace it's monitor with a custom monitor object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.



 I didn't know, thx. Where is it documented?



 Only in the docs for the Mutex ctor here:

 http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html

 By the way, when using Mutex as an object monitor you currently can't  
 attach dispose event handlers to the object (ie. via std.signals).  That  
 functionality is only supported by the default monitor.  I've considered  
 changing this, but doing so imposes some weird requirements on the  
 creators of an external object monitor, like Mutex.



Sean, I could have sworn that mutex can take over multiple objects'  
monitors.  This would be highly desirable in a complex structure where  
members of an object should use the same mutex for synchronized calls.

But looking at the docs once again, it looks like it can only be the  
monitor for one object (as the target object is accepted only on  
construction).  Is that a limitation we cannot remove?

-Steve
Feb 07 2011
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Mon, 07 Feb 2011 08:20:20 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
 wrote:


 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:


 Steven Schveighoffer napisał:


 D also allows you to replace it's monitor with a custom monitor object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.



 I didn't know, thx. Where is it documented?



 Only in the docs for the Mutex ctor here:

 http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html

 By the way, when using Mutex as an object monitor you currently can't  
 attach dispose event handlers to the object (ie. via std.signals).   
 That functionality is only supported by the default monitor.  I've  
 considered changing this, but doing so imposes some weird requirements  
 on the creators of an external object monitor, like Mutex.



 Sean, I could have sworn that mutex can take over multiple objects'  
 monitors.  This would be highly desirable in a complex structure where  
 members of an object should use the same mutex for synchronized calls.

 But looking at the docs once again, it looks like it can only be the  
 monitor for one object (as the target object is accepted only on  
 construction).  Is that a limitation we cannot remove?

 -Steve



Steve, you can always assign to an object's monitor variable manually. But  
adding this functionality to Mutex's and Object's API would be appreciated.
Feb 07 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 07 Feb 2011 10:33:29 -0500, Robert Jacques <sandford jhu.edu>  
wrote:


 On Mon, 07 Feb 2011 08:20:20 -0500, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:


 On Fri, 04 Feb 2011 18:29:08 -0500, Sean Kelly <sean invisibleduck.org>  
 wrote:


 On Feb 4, 2011, at 3:06 PM, Tomek Sowiński wrote:


 Steven Schveighoffer napisał:


 D also allows you to replace it's monitor with a custom monitor  
 object
 (i.e. core.sync.Mutex) so you can have more control over the mutex,  
 assign
 the same mutex to multiple objects, use conditions, etc.  It's much  
 more
 flexible than Java or C# IMO.



 I didn't know, thx. Where is it documented?



 Only in the docs for the Mutex ctor here:

 http://www.digitalmars.com/d/2.0/phobos/core_sync_mutex.html

 By the way, when using Mutex as an object monitor you currently can't  
 attach dispose event handlers to the object (ie. via std.signals).   
 That functionality is only supported by the default monitor.  I've  
 considered changing this, but doing so imposes some weird requirements  
 on the creators of an external object monitor, like Mutex.



 Sean, I could have sworn that mutex can take over multiple objects'  
 monitors.  This would be highly desirable in a complex structure where  
 members of an object should use the same mutex for synchronized calls.

 But looking at the docs once again, it looks like it can only be the  
 monitor for one object (as the target object is accepted only on  
 construction).  Is that a limitation we cannot remove?

 -Steve



 Steve, you can always assign to an object's monitor variable manually.  
 But adding this functionality to Mutex's and Object's API would be  
 appreciated.



Sure, Mutex does this already.  What I was simply asking is if it will  
blow up or not :)

-Steve
Feb 07 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 07 Feb 2011 14:29:37 -0500, Sean Kelly <sean invisibleduck.org>  
wrote:


 Steven Schveighoffer Wrote:


 On Mon, 07 Feb 2011 10:33:29 -0500, Robert Jacques <sandford jhu.edu>
 wrote:

 Steve, you can always assign to an object's monitor variable manually.
 But adding this functionality to Mutex's and Object's API would be
 appreciated.



 Sure, Mutex does this already.  What I was simply asking is if it will
 blow up or not :)



 It'll work fine, so long as the monitor is a Mutex.  If you want to  
 share the default monitor, that will require some work, because the  
 instance is manually allocated.



I think sharing a default monitor is not necessary, we can make do with  
Mutex.


 Oh, another issue with using Mutex as an object monitor is that it's  
 allocated on the GC heap, so a synchronized block in the object's dtor  
 could fail.  All the usual workarounds apply--just something to be aware  
 of.



It might be a good idea to identify these limitations (the signal one and  
this one) in the docs if they aren't already...

And thanks for looking at this.

-Steve
Feb 07 2011
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
bearophile Wrote:


 a certain programming language which starts with "J" that saw fit to make
EVERY object have its very own monitor....<
 So is the design choice of copying this part of the Java design inside D good?
I'd like opinions on this topic.



C# has this design too, but locking a common object directly is not used. C#
used to create separate object for locking, the property to access it being
called SyncRoot. It's implementation is a usual combination of new object() and
CompareExchange.

http://msdn.microsoft.com/en-us/library/system.collections.icollection.syncroot.aspx

I don't feel the need for locking functionality in Object, this is usually a
major design decision for a couple of classes - even in a large project. After
all, after this design decision you should also carefully implement the locking
in the code using the object, this will take some time, so it can't be a
light-minded decision.
Feb 04 2011
parent Kagamin <spam here.lot> writes:
Kagamin Wrote:


 bearophile Wrote:
 

 a certain programming language which starts with "J" that saw fit to make
EVERY object have its very own monitor....<
 So is the design choice of copying this part of the Java design inside D good?
I'd like opinions on this topic.



 C# has this design too, but locking a common object directly is not used. C#
used to create separate object for locking, the property to access it being
called SyncRoot. It's implementation is a usual combination of new object() and
CompareExchange.



Ah, I forgot to say, that C# has a sort of guideline to lock the SyncRoot
instead of collection itself.
Feb 04 2011
prev sibling next sibling parent Tomek =?UTF-8?B?U293acWEc2tp?= <just ask.me> writes:
Steven Schveighoffer napisa=C5=82:


 D's monitors are lazily created, so there should be no issue with resourc=




 allocation.  If you don't ever lock an object instance, it's not going to=




 consume any resources.  Most of the time the extra word isn't noticed =20
 because the memory size of a class is usually not exactly a power of 2.



Except when you put'em in an array. Could happen.


 D also allows you to replace it's monitor with a custom monitor object =20
 (i.e. core.sync.Mutex) so you can have more control over the mutex, assig=




 the same mutex to multiple objects, use conditions, etc.  It's much more =




 flexible than Java or C# IMO.



I didn't know, thx. Where is it documented?

--=20
Tomek
Feb 04 2011
prev sibling next sibling parent Tomek =?UTF-8?B?U293acWEc2tp?= <just ask.me> writes:
Tomek Sowi=C5=84ski napisa=C5=82:


 D's monitors are lazily created, so there should be no issue with resou=







 allocation.  If you don't ever lock an object instance, it's not going =







 consume any resources.  Most of the time the extra word isn't noticed =







 because the memory size of a class is usually not exactly a power of 2.=







=20
 Except when you put'em in an array. Could happen.



Sorry, for some reason I thought the mutex is on the stack.

--=20
Tomek
Feb 04 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Fri, 04 Feb 2011 07:26:22 -0500, bearophile <bearophileHUGS lycos.com>  
wrote:

 I have found an interesting post by Scott Johnson in this Lambda the  
 Ultimate thread:
 http://lambda-the-ultimate.org/node/724#comment-6621

 He says:


 9th circle: Concurrent mutable state. The obnoxious practice of  
 mutating shared state from multiple threads of control, leading into a  
 predictable cycle of race conditions, deadlocks, and other assorted  
 misbehavior from which there is no return. And if a correct solution  
 (for synchronization) is found for a given program, chances are any  
 substantial change to the program will make it incorrect again. But you  
 won't find it, instead your customer will. Despite that, reams of code  
 (and TONS of middleware) has been written to try and make this  
 tractable. And don't get me started on a certain programming language  
 which starts with "J" that saw fit to make EVERY object have its very  
 own monitor....<



 This is just one quotation, but I have found similar comments four or  
 five other times around the Web.

 So is the design choice of copying this part of the Java design inside D  
 good? I'd like opinions on this topic.

 Recently I have suggested an optional  nomonitor annotation for D  
 classes (to optionally remove a word from class instances and to reduce  
 class instantiation overhead a bit). Another option is doing the  
 opposite, and defining a  withmonitor annotation where you want a class  
 to have a monitor.



Hmm... Well, I'd recommend making  nomonitor the default and then only  
annotate certain classes  withmonitor, although I'd prefer a different  
keyword, say 'shared'. Oh, wait a second. *sigh* Every since it was  
decided that a class couldn't contain both shared and non-shared  
methods/fields I've been expecting that the monitor and support for  
synchronized methods would be removed from Object or at least from  
Object's spec. But this is likely a high-cost/low-gain optimization and a  
lot of things regarding shared/immutable classes need to be fixed before  
it can happen. I do think it should happen, not so much for the word of  
memory, but in order to prevent objects not designed to be shared from  
being shared. Which is what the 9th circle is talking about.

One of the cool thing about D's monitors, is that by manually setting  
them, you can protect multiple objects with a single monitor. So, for a  
set of interwoven objects (i.e. trees, etc) you're not acquiring a new  
lock every method call and you are not going to have an internal  
deadlock/race. This is actually the essential runtime feature of many  
ownership systems. (Although, without an actual ownership-type system you  
can't elide synchronization entirely)
Feb 04 2011