www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - shared/unshared classes

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
I was working on an I/O library that I plan to use in development, and  
possibly submit to phobos, and I thought of this case.

A standard file can be shared or unshared.  In C, since there is no notion  
of shared/unshared, everything is shared.  So any writes/reads from a FILE  
* lock the object.

But in D, we can avoid those locks when the file is unshared.  However,  
this means I have to write two pretty much identical functions for each  
call.

Is there an expected way to do this?  I've really avoided doing anything  
with shared or threads since the new concurrency model came out, but with  
I/O, I'll have to deal with it.

I think a logical thing to do would be to have the shared version of the  
function call the unshared version after locking the object.  Is that a  
good idea?  Is the correct way to do this to mark the shared function  
synchronized, and then cast 'this' to unshared to call the other  
function?  Does this automatically happen with shared functions?

Actually, this whole question is really general, because I'd expect to  
have to do this with any object that was shared -- putting memory barriers  
around accessing the object's members isn't enough to guarantee  
concurrency, and it makes no sense to do the memory barriers when the  
accesses are protected by a lock, right?

-Steve
Feb 19 2011
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development, and
 possibly submit to phobos, and I thought of this case.
 
 A standard file can be shared or unshared.  In C, since there is no notion
 of shared/unshared, everything is shared.  So any writes/reads from a FILE
 * lock the object.
 
 But in D, we can avoid those locks when the file is unshared.  However,
 this means I have to write two pretty much identical functions for each
 call.
 
 Is there an expected way to do this?  I've really avoided doing anything
 with shared or threads since the new concurrency model came out, but with
 I/O, I'll have to deal with it.
 
 I think a logical thing to do would be to have the shared version of the
 function call the unshared version after locking the object.  Is that a
 good idea?  Is the correct way to do this to mark the shared function
 synchronized, and then cast 'this' to unshared to call the other
 function?  Does this automatically happen with shared functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal. - Jonathan M Davis
Feb 19 2011
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development, and
 possibly submit to phobos, and I thought of this case.

 A standard file can be shared or unshared.  In C, since there is no  
 notion
 of shared/unshared, everything is shared.  So any writes/reads from a  
 FILE
 * lock the object.

 But in D, we can avoid those locks when the file is unshared.  However,
 this means I have to write two pretty much identical functions for each
 call.

 Is there an expected way to do this?  I've really avoided doing anything
 with shared or threads since the new concurrency model came out, but  
 with
 I/O, I'll have to deal with it.

 I think a logical thing to do would be to have the shared version of the
 function call the unshared version after locking the object.  Is that a
 good idea?  Is the correct way to do this to mark the shared function
 synchronized, and then cast 'this' to unshared to call the other
 function?  Does this automatically happen with shared functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

OK, I kind of remember that now. So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one? I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts. -Steve
Feb 19 2011
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 19 February 2011 19:12:17 Jonathan M Davis wrote:
 On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
 On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development, and
 possibly submit to phobos, and I thought of this case.
 
 A standard file can be shared or unshared.  In C, since there is no
 notion
 of shared/unshared, everything is shared.  So any writes/reads from a
 FILE
 * lock the object.
 
 But in D, we can avoid those locks when the file is unshared. 
 However, this means I have to write two pretty much identical
 functions for each call.
 
 Is there an expected way to do this?  I've really avoided doing
 anything with shared or threads since the new concurrency model came
 out, but with
 I/O, I'll have to deal with it.
 
 I think a logical thing to do would be to have the shared version of
 the function call the unshared version after locking the object.  Is
 that a good idea?  Is the correct way to do this to mark the shared
 function synchronized, and then cast 'this' to unshared to call the
 other function?  Does this automatically happen with shared
 functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

OK, I kind of remember that now. So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one? I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.

I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates.

Actually, now that I think about it, we're probably going to want a template of some kind in Phobos which wraps an unsynchronized class with a synchronized one. There are going to be plenty of cases where it would be nice to have an unsynchronized class made synchronized (a prime example would be a container class), and it would be annoying to have to keep creating your own wrappers every time that you want to do that. - Jonathan M Davis
Feb 19 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
 On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development, and
 possibly submit to phobos, and I thought of this case.
 
 A standard file can be shared or unshared.  In C, since there is no
 notion
 of shared/unshared, everything is shared.  So any writes/reads from a
 FILE
 * lock the object.
 
 But in D, we can avoid those locks when the file is unshared.  However,
 this means I have to write two pretty much identical functions for each
 call.
 
 Is there an expected way to do this?  I've really avoided doing anything
 with shared or threads since the new concurrency model came out, but
 with
 I/O, I'll have to deal with it.
 
 I think a logical thing to do would be to have the shared version of the
 function call the unshared version after locking the object.  Is that a
 good idea?  Is the correct way to do this to mark the shared function
 synchronized, and then cast 'this' to unshared to call the other
 function?  Does this automatically happen with shared functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

OK, I kind of remember that now. So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one? I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.

I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates. - Jonathan M Davis
Feb 19 2011
prev sibling next sibling parent Johannes Pfau <spam example.com> writes:
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Steven Schveighoffer wrote:
On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis
<jmdavisProg gmx.com> wrote:

 On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development,
 and possibly submit to phobos, and I thought of this case.

 A standard file can be shared or unshared.  In C, since there is
 no notion
 of shared/unshared, everything is shared.  So any writes/reads from
 a FILE
 * lock the object.

 But in D, we can avoid those locks when the file is unshared.
 However, this means I have to write two pretty much identical
 functions for each call.

 Is there an expected way to do this?  I've really avoided doing
 anything with shared or threads since the new concurrency model
 came out, but with
 I/O, I'll have to deal with it.

 I think a logical thing to do would be to have the shared version
 of the function call the unshared version after locking the
 object.  Is that a good idea?  Is the correct way to do this to
 mark the shared function synchronized, and then cast 'this' to
 unshared to call the other function?  Does this automatically
 happen with shared functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the =20 eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

OK, I kind of remember that now. So that means, I need to create two =20 identical hierarchies, a synchronized one and a non-synchronized one? I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts. -Steve

Creating two types, one synchronized, one not, doesn't sound like a good solution. If you need fine grained locking you can still do that manually (mark the function as shared, use the synchronized statement in the function). When I tried to do that for my std.signals replacement (which reminds me that I finally have to propose it for review) I found quite some bugs in the shared system. There's no shared signal because of those: http://d.puremagic.com/issues/show_bug.cgi?id=3D3733 This one is very annoying and I think I hit one or two cases where even the workaround didn't work. IIRC operators couldn't be overloaded with shared functions. Then I hit this issue: http://lists.puremagic.com/pipermail/digitalmars-d-learn/2010-November/0198= 58.html And I gave up ;-) --=20 Johannes Pfau
Feb 20 2011
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On 02/20/2011 04:41 AM, Jonathan M Davis wrote:
 On Saturday 19 February 2011 19:12:17 Jonathan M Davis wrote:
 On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
 On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis
 <jmdavisProg gmx.com>

 wrote:
 On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development, and
 possibly submit to phobos, and I thought of this case.

 A standard file can be shared or unshared.  In C, since there is no
 notion
 of shared/unshared, everything is shared.  So any writes/reads from a
 FILE
 * lock the object.

 But in D, we can avoid those locks when the file is unshared.
 However, this means I have to write two pretty much identical
 functions for each call.

 Is there an expected way to do this?  I've really avoided doing
 anything with shared or threads since the new concurrency model came
 out, but with
 I/O, I'll have to deal with it.

 I think a logical thing to do would be to have the shared version of
 the function call the unshared version after locking the object.  Is
 that a good idea?  Is the correct way to do this to mark the shared
 function synchronized, and then cast 'this' to unshared to call the
 other function?  Does this automatically happen with shared
 functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

OK, I kind of remember that now. So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one? I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.

I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates.

Actually, now that I think about it, we're probably going to want a template of some kind in Phobos which wraps an unsynchronized class with a synchronized one. There are going to be plenty of cases where it would be nice to have an unsynchronized class made synchronized (a prime example would be a container class), and it would be annoying to have to keep creating your own wrappers every time that you want to do that.

I have not had a look at D's threading implementation, but from what I read I thought what you describe precisely was the whole point of unshared-by-default and the 'shared' keyword. Eg I thought something like: class Foo {...} shared foo = new Foo(); would do what you write above. Else, what does 'shared' mean? Denis -- _________________ vita es estrany spir.wikidot.com
Feb 20 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 20 February 2011 05:36:25 spir wrote:
 On 02/20/2011 04:41 AM, Jonathan M Davis wrote:
 On Saturday 19 February 2011 19:12:17 Jonathan M Davis wrote:
 On Saturday 19 February 2011 19:01:16 Steven Schveighoffer wrote:
 On Sat, 19 Feb 2011 21:55:53 -0500, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On Saturday 19 February 2011 18:26:25 Steven Schveighoffer wrote:
 I was working on an I/O library that I plan to use in development,
 and possibly submit to phobos, and I thought of this case.
 
 A standard file can be shared or unshared.  In C, since there is no
 notion
 of shared/unshared, everything is shared.  So any writes/reads from a
 FILE
 * lock the object.
 
 But in D, we can avoid those locks when the file is unshared.
 However, this means I have to write two pretty much identical
 functions for each call.
 
 Is there an expected way to do this?  I've really avoided doing
 anything with shared or threads since the new concurrency model came
 out, but with
 I/O, I'll have to deal with it.
 
 I think a logical thing to do would be to have the shared version of
 the function call the unshared version after locking the object.  Is
 that a good idea?  Is the correct way to do this to mark the shared
 function synchronized, and then cast 'this' to unshared to call the
 other function?  Does this automatically happen with shared
 functions?

I would point out that per TDPL, either an entire class is synchronized or none of it is. You don't synchronize individual functions. Now, I don' think that that's the way that it's implemented at the moment, but that's the eventual situation as I understand it. So, your class shouldn't have a mixture of synchronized or unsynchronized. According to TDPL, it's illegal.

OK, I kind of remember that now. So that means, I need to create two identical hierarchies, a synchronized one and a non-synchronized one? I think what I'll do for now is write the non-synchronized versions, and then see about maybe automating the synchronized parts.

I'd have to study up on it more to know what the best way to handle it is (I've generally avoided dealing with shared), but what you'd probably end up doing is creating a wrapper class which was synchronized. However, you might be able to get it to just automatically use the right one (unsynchronized for unshared and synchronized for shared) if you use templates.

Actually, now that I think about it, we're probably going to want a template of some kind in Phobos which wraps an unsynchronized class with a synchronized one. There are going to be plenty of cases where it would be nice to have an unsynchronized class made synchronized (a prime example would be a container class), and it would be annoying to have to keep creating your own wrappers every time that you want to do that.

I have not had a look at D's threading implementation, but from what I read I thought what you describe precisely was the whole point of unshared-by-default and the 'shared' keyword. Eg I thought something like: class Foo {...} shared foo = new Foo(); would do what you write above. Else, what does 'shared' mean?

shared means that it's not thread local, and that means that you need to worry about mutexes and synchronized and the like. So, Steve's concern is that if you use the class and it's shared, it needs to be synchronized. But if it's not shared, you don't want the overhead of synchronization. So, you make a version which is not synchronized and then whoever uses it has to worry about using mutexes to protect it. Creating a wrapper class which is synchronized is a good way to do so. Having a template which produced such a wrapper for you would be particularly useful so that you don't have to write a new wrapper class by hand every time that you need have a synchronized version of an unsynchronized class. - Jonathan M Davis - Jonathan M Davis
Feb 20 2011