www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - std.concurrency.spawn does not accept delegates

reply teo <teo.ubuntu yahoo.com> writes:
It looks like std.concurrency.spawn does not accept delegates. Is there 
any reason for that?
Jul 17 2011
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday 17 July 2011 19:29:02 teo wrote:
 It looks like std.concurrency.spawn does not accept delegates. Is there
 any reason for that?

Probably because it would have to accept a delegate where every variable that it had access too outside of its own scope was immutable, and I don't think that there's any way for the compiler to make such guarantees with a delegate. - Jonathan M Davis
Jul 17 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com> wrote:

 It looks like std.concurrency.spawn does not accept delegates. Is there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn cannot guarantee it doesn't point to unshared data. -Steve
Jul 18 2011
prev sibling next sibling parent teo <teo.ubuntu yahoo.com> writes:
On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:

 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com> wrote:
 
 It looks like std.concurrency.spawn does not accept delegates. Is there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn cannot guarantee it doesn't point to unshared data. -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.
Jul 18 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com> wrote:
 It looks like std.concurrency.spawn does not accept delegates. Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn cannot guarantee it doesn't point to unshared data. -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of it ending up getting altered by both threads (or altered by one when the other one is using at it). Sometimes, that can be a bit restrictive (particularly when you _know_ that something isn't going to be altered by the thread sending it after its sent but the compiler doesn't), but it avoids all kinds of problems. If you want to send a class object across, then it needs to be immutable (which tends to be a bit of a pain to do for classes, since they need to have an immutable constructor, which is often a pain to do). - Jonathan M Davis
Jul 18 2011
prev sibling next sibling parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com> wrote:
 It looks like std.concurrency.spawn does not accept delegates. Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

 guarantee it doesn't point to unshared data.

 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of it ending up getting altered by both threads (or altered by one when the other one is using at it). Sometimes, that can be a bit restrictive (particularly when you _know_ that something isn't going to be altered by the thread sending it after its sent but the compiler doesn't), but it avoids all kinds of problems. If you want to send a class object across, then it needs to be immutable (which tends to be a bit of a pain to do for classes, since they need to have an immutable constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request. -- Simen
Jul 18 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-18 10:54, Simen Kjaeraas wrote:
 On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com> wrote:
 It looks like std.concurrency.spawn does not accept delegates. Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

cannot
 guarantee it doesn't point to unshared data.
 
 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of it ending up getting altered by both threads (or altered by one when the other one is using at it). Sometimes, that can be a bit restrictive (particularly when you _know_ that something isn't going to be altered by the thread sending it after its sent but the compiler doesn't), but it avoids all kinds of problems. If you want to send a class object across, then it needs to be immutable (which tends to be a bit of a pain to do for classes, since they need to have an immutable constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request.

There have been discussions about how to do it in the past. Whether assumeUnique will work depends on whether casting to immutable(C) will work (where C is the class' type), and I don't know whether that cast will work or not. If it does, then assumeUnique will do it, since all it does is cast to immutable, but it's the sort of thing that requires language support. There _might_ be a way to solve the problem with some sort of Unique template that spawn and send knew about, but nothing of the sort has been done yet. But until a number of the issues with const and immutable in the compiler have been sorted out, that sort of thing would probably be problematic anyway. - Jonathan M Davis
Jul 18 2011
prev sibling next sibling parent teo <teo.ubuntu yahoo.com> writes:
On Mon, 18 Jul 2011 18:14:45 +0000, Jonathan M Davis wrote:

 On 2011-07-18 10:54, Simen Kjaeraas wrote:
 On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com>
 wrote:
 It looks like std.concurrency.spawn does not accept delegates. Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

cannot
 guarantee it doesn't point to unshared data.
 
 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of it ending up getting altered by both threads (or altered by one when the other one is using at it). Sometimes, that can be a bit restrictive (particularly when you _know_ that something isn't going to be altered by the thread sending it after its sent but the compiler doesn't), but it avoids all kinds of problems. If you want to send a class object across, then it needs to be immutable (which tends to be a bit of a pain to do for classes, since they need to have an immutable constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request.

There have been discussions about how to do it in the past. Whether assumeUnique will work depends on whether casting to immutable(C) will work (where C is the class' type), and I don't know whether that cast will work or not. If it does, then assumeUnique will do it, since all it does is cast to immutable, but it's the sort of thing that requires language support. There _might_ be a way to solve the problem with some sort of Unique template that spawn and send knew about, but nothing of the sort has been done yet. But until a number of the issues with const and immutable in the compiler have been sorted out, that sort of thing would probably be problematic anyway. - Jonathan M Davis

This is a bit too restrictive in my opinion. Only the shared data between two threads should be immutable. But the threads can access all sorts of mutable data as well. And in this case we are actually talking about the control function (or start routine) of a thread. As long as it's address is fixed within the memory of a process its usage for that purpose should be fine. Nobody is going to pass that address around. Am I missing anything here?
Jul 18 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-18 15:15, teo wrote:
 On Mon, 18 Jul 2011 18:14:45 +0000, Jonathan M Davis wrote:
 On 2011-07-18 10:54, Simen Kjaeraas wrote:
 On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com>
 
 wrote:
 It looks like std.concurrency.spawn does not accept delegates. Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

cannot
 guarantee it doesn't point to unshared data.
 
 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of it ending up getting altered by both threads (or altered by one when the other one is using at it). Sometimes, that can be a bit restrictive (particularly when you _know_ that something isn't going to be altered by the thread sending it after its sent but the compiler doesn't), but it avoids all kinds of problems. If you want to send a class object across, then it needs to be immutable (which tends to be a bit of a pain to do for classes, since they need to have an immutable constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request.

There have been discussions about how to do it in the past. Whether assumeUnique will work depends on whether casting to immutable(C) will work (where C is the class' type), and I don't know whether that cast will work or not. If it does, then assumeUnique will do it, since all it does is cast to immutable, but it's the sort of thing that requires language support. There _might_ be a way to solve the problem with some sort of Unique template that spawn and send knew about, but nothing of the sort has been done yet. But until a number of the issues with const and immutable in the compiler have been sorted out, that sort of thing would probably be problematic anyway. - Jonathan M Davis

This is a bit too restrictive in my opinion. Only the shared data between two threads should be immutable. But the threads can access all sorts of mutable data as well. And in this case we are actually talking about the control function (or start routine) of a thread. As long as it's address is fixed within the memory of a process its usage for that purpose should be fine. Nobody is going to pass that address around. Am I missing anything here?

When passing data between threads, it must be immutable. If it weren't, then you'd have to worry about mutexes and the like. Data is thread-local by default, so one thread does _not_ have access to the data in another thread unless it's shared. spawn starts a new thread with the data that it's given, and send allows you to send data to another thread, but if it's not immutable, then you're running into issues when multiple threads are dealing with mutabel data and could change it. And that's not allowed unless the data is shared - in which case there's no need for send, and you have to use mutexes or synchronized blocks to control access to it, ord you're going to have concurrency bugs. - Jonathan M Davis
Jul 18 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Jul 2011 18:39:01 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On 2011-07-18 15:15, teo wrote:
 On Mon, 18 Jul 2011 18:14:45 +0000, Jonathan M Davis wrote:
 On 2011-07-18 10:54, Simen Kjaeraas wrote:
 On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis
 <jmdavisProg gmx.com>

 wrote:
 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com>

 wrote:
 It looks like std.concurrency.spawn does not accept delegates.  






 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

cannot
 guarantee it doesn't point to unshared data.

 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of



 ending up
 getting altered by both threads (or altered by one when the other  



 is using
 at it). Sometimes, that can be a bit restrictive (particularly when
 you _know_
 that something isn't going to be altered by the thread sending it
 after its
 sent but the compiler doesn't), but it avoids all kinds of  



 If you
 want to send a class object across, then it needs to be immutable
 (which tends
 to be a bit of a pain to do for classes, since they need to have an
 immutable
 constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request.

There have been discussions about how to do it in the past. Whether assumeUnique will work depends on whether casting to immutable(C) will work (where C is the class' type), and I don't know whether that cast will work or not. If it does, then assumeUnique will do it, since all

 does is cast to immutable, but it's the sort of thing that requires
 language support. There _might_ be a way to solve the problem with  

 sort of Unique template that spawn and send knew about, but nothing of
 the sort has been done yet. But until a number of the issues with  

 and immutable in the compiler have been sorted out, that sort of thing
 would probably be problematic anyway.

 - Jonathan M Davis

This is a bit too restrictive in my opinion. Only the shared data between two threads should be immutable. But the threads can access all sorts of mutable data as well. And in this case we are actually talking about the control function (or start routine) of a thread. As long as it's address is fixed within the memory of a process its usage for that purpose should be fine. Nobody is going to pass that address around. Am I missing anything here?

When passing data between threads, it must be immutable.

I have to jump in and correct you, nobody else has. You can also pass data marked as shared. A solution could be to cast the class as shared, pass it, then cast it back to unshared (ensuring you don't access the class from the originator anymore). This is not a compiler-enforced solution, but it gets the job done. But there is risk of concurrency errors if you don't do it right. My recommendation is to isolate the parts that create and pass the shared data. -Steve
Jul 19 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-07-19 05:40, Steven Schveighoffer wrote:
 On Mon, 18 Jul 2011 18:39:01 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:
 On 2011-07-18 15:15, teo wrote:
 On Mon, 18 Jul 2011 18:14:45 +0000, Jonathan M Davis wrote:
 On 2011-07-18 10:54, Simen Kjaeraas wrote:
 On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo <teo.ubuntu yahoo.com>
 
 wrote:
 It looks like std.concurrency.spawn does not accept delegates.






Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

cannot
 guarantee it doesn't point to unshared data.
 
 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of



it
 ending up
 getting altered by both threads (or altered by one when the other



one
 is using
 at it). Sometimes, that can be a bit restrictive (particularly when
 you _know_
 that something isn't going to be altered by the thread sending it
 after its
 sent but the compiler doesn't), but it avoids all kinds of



problems.
 If you
 want to send a class object across, then it needs to be immutable
 (which tends
 to be a bit of a pain to do for classes, since they need to have an
 immutable
 constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request.

There have been discussions about how to do it in the past. Whether assumeUnique will work depends on whether casting to immutable(C) will work (where C is the class' type), and I don't know whether that cast will work or not. If it does, then assumeUnique will do it, since all

it
 does is cast to immutable, but it's the sort of thing that requires
 language support. There _might_ be a way to solve the problem with

some
 sort of Unique template that spawn and send knew about, but nothing of
 the sort has been done yet. But until a number of the issues with

const
 and immutable in the compiler have been sorted out, that sort of thing
 would probably be problematic anyway.
 
 - Jonathan M Davis

This is a bit too restrictive in my opinion. Only the shared data between two threads should be immutable. But the threads can access all sorts of mutable data as well. And in this case we are actually talking about the control function (or start routine) of a thread. As long as it's address is fixed within the memory of a process its usage for that purpose should be fine. Nobody is going to pass that address around. Am I missing anything here?

When passing data between threads, it must be immutable.

I have to jump in and correct you, nobody else has. You can also pass data marked as shared. A solution could be to cast the class as shared, pass it, then cast it back to unshared (ensuring you don't access the class from the originator anymore). This is not a compiler-enforced solution, but it gets the job done. But there is risk of concurrency errors if you don't do it right. My recommendation is to isolate the parts that create and pass the shared data.

I thought that spawn and send disallowed shared and that you had to deal with shared separately, but if that's not the case, then that's not the case. Regardless, you can't just pass anything with spawn or send (as the OP seems to be trying to do). They have restrictions to avoid concurrency bugs. - Jonathan M Davis
Jul 19 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 19 Jul 2011 14:31:19 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:

 On 2011-07-19 05:40, Steven Schveighoffer wrote:
 On Mon, 18 Jul 2011 18:39:01 -0400, Jonathan M Davis I have to jump in  
 and correct you, nobody else has.

 You can also pass data marked as shared.

 A solution could be to cast the class as shared, pass it, then cast it
 back to unshared (ensuring you don't access the class from the  
 originator
 anymore).

 This is not a compiler-enforced solution, but it gets the job done. But
 there is risk of concurrency errors if you don't do it right. My
 recommendation is to isolate the parts that create and pass the shared
 data.

I thought that spawn and send disallowed shared and that you had to deal with shared separately, but if that's not the case, then that's not the case. Regardless, you can't just pass anything with spawn or send (as the OP seems to be trying to do). They have restrictions to avoid concurrency bugs.

send requires the data pass this test: !hasLocalAliasing!(T) which maps to: http://www.digitalmars.com/d/2.0/phobos/std_traits.html#hasUnsharedAliasing So yes, you can pass shared data. However, there is still no sanctioned way to "pass and forget" mutable unique thread-local data. That is, you pass data to another thread, and it becomes local to that other thread instead of to the thread passed from. You are correct that you can't just pass anything (i.e. thread-local mutable data), but in some cases, you have to force the issue. Just because the compiler can't prove it's valid doesn't mean it isn't. But it's definitely opening up a possibility for concurrency bugs. -Steve
Jul 19 2011
prev sibling parent teo <teo.ubuntu yahoo.com> writes:
On Tue, 19 Jul 2011 18:31:19 +0000, Jonathan M Davis wrote:

 On 2011-07-19 05:40, Steven Schveighoffer wrote:
 On Mon, 18 Jul 2011 18:39:01 -0400, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On 2011-07-18 15:15, teo wrote:
 On Mon, 18 Jul 2011 18:14:45 +0000, Jonathan M Davis wrote:
 On 2011-07-18 10:54, Simen Kjaeraas wrote:
 On Mon, 18 Jul 2011 18:06:46 +0200, Jonathan M Davis
 <jmdavisProg gmx.com>
 
 wrote:
 On Monday 18 July 2011 15:55:52 teo wrote:
 On Mon, 18 Jul 2011 10:26:27 -0400, Steven Schveighoffer
 wrote:
 On Sun, 17 Jul 2011 15:29:02 -0400, teo
 <teo.ubuntu yahoo.com>
 
 wrote:
 It looks like std.concurrency.spawn does not accept
 delegates.






Is
 there
 any reason for that?

There is no type attached to the hidden 'this' pointer. So spawn

cannot
 guarantee it doesn't point to unshared data.
 
 -Steve

Bad. I tried to pass as an argument a pointer to an instance of a class - a this pointer. That didn't work.

Only stuff that's immutable or implicitly convertible immutable can be passed across threads using spawn and send. Otherwise, there's a risk of



it
 ending up
 getting altered by both threads (or altered by one when the
 other



one
 is using
 at it). Sometimes, that can be a bit restrictive (particularly
 when you _know_
 that something isn't going to be altered by the thread sending
 it after its
 sent but the compiler doesn't), but it avoids all kinds of



problems.
 If you
 want to send a class object across, then it needs to be
 immutable (which tends
 to be a bit of a pain to do for classes, since they need to
 have an immutable
 constructor, which is often a pain to do).

It could be that assumeUnique should handle this, by transforming the delegate into something spawn() and friends could handle. Might be worth an enhancement request.

There have been discussions about how to do it in the past. Whether assumeUnique will work depends on whether casting to immutable(C) will work (where C is the class' type), and I don't know whether that cast will work or not. If it does, then assumeUnique will do it, since all

it
 does is cast to immutable, but it's the sort of thing that
 requires language support. There _might_ be a way to solve the
 problem with

some
 sort of Unique template that spawn and send knew about, but
 nothing of the sort has been done yet. But until a number of the
 issues with

const
 and immutable in the compiler have been sorted out, that sort of
 thing would probably be problematic anyway.
 
 - Jonathan M Davis

This is a bit too restrictive in my opinion. Only the shared data between two threads should be immutable. But the threads can access all sorts of mutable data as well. And in this case we are actually talking about the control function (or start routine) of a thread. As long as it's address is fixed within the memory of a process its usage for that purpose should be fine. Nobody is going to pass that address around. Am I missing anything here?

When passing data between threads, it must be immutable.

I have to jump in and correct you, nobody else has. You can also pass data marked as shared. A solution could be to cast the class as shared, pass it, then cast it back to unshared (ensuring you don't access the class from the originator anymore). This is not a compiler-enforced solution, but it gets the job done. But there is risk of concurrency errors if you don't do it right. My recommendation is to isolate the parts that create and pass the shared data.

I thought that spawn and send disallowed shared and that you had to deal with shared separately, but if that's not the case, then that's not the case. Regardless, you can't just pass anything with spawn or send (as the OP seems to be trying to do). They have restrictions to avoid concurrency bugs. - Jonathan M Davis

Well, I tried to pass a this pointer, because I wasn't able to use a delegate as a control function of the thread. All I need is access to the data within a class instance. That isn't static data. Basically I have an object which encapsulates the access to some resource. Let's call it a generator. One can configure some parameters and later just switch it on. That should be in a worker thread, because some operations are blocking. I also need certain events to be communicated to other threads within the process. Passing messages is fine, because the data is immutable. Now with spawn I have to use either a regular function or a static method of the class. The only other option that comes to my mind is to copy the implementation of the message box from std.concurrency and use the core.thread.Thread directly instead of spawn. Perhaps that will work.
Jul 19 2011