www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - pass array of objects to spawn

reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
Hi folks,

   I need to create thread and pass to it array of objects wich will no 
longer use in main thread.

   The problem is that spawn only accepts immutables and i have no idea 
how to tell him that i transfer full ownership of the data to child 
thread.

   the code:

import std.concurrency;

class Foo
{

}

void worker( Foo[] data )
{
	//...
}

void main()
{
	auto data = new Foo[10];
	spawn( &worker, data );
}


P.S. I do need to create data in main thread and distribute it to child 
threads.

-- 
BR, Ruslan Mullakhmetov
Nov 10 2011
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 11/10/2011 01:06 PM, Ruslan Mullakhmetov wrote:
 Hi folks,

 I need to create thread and pass to it array of objects wich will no
 longer use in main thread.

 The problem is that spawn only accepts immutables and i have no idea how
 to tell him that i transfer full ownership of the data to child thread.

 the code:

 import std.concurrency;

 class Foo
 {

 }

 void worker( Foo[] data )
 {
 //...
 }

 void main()
 {
 auto data = new Foo[10];
 spawn( &worker, data );
 }


 P.S. I do need to create data in main thread and distribute it to child
 threads.

I haven't tested these at runtime, but they compile: 1) (This is not actually what you want, as you want to mutate the array further in the worker.) Take array as 'immutable' and use assumeUnique (I don't know why it's a part of std.exception :)): import std.exception; import std.concurrency; class Foo { } void worker( immutable(Foo)[] data ) { //... } void main() { auto data = new Foo[10]; spawn( &worker, assumeUnique(data) ); } 2) Make data 'shared' and take the responsibility of ownership and sharing: import std.exception; import std.concurrency; class Foo { } void worker( shared(Foo)[] data ) { //... } void main() { auto data = new shared(Foo)[10]; spawn( &worker, data ); } Ali
Nov 10 2011
parent reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
On 2011-11-11 01:21:09 +0400, Ali Çehreli said:

 class Foo
 {
 
 }
 
 void worker( shared(Foo)[] data )
 {
      //...
 }
 
 void main()
 {
      auto data = new shared(Foo)[10];
      spawn( &worker, data );
 }

Thanks. I tried to use the second version, a lttle bit modified it for actual mutation and it is failed to compile with error thread.d(17): Error: function thread.Foo.mutate () is not callable using argument types () shared the code: import std.exception; import std.concurrency; class Foo { public int val; void mutate() { val = 1; } } void worker( shared(Foo)[] data ) { data[0].mutate(); //... } void main() { auto data = new shared(Foo)[10]; spawn( &worker, data ); } -- BR, Ruslan Mullakhmetov
Nov 10 2011
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 11/10/2011 01:57 PM, Ruslan Mullakhmetov wrote:
 On 2011-11-11 01:21:09 +0400, Ali Çehreli said:

 class Foo
 {

 }

 void worker( shared(Foo)[] data )
 {
 //...
 }

 void main()
 {
 auto data = new shared(Foo)[10];
 spawn( &worker, data );
 }

Thanks. I tried to use the second version, a lttle bit modified it for actual mutation and it is failed to compile with error thread.d(17): Error: function thread.Foo.mutate () is not callable using argument types () shared the code: import std.exception; import std.concurrency; class Foo { public int val; void mutate() { val = 1; } } void worker( shared(Foo)[] data ) { data[0].mutate(); //... } void main() { auto data = new shared(Foo)[10]; spawn( &worker, data ); }

Some of this is black magic for me. Timon's casts help: import std.concurrency; class Foo { public int val; void mutate() { val = 1; } } void worker( shared(Foo)[] data_arg ) { auto data = cast(Foo[])data_arg; data[0].mutate(); //... } void main() { auto data = new shared(Foo)[10]; foreach (ref element; data) { element = new shared(Foo); } spawn( &worker, data ); } Ali
Nov 10 2011
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 11/10/2011 11:23 PM, Ali Çehreli wrote:
 On 11/10/2011 01:57 PM, Ruslan Mullakhmetov wrote:
 On 2011-11-11 01:21:09 +0400, Ali Çehreli said:

 class Foo
 {

 }

 void worker( shared(Foo)[] data )
 {
 //...
 }

 void main()
 {
 auto data = new shared(Foo)[10];
 spawn( &worker, data );
 }

Thanks. I tried to use the second version, a lttle bit modified it for actual mutation and it is failed to compile with error thread.d(17): Error: function thread.Foo.mutate () is not callable using argument types () shared the code: import std.exception; import std.concurrency; class Foo { public int val; void mutate() { val = 1; } } void worker( shared(Foo)[] data ) { data[0].mutate(); //... } void main() { auto data = new shared(Foo)[10]; spawn( &worker, data ); }

Some of this is black magic for me. Timon's casts help: import std.concurrency; class Foo { public int val; void mutate() { val = 1; } } void worker( shared(Foo)[] data_arg ) { auto data = cast(Foo[])data_arg; data[0].mutate(); //... } void main() { auto data = new shared(Foo)[10]; foreach (ref element; data) { element = new shared(Foo); } spawn( &worker, data ); } Ali

class Foo { public int val; void mutate() shared { val = 1; } } But you really really only want the shared modifier to persist after message passing if the data is not owned by a single thread afterwards.
Nov 10 2011
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/10/2011 10:06 PM, Ruslan Mullakhmetov wrote:
 Hi folks,

 I need to create thread and pass to it array of objects wich will no
 longer use in main thread.

 The problem is that spawn only accepts immutables and i have no idea how
 to tell him that i transfer full ownership of the data to child thread.

 the code:

 import std.concurrency;

 class Foo
 {

 }

 void worker( Foo[] data )
 {
 //...
 }

 void main()
 {
 auto data = new Foo[10];
 spawn( &worker, data );
 }


 P.S. I do need to create data in main thread and distribute it to child
 threads.

There is no checkable type safe way to do this (because the type system is not powerful enough). You have to use type casts and work for the correctness guarantees yourself: class Foo { } void worker( shared(Foo[]) data_ ) { Foo[] data = cast() data_; // this cast is valid because data_ is never read from another thread after the cast //... } void main() { { auto data = new Foo[10]; spawn( &worker, cast(shared)data ); // this cast is valid because data is an unique reference (therefore there are no unshared aliases) } // the sole reference to data in the main thread dies -> it will never be read from this thread again }
Nov 10 2011
parent reply Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
On 2011-11-11 01:23:01 +0400, Timon Gehr said:

 class Foo
 {
 
 }
 
 void worker( shared(Foo[]) data_ )
 {
      Foo[] data = cast() data_; // this cast is valid because data_ is 
 never read from another thread after the cast
      //...
 }
 
 void main()
 {
      {
          auto data = new Foo[10];
          spawn( &worker, cast(shared)data ); // this cast is valid 
 because data is an unique reference (therefore there are no unshared 
 aliases)
      } // the sole reference to data in the main thread dies -> it will 
 never be read from this thread again
 }

Thank you too. Unfortunately i got compilation error thread.d(16): Error: cannot implicitly convert expression (data_) of type shared(Foo)[] to Foo[] -- BR, Ruslan Mullakhmetov
Nov 10 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11/10/2011 11:00 PM, Ruslan Mullakhmetov wrote:
 On 2011-11-11 01:23:01 +0400, Timon Gehr said:

 class Foo
 {

 }

 void worker( shared(Foo[]) data_ )
 {
 Foo[] data = cast() data_; // this cast is valid because data_ is
 never read from another thread after the cast
 //...
 }

 void main()
 {
 {
 auto data = new Foo[10];
 spawn( &worker, cast(shared)data ); // this cast is valid because data
 is an unique reference (therefore there are no unshared aliases)
 } // the sole reference to data in the main thread dies -> it will
 never be read from this thread again
 }

Thank you too. Unfortunately i got compilation error thread.d(16): Error: cannot implicitly convert expression (data_) of type shared(Foo)[] to Foo[]

Interesting, apparently cast() does not remove shared. Sorry about that, use this (this time I tested it). import std.concurrency; class Foo { } void worker( shared(Foo[]) data_ ) { auto data = cast(Foo[]) data_; //... } void main() { { auto data = new Foo[10]; spawn( &worker, cast(shared)data ); } }
Nov 10 2011
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Timon Gehr:

 Interesting, apparently cast() does not remove shared.

The semantics of cast() was never formalized, I think. People use this idiom, so I think we have to formalize what cast() exactly does. This is meat for Bugzilla or even for discussion topic in the main D newsgroup. Generally I don't love cast() a lot, despite it's handy, because it's under-specified, too much powerful, and too much generic (I prefer something like Deconst!(), Deshared!(), etc templates in Phobos, that do only one thing). Bye, bearophile
Nov 10 2011
prev sibling parent Ruslan Mullakhmetov <tiabaldu gmail.com> writes:
On 2011-11-11 02:48:52 +0400, Timon Gehr said:

 On 11/10/2011 11:00 PM, Ruslan Mullakhmetov wrote:
 On 2011-11-11 01:23:01 +0400, Timon Gehr said:
 
 class Foo
 {
 
 }
 
 void worker( shared(Foo[]) data_ )
 {
 Foo[] data = cast() data_; // this cast is valid because data_ is
 never read from another thread after the cast
 //...
 }
 
 void main()
 {
 {
 auto data = new Foo[10];
 spawn( &worker, cast(shared)data ); // this cast is valid because data
 is an unique reference (therefore there are no unshared aliases)
 } // the sole reference to data in the main thread dies -> it will
 never be read from this thread again
 }

Thank you too. Unfortunately i got compilation error thread.d(16): Error: cannot implicitly convert expression (data_) of type shared(Foo)[] to Foo[]

Interesting, apparently cast() does not remove shared. Sorry about that, use this (this time I tested it). import std.concurrency; class Foo { } void worker( shared(Foo[]) data_ ) { auto data = cast(Foo[]) data_; //... } void main() { { auto data = new Foo[10]; spawn( &worker, cast(shared)data ); } }

thank you very much. For now it's working. But it would be quite interesting to add explicit ownership semantics to the language as Ali sugested once by keyword unique. By the way, I realized that it would be better to create data in thread rather pass it to thread and pass some seeds for creating data. Of course, this is one case, sometimes transfer of ownership is not avoidable as it seems to me. -- BR, Ruslan Mullakhmetov
Nov 10 2011