www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Concurrency message passing

reply JG <JG somewhere.com> writes:
Hi

I have a program with two threads. One thread produces data that 
is put in a queue
and then consumed by the other thread. I initially built a custom 
queue to do this, but thought this should have some standard 
solution in D? I looked at std.concurrency and thought that 
message passing could be used. However, the problem is that I get 
the following error.

Error: static assert:  "Aliases to mutable thread-local data not 
allowed."

I am not sure how to solve this.  Maybe message parsing isn't the 
correct solution?
Is there some standard solution to this in D?
Aug 17 2021
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Tuesday, 17 August 2021 at 11:05:09 UTC, JG wrote:

 Hi

 I have a program with two threads. One thread produces data 
 that is put in a queue
 and then consumed by the other thread. I initially built a 
 custom queue to do this, but thought this should have some 
 standard solution in D? I looked at std.concurrency and thought 
 that message passing could be used. However, the problem is 
 that I get the following error.

 Error: static assert:  "Aliases to mutable thread-local data 
 not allowed."

 I am not sure how to solve this.


The error tells you what you can't do. You can do anything other 
than that. What you *should* do depends on what exactly you're 
trying to do. Here are some examples:

Sending shared mutable data:

```d
import std.concurrency, std.stdio, core.atomic;

void incr() {
     auto counts = receiveOnly!(shared(int)[]);
     foreach (ref n; counts)
         atomicOp!"+="(n, 1); // shared(int) can't just be +='d
     ownerTid.send(true);
}

void main() {
     shared(int)[] counts = [0, 0, 0]; // no issues passing this
     spawn(&incr).send(counts);
     receiveOnly!bool;
     writeln(counts);
}
```

Sending immutable data:

```d
import std.concurrency, std.stdio, std.typecons;

void greeter() {
     auto who = receiveOnly!(string);
     writeln("Hello, ", who);
}

void main() {
     char[] who = "John".dup; // mutable&thread-local, can't be 
sent
     spawn(&greeter).send(who.idup); // create immutable copy to 
send
}
```

Sending scalar data:

```d
import std.concurrency, std.stdio;

__gshared int[3] counts;

void incr() {
     auto indexes = receiveOnly!(int, int);
     foreach (ref n; counts[indexes[0] .. indexes[1]])
         n++;
     ownerTid.send(true);
}

void main() {
     spawn(&incr).send(1, 2);
     receiveOnly!bool;
     writeln(counts);
}
```
Aug 17 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/17/21 7:05 AM, JG wrote:

 Hi
 
 I have a program with two threads. One thread produces data that is put 
 in a queue
 and then consumed by the other thread. I initially built a custom queue 
 to do this, but thought this should have some standard solution in D? I 
 looked at std.concurrency and thought that message passing could be 
 used. However, the problem is that I get the following error.
 
 Error: static assert:  "Aliases to mutable thread-local data not allowed."
 
 I am not sure how to solve this.  Maybe message parsing isn't the 
 correct solution?
 Is there some standard solution to this in D?


Data with references needs to be marked either immutable or shared in 
order to be passed using std.concurrency. D is strict about not sharing 
thread-local data, because then you can use the type system to prove 
lock-free code is valid.

However, data that has no references (aliases) should be passable 
regardless of mutability, because you are passing a copy.

-Steve
Aug 17 2021
parent reply JG <JG somewhere.com> writes:
On Tuesday, 17 August 2021 at 12:24:14 UTC, Steven Schveighoffer 
wrote:

 On 8/17/21 7:05 AM, JG wrote:

 Hi
 
 I have a program with two threads. One thread produces data 
 that is put in a queue
 and then consumed by the other thread. I initially built a 
 custom queue to do this, but thought this should have some 
 standard solution in D? I looked at std.concurrency and 
 thought that message passing could be used. However, the 
 problem is that I get the following error.
 
 Error: static assert:  "Aliases to mutable thread-local data 
 not allowed."
 
 I am not sure how to solve this.  Maybe message parsing isn't 
 the correct solution?
 Is there some standard solution to this in D?


 Data with references needs to be marked either immutable or 
 shared in order to be passed using std.concurrency. D is strict 
 about not sharing thread-local data, because then you can use 
 the type system to prove lock-free code is valid.

 However, data that has no references (aliases) should be 
 passable regardless of mutability, because you are passing a 
 copy.

 -Steve


Thanks for the suggestions and explanations. I am not sure what 
to do in my case though. The situation is as follows. I have a 
struct that is populated via user input not necessarily at single 
instance (so that seems to rule out immutable). On the other
hand while it is being populate it is only accessible from one 
thread so that
makes using shared messy. After being populated it should be 
passed to the other thread and no references are kept.

What I am doing currently is populating the struct and casting to 
shared when I push into a synchronized queue (no references to 
its data are kept in the first thread). Is what I am doing wrong 
and can it be achieved using message passing?
Aug 17 2021
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 8/17/21 11:36 AM, JG wrote:


 Maybe message parsing isn't the
 correct solution?






I use message passing in many of my programs.


 After being populated it should be passed to
 the other thread and no references are kept.


Then you simply cast to-and-from 'shared' and be happy with it. :) I 
explain what I do here:

   https://www.youtube.com/watch?v=dRORNQIB2wA&t=1735s

Ali
Aug 17 2021
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 8/17/21 2:36 PM, JG wrote:

 Thanks for the suggestions and explanations. I am not sure what to do in 
 my case though. The situation is as follows. I have a struct that is 
 populated via user input not necessarily at single instance (so that 
 seems to rule out immutable). On the other
 hand while it is being populate it is only accessible from one thread so 
 that
 makes using shared messy. After being populated it should be passed to 
 the other thread and no references are kept.


You are allowed to cast to immutable if no other mutable references are 
used. I recommend using `assumeUnique`.


 What I am doing currently is populating the struct and casting to shared 
 when I push into a synchronized queue (no references to its data are 
 kept in the first thread). Is what I am doing wrong and can it be 
 achieved using message passing?


Yeah, build it like Ali says, and then cast in order to pass it.

-Steve
Aug 17 2021