www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating immutable data and sharing it

reply Lutger <lutger.blijdestijn gmail.com> writes:
I'm still a bit fuzzy on how to create immutable data and when said data is
safe 
to share across threads.

To begin with, there is .idup, string literals and constructors of immutable 
objects. Those can be safely shared, no problem, right?

But then, the spec mentions casting to immutable is ok if you do not have any 
mutable aliases left (http://www.digitalmars.com/d/2.0/const3.html):

char[] s = ...;
immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

I do not understand how that works with sharing. Since immutable data is 
implicitly shared but the data that p refers to is allocated on the TLS, how
can 
you share this? I always thought that threads do not have access to each others 
TLS at all?

Finally a practical question: when you have a data structure that is too
complex 
to create in a constructor, want to create it and then make it immutable, what 
is the current way to go about this? If it is created by one thread, would it
be 
ok to type it as __gshared, cast to immutable and then send a message to other 
threads? Is __gshared required in this case? 
Sep 21 2010
next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:

 char[] s = ...;
 immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

 I do not understand how that works with sharing. Since immutable data is
 implicitly shared but the data that p refers to is allocated on the TLS,  
 how can
 you share this? I always thought that threads do not have access to each  
 others
 TLS at all?

Only p itself is in TLS - the pointed-to data is on the heap.
 Finally a practical question: when you have a data structure that is too  
 complex
 to create in a constructor, want to create it and then make it  
 immutable, what
 is the current way to go about this? If it is created by one thread,  
 would it be
 ok to type it as __gshared, cast to immutable and then send a message to  
 other
 threads? Is __gshared required in this case?

Immutable global state may be instantiated from non-immutable data in module constructors. I believe that is the canonical way. -- Simen
Sep 21 2010
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Simen kjaeraas wrote:

 Lutger <lutger.blijdestijn gmail.com> wrote:
 
 char[] s = ...;
 immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

 I do not understand how that works with sharing. Since immutable data is
 implicitly shared but the data that p refers to is allocated on the TLS,
 how can
 you share this? I always thought that threads do not have access to each
 others
 TLS at all?

Only p itself is in TLS - the pointed-to data is on the heap.

Aha, thanks. I have made a conceptual diagram to help understand this, would you care to take a look and confirm whether this is correct or not? http://picasaweb.google.com/Lutger.blijdestijn/Illustration#5519337139518828386 I hope it explains itself, the edges reflect the types and the nodes the memory storage.
 
 Finally a practical question: when you have a data structure that is too
 complex
 to create in a constructor, want to create it and then make it
 immutable, what
 is the current way to go about this? If it is created by one thread,
 would it be
 ok to type it as __gshared, cast to immutable and then send a message to
 other
 threads? Is __gshared required in this case?

Immutable global state may be instantiated from non-immutable data in module constructors. I believe that is the canonical way.

Sometimes this is not possible. For example if you want to create a data structure from user input once, then use it read-only for the rest of the program.
Sep 21 2010
parent Lutger <lutger.blijdestijn gmail.com> writes:
Simen kjaeraas wrote:

 Lutger <lutger.blijdestijn gmail.com> wrote:
 
 Aha, thanks. I have made a conceptual diagram to help understand this,
 would you
 care to take a look and confirm whether this is correct or not?

 


 I hope it explains itself, the edges reflect the types and the nodes the
 memory
 storage.

That looks very much correct. I'm somewhat confused by the 'static data' part, as it connected only to one thread, and static data is in TLS, as far as I know.

Good, things start falling into places. About the static part, I realized it too late, I meant the readonly data (like .rodata) part of the executable where string literals and such are dumped. The missing connection was for uncluttering.
 Immutable global state may be instantiated from non-immutable data in
 module constructors. I believe that is the canonical way.

Sometimes this is not possible. For example if you want to create a data structure from user input once, then use it read-only for the rest of the program.

Indeed. In that case, would a shared pointer/array to immutable data work for you? In such a case, you are free to have local (non-shared) pointers to the very same data, and thus have no need for locking (which might be needed to dereference the shared pointer).

I would love to do something like that: 1. create a thread that makes a complex data structure from user input 2. said thread dies when it is finished, but gives a message back to the parent consisting of a pointer to the data that is now cast as immutable. Now all mutable aliases are destroyed and the program can enjoy doing parellel work on the data safely, avoiding any locking. It looks like this is possible.
Sep 21 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 21 Sep 2010 05:17:10 -0400, Lutger <lutger.blijdestijn gmail.com>  
wrote:

 I'm still a bit fuzzy on how to create immutable data and when said data  
 is safe
 to share across threads.

 To begin with, there is .idup, string literals and constructors of  
 immutable
 objects. Those can be safely shared, no problem, right?

idup is not safe (along with dup), it is the equivalent of a cast right now. You must still ensure the data has no aliases, or the data idup'd consists of all value types. See this bug report: http://d.puremagic.com/issues/show_bug.cgi?id=3550
 But then, the spec mentions casting to immutable is ok if you do not  
 have any
 mutable aliases left (http://www.digitalmars.com/d/2.0/const3.html):

 char[] s = ...;
 immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

This used to be the only valid way to get immutable data besides idup.
 I do not understand how that works with sharing. Since immutable data is
 implicitly shared but the data that p refers to is allocated on the TLS,  
 how can
 you share this? I always thought that threads do not have access to each  
 others
 TLS at all?

TLS is only for global and static variables. Data on the heap is not TLS. -Steve
Sep 21 2010
parent Lutger <lutger.blijdestijn gmail.com> writes:
Steven Schveighoffer wrote:

 On Tue, 21 Sep 2010 05:17:10 -0400, Lutger <lutger.blijdestijn gmail.com>
 wrote:
 
 I'm still a bit fuzzy on how to create immutable data and when said data
 is safe
 to share across threads.

 To begin with, there is .idup, string literals and constructors of
 immutable
 objects. Those can be safely shared, no problem, right?

idup is not safe (along with dup), it is the equivalent of a cast right now. You must still ensure the data has no aliases, or the data idup'd consists of all value types. See this bug report: http://d.puremagic.com/issues/show_bug.cgi?id=3550

Good to know, I voted for it. There are quite a few bugs related to this topic, looks like I have some more studying to do.
Sep 21 2010
prev sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:

 Aha, thanks. I have made a conceptual diagram to help understand this,  
 would you
 care to take a look and confirm whether this is correct or not?

 http://picasaweb.google.com/Lutger.blijdestijn/Illustration#5519337139518828386

 I hope it explains itself, the edges reflect the types and the nodes the  
 memory
 storage.

That looks very much correct. I'm somewhat confused by the 'static data' part, as it connected only to one thread, and static data is in TLS, as far as I know.
 Immutable global state may be instantiated from non-immutable data in
 module constructors. I believe that is the canonical way.

Sometimes this is not possible. For example if you want to create a data structure from user input once, then use it read-only for the rest of the program.

Indeed. In that case, would a shared pointer/array to immutable data work for you? In such a case, you are free to have local (non-shared) pointers to the very same data, and thus have no need for locking (which might be needed to dereference the shared pointer). -- Simen
Sep 21 2010