www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating immutable data and sharing it

reply Lutger <lutger.blijdestijn gmail.com> writes:
I'm still a bit fuzzy on how to create immutable data and when said data is
safe 
to share across threads.

To begin with, there is .idup, string literals and constructors of immutable 
objects. Those can be safely shared, no problem, right?

But then, the spec mentions casting to immutable is ok if you do not have any 
mutable aliases left (http://www.digitalmars.com/d/2.0/const3.html):

char[] s = ...;
immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

I do not understand how that works with sharing. Since immutable data is 
implicitly shared but the data that p refers to is allocated on the TLS, how
can 
you share this? I always thought that threads do not have access to each others 
TLS at all?

Finally a practical question: when you have a data structure that is too
complex 
to create in a constructor, want to create it and then make it immutable, what 
is the current way to go about this? If it is created by one thread, would it
be 
ok to type it as __gshared, cast to immutable and then send a message to other 
threads? Is __gshared required in this case?
Sep 21 2010
next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:


 char[] s = ...;
 immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

 I do not understand how that works with sharing. Since immutable data is
 implicitly shared but the data that p refers to is allocated on the TLS,  
 how can
 you share this? I always thought that threads do not have access to each  
 others
 TLS at all?


Only p itself is in TLS - the pointed-to data is on the heap.



 Finally a practical question: when you have a data structure that is too  
 complex
 to create in a constructor, want to create it and then make it  
 immutable, what
 is the current way to go about this? If it is created by one thread,  
 would it be
 ok to type it as __gshared, cast to immutable and then send a message to  
 other
 threads? Is __gshared required in this case?


Immutable global state may be instantiated from non-immutable data in
module constructors. I believe that is the canonical way.

-- 
Simen
Sep 21 2010
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Simen kjaeraas wrote:


 Lutger <lutger.blijdestijn gmail.com> wrote:
 

 char[] s = ...;
 immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

 I do not understand how that works with sharing. Since immutable data is
 implicitly shared but the data that p refers to is allocated on the TLS,
 how can
 you share this? I always thought that threads do not have access to each
 others
 TLS at all?


 
 Only p itself is in TLS - the pointed-to data is on the heap.


Aha, thanks. I have made a conceptual diagram to help understand this, would
you 
care to take a look and confirm whether this is correct or not?



I hope it explains itself, the edges reflect the types and the nodes the memory 
storage. 
 

 

 Finally a practical question: when you have a data structure that is too
 complex
 to create in a constructor, want to create it and then make it
 immutable, what
 is the current way to go about this? If it is created by one thread,
 would it be
 ok to type it as __gshared, cast to immutable and then send a message to
 other
 threads? Is __gshared required in this case?


 
 Immutable global state may be instantiated from non-immutable data in
 module constructors. I believe that is the canonical way.
 


Sometimes this is not possible. For example if you want to create a data 
structure from user input once, then use it read-only for the rest of the 
program.
Sep 21 2010
parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Lutger <lutger.blijdestijn gmail.com> wrote:


 Aha, thanks. I have made a conceptual diagram to help understand this,  
 would you
 care to take a look and confirm whether this is correct or not?



 I hope it explains itself, the edges reflect the types and the nodes the  
 memory
 storage.


That looks very much correct. I'm somewhat confused by the 'static data'
part, as it connected only to one thread, and static data is in TLS, as far
as I know.



 Immutable global state may be instantiated from non-immutable data in
 module constructors. I believe that is the canonical way.


 Sometimes this is not possible. For example if you want to create a data
 structure from user input once, then use it read-only for the rest of the
 program.


Indeed. In that case, would a shared pointer/array to immutable data work  
for
you? In such a case, you are free to have local (non-shared) pointers to  
the
very same data, and thus have no need for locking (which might be needed to
dereference the shared pointer).


-- 
Simen
Sep 21 2010
parent Lutger <lutger.blijdestijn gmail.com> writes:
Simen kjaeraas wrote:


 Lutger <lutger.blijdestijn gmail.com> wrote:
 

 Aha, thanks. I have made a conceptual diagram to help understand this,
 would you
 care to take a look and confirm whether this is correct or not?

 






 I hope it explains itself, the edges reflect the types and the nodes the
 memory
 storage.


 
 That looks very much correct. I'm somewhat confused by the 'static data'
 part, as it connected only to one thread, and static data is in TLS, as far
 as I know.


Good, things start falling into places. About the static part, I realized it
too 
late, I meant the readonly data (like .rodata) part of the executable where 
string literals and such are dumped. The missing connection was for 
uncluttering.
 

 Immutable global state may be instantiated from non-immutable data in
 module constructors. I believe that is the canonical way.


 Sometimes this is not possible. For example if you want to create a data
 structure from user input once, then use it read-only for the rest of the
 program.


 
 Indeed. In that case, would a shared pointer/array to immutable data work
 for
 you? In such a case, you are free to have local (non-shared) pointers to
 the
 very same data, and thus have no need for locking (which might be needed to
 dereference the shared pointer).


I would love to do something like that:

1. create a thread that makes a complex data structure from user input
2. said thread dies when it is finished, but gives a message back to the parent 
consisting of a pointer to the data that is now cast as immutable.

Now all mutable aliases are destroyed and the program can enjoy doing parellel 
work on the data safely, avoiding any locking. It looks like this is possible.
Sep 21 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 21 Sep 2010 05:17:10 -0400, Lutger <lutger.blijdestijn gmail.com>  
wrote:


 I'm still a bit fuzzy on how to create immutable data and when said data  
 is safe
 to share across threads.

 To begin with, there is .idup, string literals and constructors of  
 immutable
 objects. Those can be safely shared, no problem, right?


idup is not safe (along with dup), it is the equivalent of a cast right  
now.  You must still ensure the data has no aliases, or the data idup'd  
consists of all value types.

See this bug report: http://d.puremagic.com/issues/show_bug.cgi?id=3550


 But then, the spec mentions casting to immutable is ok if you do not  
 have any
 mutable aliases left (http://www.digitalmars.com/d/2.0/const3.html):

 char[] s = ...;
 immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference


This used to be the only valid way to get immutable data besides idup.


 I do not understand how that works with sharing. Since immutable data is
 implicitly shared but the data that p refers to is allocated on the TLS,  
 how can
 you share this? I always thought that threads do not have access to each  
 others
 TLS at all?


TLS is only for global and static variables.  Data on the heap is not TLS.

-Steve
Sep 21 2010
parent Lutger <lutger.blijdestijn gmail.com> writes:
Steven Schveighoffer wrote:


 On Tue, 21 Sep 2010 05:17:10 -0400, Lutger <lutger.blijdestijn gmail.com>
 wrote:
 

 I'm still a bit fuzzy on how to create immutable data and when said data
 is safe
 to share across threads.

 To begin with, there is .idup, string literals and constructors of
 immutable
 objects. Those can be safely shared, no problem, right?


 
 idup is not safe (along with dup), it is the equivalent of a cast right
 now.  You must still ensure the data has no aliases, or the data idup'd
 consists of all value types.
 
 See this bug report: http://d.puremagic.com/issues/show_bug.cgi?id=3550


Good to know, I voted for it. There are quite a few bugs related to this topic, 
looks like I have some more studying to do.
Sep 21 2010