www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Threads, shread and TLS

reply Adam Conner-Sax <adam_conner_sax yahoo.com> writes:
So, I thought I sort of understood "shared" and now I think I don't.

If I have a class:

class foo {
  int x;
  static int y;
  shared static int z;

}

So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?

If that's true (and I realize it might not be), and I want to initialize these
variables in constructors, how does that work?

I think

class foo {
...(as before)

this() { x = 2; } // ok

static this() { y = 3; } // is this called once per thread?

shared static this() { z = 3;} // also, okay, called before main
}

but I don't understand what happens with threads and the "static this"
constructor.  How/when are the thread-local copies constructed?  How do you
initialize/construct the thread-local static data?

Thanks!

Adam
Jan 07 2011
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 07 Jan 2011 09:24:18 -0500, Adam Conner-Sax  
<adam_conner_sax yahoo.com> wrote:

 So, I thought I sort of understood "shared" and now I think I don't.

 If I have a class:

 class foo {
   int x;
   static int y;
   shared static int z;

 }

 So x is one instance per class and is thread-local?
 y is one instance per thread?
 z is one instance per application, i.e., global?

Yes to all
 If that's true (and I realize it might not be), and I want to initialize  
 these
 variables in constructors, how does that work?

 I think

 class foo {
 ...(as before)

 this() { x = 2; } // ok

 static this() { y = 3; } // is this called once per thread?

 shared static this() { z = 3;} // also, okay, called before main
 }

 but I don't understand what happens with threads and the "static this"
 constructor.  How/when are the thread-local copies constructed?  How do  
 you
 initialize/construct the thread-local static data?

static this() is run upon thread creation (and once at the beginning of the program for the main thread), and static ~this() is run when a thread is destroyed. All static this() and shared static this() functions are assembled by the runtime into an array of constructors, built in the correct order based on import dependencies. So each time a thread starts, it simply goes through an array and calls each function. -Steve
Jan 07 2011
prev sibling parent reply Adam Conner-Sax <adam_conner_sax yahoo.com> writes:
== Quote from Adam Conner-Sax (adam_conner_sax yahoo.com)'s article
 So, I thought I sort of understood "shared" and now I think I don't.
 If I have a class:
 class foo {
   int x;
   static int y;
   shared static int z;
 }
 So x is one instance per class and is thread-local?
 y is one instance per thread?
 z is one instance per application, i.e., global?
 If that's true (and I realize it might not be), and I want to initialize these
 variables in constructors, how does that work?
 I think
 class foo {
 ...(as before)
 this() { x = 2; } // ok
 static this() { y = 3; } // is this called once per thread?
 shared static this() { z = 3;} // also, okay, called before main
 }
 but I don't understand what happens with threads and the "static this"
 constructor.  How/when are the thread-local copies constructed?  How do you
 initialize/construct the thread-local static data?
 Thanks!
 Adam

Nevermind. Answered myself with the following: import core.thread; import std.c.stdio; class foo { int a; shared int b; static int x; shared static int y; shared static int[] arr1; shared static int[] arr2; this() { a = 1; b=10; } static this() { x=100; arr1 ~= x; } shared static this() { y=1000; arr2 ~= y; } static void A() { x++; y++; } void B() { a++; b++; } void report() { printf("a=%i; b=%i; x=%i; y=%i; arr1.length=%i; arr2.length=%i\n",a,b,x,y,arr1.length, arr2.length); } } void main() { auto f = new foo(); void call_foo_functions() { f.A(); f.B(); f.report(); } auto tg = new ThreadGroup(); foreach (k; 0..3) { auto t = new Thread(&call_foo_functions); tg.add(t); t.start(); } tg.joinAll(); printf("back in main: "); f.report(); } which output: a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1 a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1 a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1 back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1 which all makes sense (to me) except a. Why is a acting global? Is it since it isn't static so it belongs to the class and there is only one copy of the class? Then what makes a and b different? Also, though maybe it's obvious to everybody else, I think the docs should explain someplace that "static this()" gets called once *per thread*. Maybe I just missed it. Adam
Jan 07 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 07 Jan 2011 09:55:33 -0500, Adam Conner-Sax  
<adam_conner_sax yahoo.com> wrote:

 == Quote from Adam Conner-Sax (adam_conner_sax yahoo.com)'s article
 So, I thought I sort of understood "shared" and now I think I don't.
 If I have a class:
 class foo {
   int x;
   static int y;
   shared static int z;
 }
 So x is one instance per class and is thread-local?
 y is one instance per thread?
 z is one instance per application, i.e., global?
 If that's true (and I realize it might not be), and I want to  
 initialize these
 variables in constructors, how does that work?
 I think
 class foo {
 ...(as before)
 this() { x = 2; } // ok
 static this() { y = 3; } // is this called once per thread?
 shared static this() { z = 3;} // also, okay, called before main
 }
 but I don't understand what happens with threads and the "static this"
 constructor.  How/when are the thread-local copies constructed?  How do  
 you
 initialize/construct the thread-local static data?
 Thanks!
 Adam

Nevermind. Answered myself with the following: import core.thread; import std.c.stdio; class foo { int a; shared int b; static int x; shared static int y; shared static int[] arr1; shared static int[] arr2; this() { a = 1; b=10; } static this() { x=100; arr1 ~= x; } shared static this() { y=1000; arr2 ~= y; } static void A() { x++; y++; } void B() { a++; b++; } void report() { printf("a=%i; b=%i; x=%i; y=%i; arr1.length=%i; arr2.length=%i\n",a,b,x,y,arr1.length, arr2.length); } } void main() { auto f = new foo(); void call_foo_functions() { f.A(); f.B(); f.report(); } auto tg = new ThreadGroup(); foreach (k; 0..3) { auto t = new Thread(&call_foo_functions); tg.add(t); t.start(); } tg.joinAll(); printf("back in main: "); f.report(); } which output: a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1 a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1 a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1 back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1 which all makes sense (to me) except a. Why is a acting global? Is it since it isn't static so it belongs to the class and there is only one copy of the class? Then what makes a and b different?

OK, so here is what is happening :) call_foo_functions is a delegate, which means it has a frame pointer to the main function. So all three threads' f is the *same* f (the one defined in main()). I would suggest that you move call_foo_functions outside main, and instantiate an additional f *inside* the function. This would be more correct. Incidentally, it appears that this allows untagged sharing (i.e. sharing data that isn't tagged with shared) I wonder if this issue has been reported before? Sean?
 Also, though maybe it's obvious to everybody else, I think the docs  
 should explain
 someplace that "static this()" gets called once *per thread*.  Maybe I  
 just missed it.

Yes, the documentation is out of date. Could you file a bugzilla report on this? -Steve
Jan 07 2011
parent Adam Conner-Sax <adam_conner_sax yahoo.com> writes:
Thanks!

It's not really about correctness as much as trying to understand how these
different storage classes work.  I understand that there is only one foo object.
I wanted to see which parts are thread-local and which are shared and how the
constructors work.

I'm working (way over my head) on a more complex threading issue and I realized
that I didn't quite understand the storage classes and constructors. Now I get
it
a bit more.

By "untagged sharing" do you mean the "a" variable which is shared in the sense
that all threads see the same copy but does not have storage class "shared"? 
Yes,
that confuses me too.  Should it be an error?

I have never dealt with bugzilla but I will try to figure out how to do what you
ask :)

Thanks again.

Adam



== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article
 On Fri, 07 Jan 2011 09:55:33 -0500, Adam Conner-Sax
 <adam_conner_sax yahoo.com> wrote:
 == Quote from Adam Conner-Sax (adam_conner_sax yahoo.com)'s article
 So, I thought I sort of understood "shared" and now I think I don't.
 If I have a class:
 class foo {
   int x;
   static int y;
   shared static int z;
 }
 So x is one instance per class and is thread-local?
 y is one instance per thread?
 z is one instance per application, i.e., global?
 If that's true (and I realize it might not be), and I want to
 initialize these
 variables in constructors, how does that work?
 I think
 class foo {
 ...(as before)
 this() { x = 2; } // ok
 static this() { y = 3; } // is this called once per thread?
 shared static this() { z = 3;} // also, okay, called before main
 }
 but I don't understand what happens with threads and the "static this"
 constructor.  How/when are the thread-local copies constructed?  How do
 you
 initialize/construct the thread-local static data?
 Thanks!
 Adam

Nevermind. Answered myself with the following: import core.thread; import std.c.stdio; class foo { int a; shared int b; static int x; shared static int y; shared static int[] arr1; shared static int[] arr2; this() { a = 1; b=10; } static this() { x=100; arr1 ~= x; } shared static this() { y=1000; arr2 ~= y; } static void A() { x++; y++; } void B() { a++; b++; } void report() { printf("a=%i; b=%i; x=%i; y=%i; arr1.length=%i; arr2.length=%i\n",a,b,x,y,arr1.length, arr2.length); } } void main() { auto f = new foo(); void call_foo_functions() { f.A(); f.B(); f.report(); } auto tg = new ThreadGroup(); foreach (k; 0..3) { auto t = new Thread(&call_foo_functions); tg.add(t); t.start(); } tg.joinAll(); printf("back in main: "); f.report(); } which output: a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1 a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1 a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1 back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1 which all makes sense (to me) except a. Why is a acting global? Is it since it isn't static so it belongs to the class and there is only one copy of the class? Then what makes a and b different?

call_foo_functions is a delegate, which means it has a frame pointer to the main function. So all three threads' f is the *same* f (the one defined in main()). I would suggest that you move call_foo_functions outside main, and instantiate an additional f *inside* the function. This would be more correct. Incidentally, it appears that this allows untagged sharing (i.e. sharing data that isn't tagged with shared) I wonder if this issue has been reported before? Sean?
 Also, though maybe it's obvious to everybody else, I think the docs
 should explain
 someplace that "static this()" gets called once *per thread*.  Maybe I
 just missed it.

on this? -Steve

Jan 07 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 07 Jan 2011 11:30:17 -0500, Adam Conner-Sax  
<adam_conner_sax yahoo.com> wrote:

 Thanks!

 It's not really about correctness as much as trying to understand how  
 these
 different storage classes work.  I understand that there is only one foo  
 object.
 I wanted to see which parts are thread-local and which are shared and  
 how the
 constructors work.

A class instance can be shared or unshared. Either way, changing the data on the same instance updates the same instance, there is not a copy of the whole world in each thread, just a copy of the thread local storage block. So you are sort of conflating 'per instance' with 'per thread.' int x -- per instance (shared or not) static int x -- per thread (in the TLS block) shared static int x -- per process, not in any instance.
 I'm working (way over my head) on a more complex threading issue and I  
 realized
 that I didn't quite understand the storage classes and constructors. Now  
 I get it
 a bit more.

 By "untagged sharing" do you mean the "a" variable which is shared in  
 the sense
 that all threads see the same copy but does not have storage class  
 "shared"?  Yes,
 that confuses me too.  Should it be an error?

Yes, if you have a piece of data that shared and not marked with __gshared or shared, then we have a problem. The problem is that a lot of code assumes that situation cannot happen without casts (you can always cast and override the type system), so you can make assumptions based on that. For example, a lot of C++/java code is written *just in case* an object is shared. With D, the hope is that you are *guaranteed* that it is shared or not, so you can optimize your code that way (i.e. use a lock or not). As long as the possibility exists that code not marked as shared can be easily shared without a cast, we cannot make those assumptions. I think it should be filed as a bug, but I'm not sure if someone's already reported it.
 I have never dealt with bugzilla but I will try to figure out how to do  
 what you
 ask :)

Go to http://d.puremagic.com/issues/enter_bug.cgi You will have to create a user before filing a bug, but bugzilla is relatively straightforward to use. -Steve
Jan 07 2011