www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Threads not independent...

reply Larry Cowan <Larry_member pathlink.com> writes:
| module test;
| import std.thread;
| import std.stdio;
| import locks.countdown;
|
| void main ( char[][] args )
| {
|    CountDownLatch allDone = new CountDownLatch(5);
|    Thread[5] t;
|
|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(
|            delegate int() {
|                printf("  thread %d\n",i);
|                allDone.countDown();
|                return 0;
|            });
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        t[i].start();
|
|    allDone.wait();
|    printf("All done.\n");
|}

prints:

|Starting threads.
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|All done.

How do I best do what I obviously want to?  I can thnk of ways, but they
are all somewhat convoluted and may be susceptible to similar problems.
Do the delegates actually have their own (section of the?) stack?  Do I
have to make one for them on the heap to actually get independent activity,
then have them pick up an index from a locked counter?
Apr 04 2005
next sibling parent Larry Cowan <Larry_member pathlink.com> writes:
Corrected error in code given, bad typing...

In article <d2sdsg$9ca$1 digitaldaemon.com>, Larry Cowan says...
| module test;
| import std.thread;
| import std.stdio;
| import locks.countdown;
|
| void main ( char[][] args )
| {
|    CountDownLatch allDone = new CountDownLatch(5);
|    Thread[5] t;
|
|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(
|            delegate int() {
|                printf("  thread %d\n",i);
|                allDone.countDown();
|                return 0;
|            });
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        t[j].start();
|
|    allDone.wait();
|    printf("All done.\n");
|}

prints:

|Starting threads.
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|All done.

How do I best do what I obviously want to?  I can think of ways, but they
are all somewhat convoluted and may be susceptible to similar problems.
Do the delegates actually have their own (section of the?) stack?  Do I
have to make one for them on the heap to actually get independent activity,
then have them pick up an index from a locked counter?

Apr 04 2005
prev sibling next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 4 Apr 2005 22:08:16 +0000 (UTC), Larry Cowan  
<Larry_member pathlink.com> wrote:
 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

 prints:

 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.

 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?

In the above all the unnamed delegates share the same stack. Containing the same 'i' variable. The dodgy bit, from where I'm sitting is that 'i' only exists for the duration of the for loop, so by the time you 'start' the threads... I must be missing something here? Perhaps the stack is not 'free'd (?) unless no more 'refences'(?) to it exist?
 Do I
 have to make one for them on the heap to actually get independent  
 activity,
 then have them pick up an index from a locked counter?

I would write a class i.e. class Foo { int index; this(int i) { index = i; } .. run() .. } then | for (int i=0; i < 5 ;i++) | t[i] = new Thread(new Foo(i).run); (not sure if the above is legal, but you get the idea?) This should create a delegate that's object pointer points to a Foo class which has it's own 'index' integer, not shared between all the threads. You could instead derive a clas from thread eg. class Foo : Thread { int index; this(int i, ?delegate?) { super(?delegate?); index = i; } ~this() { allDone.countDown(); } // or maybe put this in the 'stop'(?) type method? } "?delegate?" above as I cannot remember the syntax off hand. Regan
Apr 04 2005
next sibling parent reply Larry Cowan <Larry_member pathlink.com> writes:
See below.

In article <opsoqa0osi23k2f5 nrage.netwin.co.nz>, Regan Heath says...
On Mon, 4 Apr 2005 22:08:16 +0000 (UTC), Larry Cowan  
<Larry_member pathlink.com> wrote:
 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

 prints:

 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.

 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?

In the above all the unnamed delegates share the same stack. Containing the same 'i' variable. The dodgy bit, from where I'm sitting is that 'i' only exists for the duration of the for loop, so by the time you 'start' the threads... I must be missing something here? Perhaps the stack is not 'free'd (?) unless no more 'refences'(?) to it exist?

effect would be the same, but I can test this. Anyway, I know what is happening re the main() sction of the stack, but I was asking for an idea about how best to staticize the index for each thread, and I will try your "new"ing it idea, but the ref is still in main()'s stack area and may have the same problem.
 Do I
 have to make one for them on the heap to actually get independent  
 activity,
 then have them pick up an index from a locked counter?


start they have independent stacks without my help - not just as if they were called by main() and got the next stack frame - each getting the same one.
I would write a class i.e.

class Foo {
   int index;
   this(int i) { index = i; }
   .. run() ..
}

then

|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(new Foo(i).run);

(not sure if the above is legal, but you get the idea?)

This should create a delegate that's object pointer points to a Foo class  
which has it's own 'index' integer, not shared between all the threads.

You could instead derive a clas from thread eg.

class Foo : Thread {
   int index;
   this(int i, ?delegate?) { super(?delegate?); index = i; }
   ~this() { allDone.countDown(); } // or maybe put this in the 'stop'(?)  
type method?
}

"?delegate?" above as I cannot remember the syntax off hand.

Regan

I'll report back after some more experimenting... -Larry
Apr 04 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 4 Apr 2005 22:42:21 +0000 (UTC), Larry Cowan  
<Larry_member pathlink.com> wrote:
 In article <opsoqa0osi23k2f5 nrage.netwin.co.nz>, Regan Heath says...
 On Mon, 4 Apr 2005 22:08:16 +0000 (UTC), Larry Cowan
 <Larry_member pathlink.com> wrote:
 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

 prints:

 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.

 How do I best do what I obviously want to?  I can thnk of ways, but  
 they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?

In the above all the unnamed delegates share the same stack. Containing the same 'i' variable. The dodgy bit, from where I'm sitting is that 'i' only exists for the duration of the for loop, so by the time you 'start' the threads... I must be missing something here? Perhaps the stack is not 'free'd (?) unless no more 'refences'(?) to it exist?

the effect would be the same, but I can test this.

Good point. This seems likely to me.
 Anyway, I know what is
 happening re the main() sction of the stack, but I was asking for an idea
 about how best to staticize the index for each thread, and I will try  
 your
 "new"ing it idea, but the ref is still in main()'s stack area and may  
 have
 the same problem.

No, my ideas do not use main()'s stack area. Each class is allocated on the heap, and the reference to the class is not stored except in the delegate passed to the threads constructor. (or in t[] if you extend Thread with your own class - which still means it is not stored in mains stack area - only the t[] reference is). Either way the stack frame of the thread code is not main() it is the stack frame of the function in the class.
 Do I
 have to make one for them on the heap to actually get independent
 activity,
 then have them pick up an index from a locked counter?


start they have independent stacks without my help - not just as if they were called by main() and got the next stack frame - each getting the same one.

IIRC creating an unnamed delegate assigns the current stack frame to the delegates object pointer. So, creating an unnamed delegate in "main()" assigns mains stack frame to the delegate. A class method delegate has it's object pointer assigned to the class instance. The stack frame for a class delegate is the stack frame created when the method is called.
 I would write a class i.e.

 class Foo {
   int index;
   this(int i) { index = i; }
   .. run() ..
 }

 then

 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(new Foo(i).run);

 (not sure if the above is legal, but you get the idea?)

 This should create a delegate that's object pointer points to a Foo  
 class
 which has it's own 'index' integer, not shared between all the threads.

 You could instead derive a clas from thread eg.

 class Foo : Thread {
   int index;
   this(int i, ?delegate?) { super(?delegate?); index = i; }
   ~this() { allDone.countDown(); } // or maybe put this in the 'stop'(?)
 type method?
 }

 "?delegate?" above as I cannot remember the syntax off hand.

 Regan

I'll report back after some more experimenting...

I am interested to know what you find. I have done very little experimentation with threads myself, so my advice may not be entirely acurate. IIRC Russ Lewis is the person I see most often posting about delegates and stack frames, his advice you can trust (I believe). Regan
Apr 04 2005
prev sibling parent reply Larry Cowan <Larry_member pathlink.com> writes:
 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;


 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
+|    for (int i=0; i < 5 ;i++) {


 |        t[i] = new Thread(
 |            delegate int() {
+|                printf("  thread %d\n",ix.n);
 |                allDone.countDown();
 |                return 0;
+|            });


 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[j].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

+++++++ Now prints:

 |Starting threads.
 |  thread 4
 |  thread 4
 |  thread 4
 |  thread 4
 |  thread 4
 |All done.


sitting on the stack in main()'s frame, even though "test" should have stomped on it as I understand D's interspersed allocation method - apparently the internal blocks get their own frame above the main() one, and they still are there, but there is space reserved in main ()'s for the "test" allocation.
 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?


New try with results below: |// dmd test -L locks.lib |module test; |import std.thread; |import std.stdio; |import std.c.time; |import locks.countdown; | ||class Indexed { | | private int n; | private Thread t; | private CountDownLatch* cdlp; | | this ( int n, CountDownLatch* cdlp ) | { | this.n = n; | this.cdlp = cdlp; | this.t = new Thread(delegate int() { | printf(" thread %d\n",n); | cdlp.countDown(); /* (Access Violation */ | return 0; | } | ); | } | void run () { t.start(); } |} | |void main ( char[][] args ) |{ | CountDownLatch allDone = new CountDownLatch(5); | Indexed[5] ixt; | | for (int i=0; i < 5 ;i++) | ixt[i] = new Indexed(i,&allDone); | | printf("Starting threads.\n"); | | for (int j=0; j < 5 ;j++) | ixt[j].run(); | |// allDone.wait(); --> (never ends) | sleep(7); // some how supplies the printed thread # -- why? | printf("All done.\n"); |} New output is: |Starting threads. | thread 7 |Error: Access Violation | thread 7 |Error: Access Violation | thread 7 |Error: Access Violation | thread 7 |Error: Access Violation | thread 7 |Error: Access Violation |All done. This is just weird - I'm abandoning this path. See below:
From Ben Hinkle:

Glad to see someone using the Locks lib. When I was testing things out I ran 
into the same conundrum you did. A simple way (eg not very generic but it 
gets the job done) of solving the problem is to loop over t looking for the 
thread in question:

t[i] = new Thread(
 delegate int() {
  int n;
  for(n=0;n<5;n++) {
    if (Thread.getThis() is t[n]) break;
  }
  printf("  thread %d\n",n);
  allDone.countDown();
  return 0;
});

I am trying to pass a work index into the thread, and this will probably solve that part. It works for me as you coded above. Thanks for all the mintl stuff, especially the concurrent and locks. I don't understand the trouble I had with CountDownLatch - I haven't investigated it yet, but it does work in your method above. I still have some problems in my multithreaded multiple ping, but I think I'm getting independent thread activity now. -Larry
Apr 04 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Try this:

import std.c.windows.windows;
import std.thread;
import std.stdio;

class RThread : Thread
{
	int index;
	
	this(int i)
	{
		super(&run);
		index = i;
	}

	int run()
	{
		writefln("thread %d",index);
		//allDone.countDown();
		return 0;
	}
}

int main(char[][] args)
{
	//CountDownLatch allDone = new CountDownLatch(5);
	RThread[5] threads;
	
	for (int i = 0; i < threads.length; i++)
		threads[i] = new RThread(i);
	
	foreach(RThread t; threads)
		t.start();
	
	//allDone.wait();
	Sleep(10000);
	printf("All done.\n");
	return 0;
}

Note: I dont have the concurrent stuff so I have included it but commented  
it out.

Regan
Apr 04 2005
prev sibling next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Alternate solution using seperate object, not derived from Thread:

import std.c.windows.windows;
import std.thread;
import std.stdio;

class Other
{
	int index;
	
	this(int i)
	{
		index = i;
	}
	
	int run()
	{
		writefln("thread %d",index);
		//allDone.countDown();
		return 0;
	}
}

int main(char[][] args)
{
	//CountDownLatch allDone = new CountDownLatch(5);
	Thread[5] threads;
	Other o;
	
	for (int i = 0; i < threads.length; i++) {
		o = new Other(i);
		threads[i] = new Thread(&o.run);
	}
	
	foreach(Thread t; threads)
		t.start();
	
	//allDone.wait();
	Sleep(10000);
	printf("All done.\n");
	return 0;
}

Regan
Apr 04 2005
prev sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Ooops. One important correction is that the 'allDone' variable actually  
needs to be global, or otherwise accessable from the RThread or Other  
classes. eg.

import ..etc..

//global variable/singleton
CountDownLatch allDone;

//module static ctor, executed when program starts
static this()
{
	allDone = new CountDownLatch(5);
}

class RThread : Thread
{
   ..etc..
}

class Other
{
   ..etc..
}

int main(char[][] args)
{
   ..etc..
}

Regan
Apr 04 2005
prev sibling next sibling parent Larry Cowan <Larry_member pathlink.com> writes:
Corrected typing error in code...

In article <d2sdsg$9ca$1 digitaldaemon.com>, Larry Cowan says...
| module test;
| import std.thread;
| import std.stdio;
| import locks.countdown;
|
| void main ( char[][] args )
| {
|    CountDownLatch allDone = new CountDownLatch(5);
|    Thread[5] t;
|
|    for (int i=0; i < 5 ;i++)
|        t[i] = new Thread(
|            delegate int() {
|                printf("  thread %d\n",i);
|                allDone.countDown();
|                return 0;
|            });
|
|    printf("Starting threads.\n");
|
|    for (int j=0; j < 5 ;j++)
|        t[j].start();
|
|    allDone.wait();
|    printf("All done.\n");
|}

prints:

|Starting threads.
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|  thread 5
|All done.

How do I best do what I obviously want to?  I can thnk of ways, but they
are all somewhat convoluted and may be susceptible to similar problems.
Do the delegates actually have their own (section of the?) stack?  Do I
have to make one for them on the heap to actually get independent activity,
then have them pick up an index from a locked counter?

Apr 04 2005
prev sibling next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
Glad to see someone using the Locks lib. When I was testing things out I ran 
into the same conundrum you did. A simple way (eg not very generic but it 
gets the job done) of solving the problem is to loop over t looking for the 
thread in question:

 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });

t[i] = new Thread( delegate int() { int n; for(n=0;n<5;n++) { if (Thread.getThis() is t[n]) break; } printf(" thread %d\n",n); allDone.countDown(); return 0; });
Apr 04 2005
prev sibling next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
I should add that if you are on Linux and you start seeing seg-v's on 
program exit that is because D doesn't wait for all the threads to finish 
before running gc_term and exiting. So basically you need be very careful 
before you return from your main() to make sure the threads are completely 
done and - not just by waiting for a latch, either, since the threads you 
create still need to execute for a bit after they decrement the latch count. 
Mike Swieton and I had some posts ages ago about proposals to change how D 
exits w.r.t. waiting for child threads but nothing changed. Just thought I'd 
mention it because it can be very confusing trying to debug that behavior. 
See for example 
http://www.digitalmars.com/d/archives/digitalmars/D/bugs/156.html

"Larry Cowan" <Larry_member pathlink.com> wrote in message 
news:d2sdsg$9ca$1 digitaldaemon.com...
| module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}

Apr 04 2005
prev sibling parent "Carlos Santander B." <csantander619 gmail.com> writes:
Larry Cowan wrote:
 | module test;
 | import std.thread;
 | import std.stdio;
 | import locks.countdown;
 |
 | void main ( char[][] args )
 | {
 |    CountDownLatch allDone = new CountDownLatch(5);
 |    Thread[5] t;
 |
 |    for (int i=0; i < 5 ;i++)
 |        t[i] = new Thread(
 |            delegate int() {
 |                printf("  thread %d\n",i);
 |                allDone.countDown();
 |                return 0;
 |            });
 |
 |    printf("Starting threads.\n");
 |
 |    for (int j=0; j < 5 ;j++)
 |        t[i].start();
 |
 |    allDone.wait();
 |    printf("All done.\n");
 |}
 
 prints:
 
 |Starting threads.
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |  thread 5
 |All done.
 
 How do I best do what I obviously want to?  I can thnk of ways, but they
 are all somewhat convoluted and may be susceptible to similar problems.
 Do the delegates actually have their own (section of the?) stack?  Do I
 have to make one for them on the heap to actually get independent activity,
 then have them pick up an index from a locked counter?
 
 
 
 
 

The problem is, you're asking the threads to print the value of "i", but by the time they've started, "i" doesn't have the value it used to have, so that's why they're all printing 5. Extending Thread (as Regan said) is the way I've done things like this. -- Carlos Santander Bernal JP2, you'll always live in our minds
Apr 04 2005