www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - custom per-thread stuff

reply B.G. <B.G._member pathlink.com> writes:
Generally I'm looking for a 'nice' and portable way to be able to associate some
arbitrary data with a particular thread. And, what's not less important to have
a hook to be called when a thread ceases to exist, to throw it's associated
information to the GC !

For example, I have a program with lots of string work... I thought of
allocating a per-thread more-or-less persistant stringbuffer to minimize
memory [de-]allocation.

To be general there were many situations when I had headaches how to keep track
of some THINGS commonly used throughout lots of different code.
I thought of some options outlined below, but none of them seemes satisfactory
to me.

- move them along as function parameters or class members, (which I actually
hated, for I hate redundancy)
- make them global, leading to a less flexible solution 
(which I hate even more :-)
- have a global pool of reusable items and lots of mutex stuff to
make everything thread safe
(I don't like this too, because it smells like a bottleneck)

I thought, If there were a mechanism, clean, portable and efficient, to work
with per-thread data, this were a nice option in SOME situations.

Can someone suggest anything?
Apr 29 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
What about this?

class MyTread: Tread
{
   //.... your per-thread data....
    override void run()
   {
      // ...usefull thread code...
   }
}

Am I missing something?

Andrew.



"B.G." <B.G._member pathlink.com> wrote in message 
news:d4tapi$p0n$1 digitaldaemon.com...

 Generally I'm looking for a 'nice' and portable way to be able to 
 associate some
 arbitrary data with a particular thread. And, what's not less important to 
 have
 a hook to be called when a thread ceases to exist, to throw it's 
 associated
 information to the GC !

 For example, I have a program with lots of string work... I thought of
 allocating a per-thread more-or-less persistant stringbuffer to minimize
 memory [de-]allocation.

 To be general there were many situations when I had headaches how to keep 
 track
 of some THINGS commonly used throughout lots of different code.
 I thought of some options outlined below, but none of them seemes 
 satisfactory
 to me.

 - move them along as function parameters or class members, (which I 
 actually
 hated, for I hate redundancy)
 - make them global, leading to a less flexible solution
 (which I hate even more :-)
 - have a global pool of reusable items and lots of mutex stuff to
 make everything thread safe
 (I don't like this too, because it smells like a bottleneck)

 I thought, If there were a mechanism, clean, portable and efficient, to 
 work
 with per-thread data, this were a nice option in SOME situations.

 Can someone suggest anything?
Apr 29 2005
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
Doesn't help you much if you want to write a thread-agnostic
library.



"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:d4tm9a$14pq$1 digitaldaemon.com...

 What about this?

 class MyTread: Tread
 {
   //.... your per-thread data....
    override void run()
   {
      // ...usefull thread code...
   }
 }

 Am I missing something?

 Andrew.



 "B.G." <B.G._member pathlink.com> wrote in message
 news:d4tapi$p0n$1 digitaldaemon.com...

 Generally I'm looking for a 'nice' and portable way to be able to
 associate some
 arbitrary data with a particular thread. And, what's not less
 important to have
 a hook to be called when a thread ceases to exist, to throw it's
 associated
 information to the GC !

 For example, I have a program with lots of string work... I
 thought of
 allocating a per-thread more-or-less persistant stringbuffer to
 minimize
 memory [de-]allocation.

 To be general there were many situations when I had headaches how
 to keep track
 of some THINGS commonly used throughout lots of different code.
 I thought of some options outlined below, but none of them seemes
 satisfactory
 to me.

 - move them along as function parameters or class members, (which
 I actually
 hated, for I hate redundancy)
 - make them global, leading to a less flexible solution
 (which I hate even more :-)
 - have a global pool of reusable items and lots of mutex stuff to
 make everything thread safe
 (I don't like this too, because it smells like a bottleneck)

 I thought, If there were a mechanism, clean, portable and
 efficient, to work
 with per-thread data, this were a nice option in SOME situations.

 Can someone suggest anything?
Apr 29 2005
next sibling parent reply Sean Kelly <sean f4.ca> writes:
But it's a useful option to have.  And since D is garbage collected, it doesn't
have the dangers C++ has with this same approach.  Personally, I use this method
pretty frequently, as it's quite convenient for "job" objects.


Sean

In article <d4v880$2gv0$1 digitaldaemon.com>, Matthew says...

Doesn't help you much if you want to write a thread-agnostic
library.

"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:d4tm9a$14pq$1 digitaldaemon.com...

 What about this?

 class MyTread: Tread
 {
   //.... your per-thread data....
    override void run()
   {
      // ...usefull thread code...
   }
 }

 Am I missing something?

 Andrew.



 "B.G." <B.G._member pathlink.com> wrote in message
 news:d4tapi$p0n$1 digitaldaemon.com...

 Generally I'm looking for a 'nice' and portable way to be able to
 associate some
 arbitrary data with a particular thread. And, what's not less
 important to have
 a hook to be called when a thread ceases to exist, to throw it's
 associated
 information to the GC !

 For example, I have a program with lots of string work... I
 thought of
 allocating a per-thread more-or-less persistant stringbuffer to
 minimize
 memory [de-]allocation.

 To be general there were many situations when I had headaches how
 to keep track
 of some THINGS commonly used throughout lots of different code.
 I thought of some options outlined below, but none of them seemes
 satisfactory
 to me.

 - move them along as function parameters or class members, (which
 I actually
 hated, for I hate redundancy)
 - make them global, leading to a less flexible solution
 (which I hate even more :-)
 - have a global pool of reusable items and lots of mutex stuff to
 make everything thread safe
 (I don't like this too, because it smells like a bottleneck)

 I thought, If there were a mechanism, clean, portable and
 efficient, to work
 with per-thread data, this were a nice option in SOME situations.

 Can someone suggest anything?
Apr 30 2005
parent reply "Matthew" <admin stlsoft.dot.dot.dot.dot.org> writes:
"Sean Kelly" <sean f4.ca> wrote in message 
news:d51c0l$1kh0$1 digitaldaemon.com...

 But it's a useful option to have.  And since D is garbage 
 collected, it doesn't
 have the dangers C++ has with this same approach.  Personally, I 
 use this method
 pretty frequently, as it's quite convenient for "job" objects.


Not sure what you mean by 'option'? Do you mean it's a useful 
technique? If so, I agree, but it still does not represent a viable 
broad solution.

It represents an 'application-level' approach to threading, which is 
perfectly fine, and often the best solution, for many 
application-specific threading issues. But since it necessarily 
binds the code that needs/uses the per-thread opportunity, it 
cannot, by definition, be a general approach. (The only way that 
could be achieved would be by having the Thread class maintain a TSS 
dictionary, which would be at once type-unsafe and incredibly 
ineffecient.)

So the need to true TSS remains. Maybe this'll just have to be on an 
ad-hoc basis up to D 1.2? :-)


 Sean

 In article <d4v880$2gv0$1 digitaldaemon.com>, Matthew says...

Doesn't help you much if you want to write a thread-agnostic
library.

"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:d4tm9a$14pq$1 digitaldaemon.com...

 What about this?

 class MyTread: Tread
 {
   //.... your per-thread data....
    override void run()
   {
      // ...usefull thread code...
   }
 }

 Am I missing something?

 Andrew.



 "B.G." <B.G._member pathlink.com> wrote in message
 news:d4tapi$p0n$1 digitaldaemon.com...

 Generally I'm looking for a 'nice' and portable way to be able 
 to
 associate some
 arbitrary data with a particular thread. And, what's not less
 important to have
 a hook to be called when a thread ceases to exist, to throw 
 it's
 associated
 information to the GC !

 For example, I have a program with lots of string work... I
 thought of
 allocating a per-thread more-or-less persistant stringbuffer to
 minimize
 memory [de-]allocation.

 To be general there were many situations when I had headaches 
 how
 to keep track
 of some THINGS commonly used throughout lots of different code.
 I thought of some options outlined below, but none of them 
 seemes
 satisfactory
 to me.

 - move them along as function parameters or class members, 
 (which
 I actually
 hated, for I hate redundancy)
 - make them global, leading to a less flexible solution
 (which I hate even more :-)
 - have a global pool of reusable items and lots of mutex stuff 
 to
 make everything thread safe
 (I don't like this too, because it smells like a bottleneck)

 I thought, If there were a mechanism, clean, portable and
 efficient, to work
 with per-thread data, this were a nice option in SOME 
 situations.

 Can someone suggest anything?
Apr 30 2005
parent Sean Kelly <sean f4.ca> writes:
In article <d51e8j$1mdu$1 digitaldaemon.com>, Matthew says...

"Sean Kelly" <sean f4.ca> wrote in message 
news:d51c0l$1kh0$1 digitaldaemon.com...

 But it's a useful option to have.  And since D is garbage 
 collected, it doesn't
 have the dangers C++ has with this same approach.  Personally, I 
 use this method
 pretty frequently, as it's quite convenient for "job" objects.


Not sure what you mean by 'option'? Do you mean it's a useful 
technique? If so, I agree, but it still does not represent a viable 
broad solution.


std.Thread class currently allows two methods to define thread procedures:

class MyThread : Thread
{
override int run() { // do stuff }
}

or

Thread t = new Thread( &threadFunc );

I said 'option' both because this is already possible with Phobos and because I
think it should continue to be possible, even if it isn't the advertised method.
But perhaps I should have said 'technique' instead :)


It represents an 'application-level' approach to threading, which is 
perfectly fine, and often the best solution, for many 
application-specific threading issues. But since it necessarily 
binds the code that needs/uses the per-thread opportunity, it 
cannot, by definition, be a general approach.


I agree.  Though in some respects I consider this to be a good thing, as the
inheritance method shows rather explicitly that the code was inteded to be
multithread-safe.


So the need to true TSS remains. Maybe this'll just have to be on an 
ad-hoc basis up to D 1.2? :-)


By TSS do you mean thread-specific-storage?  This might be possible for 1.0, as
it's really a library issue, though this may require some work to have it play
nice with the GC.  I'll have to think about it a bit.


Sean
May 02 2005
prev sibling parent reply B.G. <B.G._member pathlink.com> writes:
What about threads created by a D library or even worse, a C library?, Is it
possible to 'bless' threads created by C libraries like UPnP SDK so that D can
treat them like 'normal' D threads (for correct GC, etc.)?


Doesn't help you much if you want to write a thread-agnostic
library.


Isn't there always at least one thread in every D application?
I mean, you can always say Thread.getThis(), right?


"Andrew Fedoniouk" <news terrainformatica.com> wrote in message
news:d4tm9a$14pq$1 digitaldaemon.com...

 What about this?

 class MyTread: Tread
 {
   //.... your per-thread data....
    override void run()
   {
      // ...usefull thread code...
   }
 }

 Am I missing something?

 Andrew.



 "B.G." <B.G._member pathlink.com> wrote in message
 news:d4tapi$p0n$1 digitaldaemon.com...

 Generally I'm looking for a 'nice' and portable way to be able to
 associate some
 arbitrary data with a particular thread. And, what's not less
 important to have
 a hook to be called when a thread ceases to exist, to throw it's
 associated
 information to the GC !

 For example, I have a program with lots of string work... I
 thought of
 allocating a per-thread more-or-less persistant stringbuffer to
 minimize
 memory [de-]allocation.

 To be general there were many situations when I had headaches how
 to keep track
 of some THINGS commonly used throughout lots of different code.
 I thought of some options outlined below, but none of them seemes
 satisfactory
 to me.

 - move them along as function parameters or class members, (which
 I actually
 hated, for I hate redundancy)
 - make them global, leading to a less flexible solution
 (which I hate even more :-)
 - have a global pool of reusable items and lots of mutex stuff to
 make everything thread safe
 (I don't like this too, because it smells like a bottleneck)

 I thought, If there were a mechanism, clean, portable and
 efficient, to work
 with per-thread data, this were a nice option in SOME situations.

 Can someone suggest anything?
May 02 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <d56bhb$8ed$1 digitaldaemon.com>, B.G. says...

What about threads created by a D library or even worse, a C library?, Is it
possible to 'bless' threads created by C libraries like UPnP SDK so that D can
treat them like 'normal' D threads (for correct GC, etc.)?


Not currently.  The GC needs to be able to pause all running threads and to get
stack/register pointers for those paused threads so it can scan them.
Currently, this works by calling Thread.pauseAll() and then iterating across the
collection returned by Thread.getAll().  You could try to fake this by deriving
from std.Thread and implementing Thread.pause() and a few other functions for
your new thread API.  This *might* work, though some code that IMO should be in
the Thread class (like the Windows routines that get a CONTEXT structure) are in
the GC code instead, and this may or may not cause problems in practice.  I
rewrote this code in Ares, but I do consider it a requirement that all managed
threads must be derived from std.Thread.

This issue is somewhat related to how to handle DLL code, as the GCs need to
cooperate at the very least.  This may lead to some means of "registering"
outside threads, but they will still likely need to be std.Thread objects for
ease of use.


Sean
May 02 2005
parent reply B.G. <B.G._member pathlink.com> writes:
In article <d56e09$2ff$1 digitaldaemon.com>, Sean Kelly says...

In article <d56bhb$8ed$1 digitaldaemon.com>, B.G. says...

What about threads created by a D library or even worse, a C library?, Is it
possible to 'bless' threads created by C libraries like UPnP SDK so that D can
treat them like 'normal' D threads (for correct GC, etc.)?


Not currently.  The GC needs to be able to pause all running threads and to get
stack/register pointers for those paused threads so it can scan them.
Currently, this works by calling Thread.pauseAll() and then iterating across the
collection returned by Thread.getAll().  You could try to fake this by deriving
from std.Thread and implementing Thread.pause() and a few other functions for
your new thread API.  This *might* work, though some code that IMO should be in
the Thread class (like the Windows routines that get a CONTEXT structure) are in
the GC code instead, and this may or may not cause problems in practice.  I
rewrote this code in Ares, but I do consider it a requirement that all managed
threads must be derived from std.Thread.

This issue is somewhat related to how to handle DLL code, as the GCs need to
cooperate at the very least.  This may lead to some means of "registering"
outside threads, but they will still likely need to be std.Thread objects for
ease of use.


Sean


Hmmm, I'm just wondering, what could be a solution for cases like this:

Given an SDK, written in C/C++, which produces
it's own threads for whatever purposes. SDK assumes that the application built
on top registers some callbacks to respond to some events.

Now, what happens if we want to implement those callbacsk in D?

They may get called from any thread, and If it's not the main thread there's
apparently no way to teach D handle those threads correctly.

I have an application written in C++ using a C SDK with the above architecture.
And I want to port it to D for the sake of practice and trying D it the
'real world'.
(...and because I got sick of smart pointers and weird syntax, long live D! ;-)

Any suggestions?
May 02 2005
parent Sean Kelly <sean f4.ca> writes:
In article <d56tdo$n7l$1 digitaldaemon.com>, B.G. says...

Hmmm, I'm just wondering, what could be a solution for cases like this:

Given an SDK, written in C/C++, which produces
it's own threads for whatever purposes. SDK assumes that the application built
on top registers some callbacks to respond to some events.

Now, what happens if we want to implement those callbacsk in D?

They may get called from any thread, and If it's not the main thread there's
apparently no way to teach D handle those threads correctly.

I have an application written in C++ using a C SDK with the above architecture.
And I want to port it to D for the sake of practice and trying D it the
'real world'.
(...and because I got sick of smart pointers and weird syntax, long live D! ;-)


So you want to be able to pass pointers to GC memory to this C library and keep
everything happy?  You'll have to modify std.Thread, but how you go about it is
up to you.  Ares (an alternate std lib for D) defines three extern (C) functions
that are used by the GC for this purpose: suspendAllThreads, scanAllThreads, and
resumeAllThreads.  Currently, these just forward the work to the appropriate
static Thread methods, but it would be easy enough to add some code to support
regitration of your custom thread API.  Only catch here of course is that Ares
is nowhere near as complete as Phobos.  For a standard D distribution you'll
need to modify the Thread class code to allow for pretty much the same thing.
It might be a little hacky, but it should work just fine.


Sean
May 03 2005