www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating a thread-local duplicate of a globally shared array

reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I have two functions running concurrently and they share data via a
globally shared array.

Generally one thread modifies an array and potentially changes its
length, the other thread reads from it. I have to avoid too many locks
and message passing wouldn't really work since I need fast access to
the array.

I can't use the array directly in the reading thread because the array
could possibly be reallocated by the writing thread (e.g. if it
changes the .length property), while at the same time the reading
thread could just have sent the .ptr value of that array to some API
function. I've had this problem occur and the app would crash due to a
reallocation while the API was reading the array.

Here's the gist of it:

__gshared int[] values;

void foo()
{
    // modify, write to the array, and possibly change the length
}

void bar()
{
    // send the .ptr field of 'values' and its length to an API
function which reads
    // 'values' in sequence and does some visual output.
}

Is it possible to lock 'values', but only once in the bar function
while creating a thread-local copy?

I mean something like this:

void bar()
{
    static int[] localValues;

    localValues = LOCK(values[]);  // temporarily lock values and copy
its contents to a thread-local array
    // values is now unlocked and foo() can modify/write to it again

    APIFun(localValues.ptr, localValues.length);  // safe to pass this
array to the API
}
Jun 30 2011
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 01 Jul 2011 01:14:26 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:

 I have two functions running concurrently and they share data via a
 globally shared array.

 Generally one thread modifies an array and potentially changes its
 length, the other thread reads from it. I have to avoid too many locks
 and message passing wouldn't really work since I need fast access to
 the array.

 I can't use the array directly in the reading thread because the array
 could possibly be reallocated by the writing thread (e.g. if it
 changes the .length property), while at the same time the reading
 thread could just have sent the .ptr value of that array to some API
 function. I've had this problem occur and the app would crash due to a
 reallocation while the API was reading the array.

 Here's the gist of it:

 __gshared int[] values;

 void foo()
 {
     // modify, write to the array, and possibly change the length
 }

It's not quite safe (even with locks) to alter __gshared array lengths from more than one thread. This is due to the assumption that any array not marked shared is thread local. This includes __gshared data, since __gshared is not part of the type. What can happen is the array block information can be cached in the thread local LRU append cache, and another thread could extend the data, thereby changing the block information. When the first thread then tries to use the block information, he gets a stale block info from the runtime, and can make incorrect assumptions. Have you tried changing __gshared to shared? shared should be supported. -Steve
Jul 01 2011
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Thanks for the help. But it appears I've ran into some kind of other bug. ddbg:

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at __aaInX
(0x0041b616) thread(552)

That seems like the hash method for checking keys. I have a static
int[int] hash which I'm not sharing with other threads, I just check
if a key is in there. Damn, this will take some work to make a good
test case..
Jul 01 2011