www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - What sync object should i use?

reply "Heinz" <thor587 gmail.com> writes:
Hi,

I'm looking for an object in "core.sync" whose internal counter 
can be 0 or 1 (signaled/not signaled), just like Event Objects in 
Win32 
(http://msdn.microsoft.com/en-us/library/windows/desktop/ms682396
28v=vs.85%29.aspx). 
The object must be waitable and able to notify waiters. A 
semaphore is the most similar thing but its internal counter can 
range from 0 to x. I can perfectly create and use an event under 
Win32 for my needs but i do not know their counterparts in 
FreeVSD, Linux and OSX; that's why i'm trying to use an already 
implemented object from the runtime.

This is what i'm trying to do:

void MyFunc()
{

}
May 13 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 13 May 2013 15:41:47 -0400, Heinz <thor587 gmail.com> wrote:


 Hi,

 I'm looking for an object in "core.sync" whose internal counter can be 0  
 or 1 (signaled/not signaled), just like Event Objects in Win32  
 (http://msdn.microsoft.com/en-us/library/windows/desktop/ms682396
28v=vs.85%29.aspx).  
 The object must be waitable and able to notify waiters. A semaphore is  
 the most similar thing but its internal counter can range from 0 to x. I  
 can perfectly create and use an event under Win32 for my needs but i do  
 not know their counterparts in FreeVSD, Linux and OSX; that's why i'm  
 trying to use an already implemented object from the runtime.

 This is what i'm trying to do:

 void MyFunc()
 {

 }


core.sync.condition and core.sync.mutex

I'd point at the docs, but they are poor.

Search online for docs on how mutexes and conditions work.  It's very  
similar to Windows events.

-Steve
May 13 2013
parent "Heinz" <thor587 gmail.com> writes:
On Monday, 13 May 2013 at 19:49:51 UTC, Steven Schveighoffer 
wrote:

 On Mon, 13 May 2013 15:41:47 -0400, Heinz <thor587 gmail.com> 
 wrote:


 Hi,

 I'm looking for an object in "core.sync" whose internal 
 counter can be 0 or 1 (signaled/not signaled), just like Event 
 Objects in Win32 
 (http://msdn.microsoft.com/en-us/library/windows/desktop/ms682396
28v=vs.85%29.aspx). 
 The object must be waitable and able to notify waiters. A 
 semaphore is the most similar thing but its internal counter 
 can range from 0 to x. I can perfectly create and use an event 
 under Win32 for my needs but i do not know their counterparts 
 in FreeVSD, Linux and OSX; that's why i'm trying to use an 
 already implemented object from the runtime.

 This is what i'm trying to do:

 void MyFunc()
 {

 }


 core.sync.condition and core.sync.mutex

 I'd point at the docs, but they are poor.

 Search online for docs on how mutexes and conditions work.  
 It's very similar to Windows events.

 -Steve


Thanks Steve for your quick reply. You answered me before i fixed 
my post.
I'll take a look at what you say and post again here in case i 
have other questions.
May 13 2013
prev sibling next sibling parent "Heinz" <thor587 gmail.com> writes:
On Monday, 13 May 2013 at 19:41:48 UTC, Heinz wrote:

 Hi,

 I'm looking for an object in "core.sync" whose internal counter 
 can be 0 or 1 (signaled/not signaled), just like Event Objects 
 in Win32 
 (http://msdn.microsoft.com/en-us/library/windows/desktop/ms682396
28v=vs.85%29.aspx). 
 The object must be waitable and able to notify waiters. A 
 semaphore is the most similar thing but its internal counter 
 can range from 0 to x. I can perfectly create and use an event 
 under Win32 for my needs but i do not know their counterparts 
 in FreeVSD, Linux and OSX; that's why i'm trying to use an 
 already implemented object from the runtime.

 This is what i'm trying to do:

 void MyFunc()
 {

 }


Damn, i hit enter and posted before completion, fail! Here's the 
code:

/////////////////////////////////////////////////
bool do_loop;

// This function runs in its own thread.
void MyFunc()
{
	while(true)
	{
		if(!do_loop)
			myobject.wait(); // Wait for sync object.
		// ... Do other stuff. "do_loop" can be set to false again here.
	}
}

// This one is called by different threads.
void loop(bool val)
{
	do_loop = val;
	if(do_loop) // Release sync object if it is waiting.
		myobject.notify();
}
/////////////////////////////////////////////////

The idea is that i have a code that loops and do a lot of stuff 
but sometimes i don't want it to loop so i set do_loop to false. 
When i want it to loop i set do_loop to true and release the 
waiting object.
The problem of using a semaphore is that if i call loop() with 
true multiple times, my code will perform extra loops until the 
internal counter gets to 0, that's why i need a 0-1 sync object.

Any suggestions?

Thank you in advance.
May 13 2013
prev sibling parent reply "Heinz" <thor587 gmail.com> writes:
Ok, here's a summary in case someone else is in the same need:

1) Want to know what "mutex/condition" are, how do they work and 
when to use them? Here's a good resource: 
http://stackoverflow.com/a/4742236

2) If you come to the question "Why a condition needs a mutex?": 
http://stackoverflow.com/questions/2763714/why-do-pthreads-condition-variable-functions-require-a-mutex

3) Need a working example in D? (by Steven Schveighoffer): 
http://forum.dlang.org/thread/j7sdte$25qm$1 digitalmars.com
May 13 2013
next sibling parent reply "Juan Manuel Cabo" <juanmanuel.cabo gmail.com> writes:
On Monday, 13 May 2013 at 20:44:37 UTC, Heinz wrote:

 Ok, here's a summary in case someone else is in the same need:

 1) Want to know what "mutex/condition" are, how do they work 
 and when to use them? Here's a good resource: 
 http://stackoverflow.com/a/4742236

 2) If you come to the question "Why a condition needs a 
 mutex?": 
 http://stackoverflow.com/questions/2763714/why-do-pthreads-condition-variable-functions-require-a-mutex

 3) Need a working example in D? (by Steven Schveighoffer): 
 http://forum.dlang.org/thread/j7sdte$25qm$1 digitalmars.com



Here is the absolute minimum of code I could think to try 
Condition. I tested it on linux 64bits and windows 32bits. There 
is one thing that should definitely added to the documentation, 
and that is what happens when one issues a notify while the 
thread hasn't yet called Condition.wait().

I seem to recall (though I might be wrong) that win32 events can 
be signalled before the thread calls WaitForSingleObject (or 
WaitForMultipleObjects), and if it was signalled, the call 
returns immediately. I think this was very useful.


import std.stdio, core.thread, core.sync.condition, 
core.sync.mutex;

//Note: The condition must be __gshared:
__gshared Condition condition;
__gshared Mutex mutex;

private void myThread() {
     writeln("Started");
     synchronized(mutex) {
         condition.wait();
     }
     writeln("Notified");
}

void main() {
     mutex = new Mutex();
     condition = new Condition(mutex);

     Thread theThread = new Thread(&myThread);
     theThread.start();

     //Note: if one doesn't wait for the thread to
     //get to condition.wait(), before calling notify(),
     //the thread is not awaken. (ie: commenting the
     //sleep makes the thread stuck).
     Thread.sleep(dur!"msecs"(100));
     synchronized(mutex) {
         condition.notify();
     }

     theThread.join();
     writeln("main finished");
}
May 13 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 13 May 2013 17:04:22 -0400, Juan Manuel Cabo  
<juanmanuel.cabo gmail.com> wrote:


 I seem to recall (though I might be wrong) that win32 events can be  
 signalled before the thread calls WaitForSingleObject (or  
 WaitForMultipleObjects), and if it was signalled, the call returns  
 immediately. I think this was very useful.


Right, the equivalent to a windows event is a D Mutex, Condition, and a  
protected boolean.  In other words, the condition aids synchronization of  
a message, but it is not the message itself!  Windows events could serve  
as the message (but not necessarily).

-Steve
May 13 2013
parent "Juan Manuel Cabo" <juanmanuel.cabo gmail.com> writes:
On Monday, 13 May 2013 at 21:55:45 UTC, Steven Schveighoffer 
wrote:

 On Mon, 13 May 2013 17:04:22 -0400, Juan Manuel Cabo 
 <juanmanuel.cabo gmail.com> wrote:


 I seem to recall (though I might be wrong) that win32 events 
 can be signalled before the thread calls WaitForSingleObject 
 (or WaitForMultipleObjects), and if it was signalled, the call 
 returns immediately. I think this was very useful.


 Right, the equivalent to a windows event is a D Mutex, 
 Condition, and a protected boolean.  In other words, the 
 condition aids synchronization of a message, but it is not the 
 message itself!  Windows events could serve as the message (but 
 not necessarily).

 -Steve


Thanks!!! Your explanation finally made it clear for me. It would 
be nice if this last paragraph of yours was in the documentation. 
It would save some headakes for people like me (I first viewed 
Condition as a drop in replacement of win32 events, didn't know 
of the pthreads way). And also, after reading the other link 
posted above:

    
http://stackoverflow.com/questions/2763714/why-do-pthreads-condition-variable-functions-require-a-mutex

I think I finally got it.

Thanks again!
--jm
May 13 2013
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 13 May 2013 16:44:36 -0400, Heinz <thor587 gmail.com> wrote:


 3) Need a working example in D? (by Steven Schveighoffer):  
 http://forum.dlang.org/thread/j7sdte$25qm$1 digitalmars.com


Well, whaddya know.  Glad my former self could help :)

BTW, given recent discussion on memory barriers, I think my previous  
statement that the mutex does not need to be locked to call notify is  
probably incorrect.

-Steve
May 13 2013
parent reply "Heinz" <thor587 gmail.com> writes:
 BTW, given recent discussion on memory barriers, I think my 
 previous statement that the mutex does not need to be locked to 
 call notify is probably incorrect.


Haven't had any issues calling notify outside a synchronized 
statement, even from multiple threads. At least this works under 
Win32 with my producer thread all wrapped inside synchronized(). 
Only a single consumer thread is inside synchronized() but then i 
have 2 more threads making naked calls to notify(). Not a single 
crash.
May 13 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
14-May-2013 08:33, Heinz пишет:

 BTW, given recent discussion on memory barriers, I think my previous
 statement that the mutex does not need to be locked to call notify is
 probably incorrect.




Have to lock it otherwise you have a race condition on a condition 
variable (wow!).


 Haven't had any issues calling notify outside a synchronized statement,
 even from multiple threads. At least this works under Win32 with my
 producer thread all wrapped inside synchronized(). Only a single
 consumer thread is inside synchronized() but then i have 2 more threads
 making naked calls to notify(). Not a single crash.


Doesn't prove anything, it could happen that you just miss a 
notification, for instance. Another common case is that it so happens 
that wait will (with luck) always happen before any of notify and 
notifications come spaced out in time.

-- 
Dmitry Olshansky
May 14 2013
next sibling parent reply "Heinz" <thor587 gmail.com> writes:
On Monday, 13 May 2013 at 21:04:23 UTC, Juan Manuel Cabo wrote:


 There is one thing that should definitely added to the 
 documentation, and that is what happens when one issues a 
 notify while the thread hasn't yet called Condition.wait().


I can confirm that under Win32 calling notify() before wait()
internally signals the condition and then calling wait() returns
immediately and actually does not wait. This is the expected
behavior and is actually how Win32 events work.

On Tuesday, 14 May 2013 at 08:58:31 UTC, Dmitry Olshansky wrote:


 Have to lock it otherwise you have a race condition on a 
 condition variable (wow!).


Ok, i'll lock it just in case. It also makes me feel my code is
more robust. This will do right?

...
synchronized(cond.mutex)
      cond.notify();
...

My internal bool variable that affects the condition (the one
that decides if the consumer thread should wait) must be setable
at any moment by any thread so i leave it outside the lock. Also,
after setting this variable i immediately call notify() with
mutex unlocked. That's why it is working i think.


 Doesn't prove anything, it could happen that you just miss a 
 notification, for instance. Another common case is that it so 
 happens that wait will (with luck) always happen before any of 
 notify and notifications come spaced out in time.


True, i guess my code work 100% as needed because of its
implementation, in my case it doesn't matter if i miss a
notification because the result of that notification gets
evaluated in the next loop inside a while, but still most
probably as you say wait() is always happening before a
notification.
A classic producer-consumer program must lock both calls, i do
understand that.
May 14 2013
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
14-May-2013 20:09, Heinz пишет:

 On Monday, 13 May 2013 at 21:04:23 UTC, Juan Manuel Cabo wrote:


 There is one thing that should definitely added to the documentation,
 and that is what happens when one issues a notify while the thread
 hasn't yet called Condition.wait().


 I can confirm that under Win32 calling notify() before wait()
 internally signals the condition and then calling wait() returns
 immediately and actually does not wait. This is the expected
 behavior and is actually how Win32 events work.




 On Tuesday, 14 May 2013 at 08:58:31 UTC, Dmitry Olshansky wrote:


 Have to lock it otherwise you have a race condition on a condition
 variable (wow!).


 Ok, i'll lock it just in case. It also makes me feel my code is
 more robust. This will do right?

 ...
 synchronized(cond.mutex)
       cond.notify();
 ...

 My internal bool variable that affects the condition (the one
 that decides if the consumer thread should wait) must be setable
 at any moment by any thread so i leave it outside the lock.


Wrong. Before setting it from some other thread you should grab the 
lock. See Sean's example.


 Also,
 after setting this variable i immediately call notify() with
 mutex unlocked. That's why it is working i think.



-- 
Dmitry Olshansky
May 14 2013
parent reply "Heinz" <thor587 gmail.com> writes:
Guys, this is a precise example of what i'm trying to do. You'll 
notice that there're 2 ways of waking up the consumer:

/////////////////////////////////////////////////////////////////////
Condition cond; // Previously instantiated.
bool loop = false; // This variable determine if the consumer 
should take the next iteration or wait until a thread calls 
SetLoop(true).
Object my_variable; // A value used for comunicating producer and 
consumer. It's not a prerequisite to be set for an iteration to 
happen.

// This function is called by multiple threads.
void SetLoop(bool val)
{
	if(val != loop)
	{
		loop = val;
		if(loop)
			cond.notify(); // Wake up consumer in case it was waiting.
	}
}

// This function is called by multiple threads.
bool GetLoop()
{
	return loop;
}

void MyConsumer()
{
	while(true)
	{
		synchronized(cond.mutex)
		{
			if(loop == false)
				cond.wait(); // Wait for producer or SetLoop(true).
			
			if(my_variable !is null)
			{
				/*
				...
				Call private function 1. SetLoop(true/false) might be called 
here.
				...
				*/
				my_variable = null; // Lock is useful here for setting 
my_variable.
			}
			
			// These conditions are intentionally placed here.
			if(loop == false)
				continue; // Jump to next iteration. Please note that 
cond.mutex gets released and reaquired in the next iteration.
			else
				loop = false; // Reset waiting on every iteration. Can be 
modified by private function 2 below.
			
			/*
			...
			Call private function 2. SetLoop(true/false) might be called 
here.
			...
			*/
		}
	}
}

void MyProducer()
{
	while(true)
	{
		if(/* Some other condition */)
		{
			synchronized(cond.mutex) // Lock is useful here for setting 
my_variable.
			{
				my_variable = /* Some value */;
				cond.notify(); // Wake up consumer in case it was waiting.
			}
		}
	}
}
/////////////////////////////////////////////////////////////////////

Wouldn't wrapping all the lines in "SetLoop()" inside a 
"synchronized(cond.mutex)" statement produce a deadlock because 
it might be called by private functions 1 or 2? Should i only 
wrap cond.notify() or is it of no meaning in this case? Like this:

void SetLoop(bool val)
{
	if(val != loop)
	{
		loop = val;
		if(loop)
		{
			synchronized(cond.mutex)
				cond.notify(); // Wake up consumer in case it was waiting.
		}
	}
}

Synchronization of variable "loop" doesn't seem important in this 
case. Do you still recommend to use an extra mutex to set it? 
like this:

void SetLoop(bool val)
{
	synchronized(extra_mutex)
	{
		if(val != loop)
		{
			loop = val;
			if(loop)
			{
				synchronized(cond.mutex)
					cond.notify(); // Wake up consumer in case it was waiting.
			}
		}
	}
}

void MyConsumer()
{
	//...
	if(loop == false)
		continue;
	else
	{
		synchronized(extra_mutex)
			loop = false;
	}
	//...
}

Thank you guys so much for your time and your expertise.
May 14 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 14 May 2013 13:59:51 -0400, Heinz <thor587 gmail.com> wrote:


 Guys, this is a precise example of what i'm trying to do. You'll notice  
 that there're 2 ways of waking up the consumer:


2 things:

1. D mutex locks are re-entrant.  That is, you can do:

synchronized(mutex)
{
    synchronized(mutex)
    {
       ...
    }
}

and all is fine.

2. The condition logic in your code is wrong.  The condition/mutex  
protects the messages being sent to the consumer.

It seems that you have two messages (which is OK):

  - my_variable is present, needs to be processed
  - Run function 2 (flag loop == run function 2), set from inside the  
consumer, and some undisclosed threads.  I find this logic rather  
dangerous, but it is your app, I have no idea of the real meaning :)

So I would do it like this (names TBD based on your real use case)

void SetLoop(bool yesOrNo)
{
    synchronized(cond.mutex)
    {
        if((loop = yesOrNo) == true)
             cond.notify();
    }
}

void MyConsumer()
{
	while(true)
	{
                 Object _my_variable = null;
		synchronized(cond.mutex)
		{
			while(!loop && my_variable is null) // always wait for *real* messages  
to be set
				cond.wait();
			_my_variable = my_variable;
			my_variable = null; // consume
                 }

		// process local consumed copy of _my_variable *outside* of lock
		if(_my_variable !is null)
		{
			/*
			...
			Call private function 1. SetLoop(true/false) might be called here.
			...
			*/
		}
			
		// These conditions are intentionally placed here.
		synchronized(cond.mutex)
		{
			if(loop == false)
				continue; // Jump to next iteration. Please note that cond.mutex gets  
released and reaquired in the next iteration.
			else
				loop = false; // Reset waiting on every iteration. Can be modified by  
private function 2 below.
		}
			
		/*
		...
		Call private function 2. SetLoop(true/false) might be called here.
		...
		*/
	}
}

Then everything should be good.  I'd also add a "SetMyVariable" function  
too, akin to SetLoop.

-Steve
May 14 2013
parent reply "Heinz" <thor587 gmail.com> writes:
To Steven:


 1. D mutex locks are re-entrant.  That is, you can do:

 synchronized(mutex)
 {
    synchronized(mutex)
    {
       ...
    }
 }

 and all is fine.


Hmmm, i guess this is what the docs meant by "mutexes are 
recursive", i didn't get it at first but now i do.
To test this i wrapped SetLoop() in synchronized(cond.mutex) and 
called it from consumer wich was already all wrapped by the same 
synchronized(cond.mutex), i thought at first it was going to be a 
deadlock but i guess this is what you mean because it works like 
a charm.

Thanks for your code example, i'll update my code with your 
suggestions. But there's one thing i don't understand in your 
example: If the producer is inside a while() generating multiple 
messages once in a while then when the lock is released in the 
consumer, the producer will replace "my_variable" a couple of 
times with the latest message until consumer gets to wait() 
again. That's why i locked the whole consumer, to sync all 
messages. Am i wrong here?

To Sean:


 Let's back up a minute.  Can you be more specific about what 
 you're trying to do?  I think you shouldn't need to use the 
 "loop" var at all, but I'm not entirely sure.  Also, loop and 
 my_variable will be thread-local given how they're declared.


Fair enough, i uploaded a bunch of code hoping that experienced 
programmers like you to understand right away what i'm doing.

I have this "main loop" wich is named here as my consumer 
function. I didn't want this main loop to be cycling all the time 
if there was nothing to do. So i needed an event object 
(condition/mutex) that locked the loop until any of the 2 program 
events (or both) occur:

1) A custom program message was sent by producer to main loop to 
notify for example mouse move, key press, timer tick, etc.
2) The user explicitly want to do something else (execute custom 
code or perform an action not covered by program messages) or did 
something that now needs to be processed inside the main loop. In 
this case the user calls SetLoop(true) to notify the main loop 
and then do what the user want.

That's why i have and need both types of messages: bool loop; and 
Object my_variable;

My code seems to work fine but i did not know if it was the best 
approach. Now i know after your help that i can and must protect 
both messages with the condition's mutex and have all in sync. 
Also i learned that wait() and notify() must be inside a lock.

To ALL:

Sorry for my incompetence. I'm not new to D but i am to D2 and 
multithreading. I got stuck in D1, specially in the DMD1.030 era 
(those were the most stable and beautiful times) but now i 
decided it was time to evolve. So far, learning D2 and 
multithreading hasn't been easy, but with your help guys i've 
accomplished many things.

Thank you again for your support.
May 14 2013
parent reply "Juan Manuel Cabo" <juanmanuel.cabo gmail.com> writes:
On Wednesday, 15 May 2013 at 02:25:02 UTC, Heinz wrote:

 ....
 I have this "main loop" wich is named here as my consumer 
 function. I didn't want this main loop to be cycling all the 
 time if there was nothing to do. So i needed an event object 
 (condition/mutex) that locked the loop until any of the 2 
 program events (or both) occur:

 1) A custom program message was sent by producer to main loop 
 to notify for example mouse move, key press, timer tick, etc.
 2) The user explicitly want to do something else (execute 
 custom code or perform an action not covered by program 
 messages) or did something that now needs to be processed 
 inside the main loop. In this case the user calls SetLoop(true) 
 to notify the main loop and then do what the user want.

 That's why i have and need both types of messages: bool loop; 
 and Object my_variable;
 ....


It sounds like you need to:
     1) use a Message Queue.
     2) Copy the message while you work on it with the consumer, 
so that you can exit the mutex.

If you don't use a message queue, then the thread that feeds the 
consumer will have to wait on the consumer to finish processing 
the last message, just to post a new message. Thus removing the 
advantage of processing the message in a different thread.

If you use a message queue, make it so that you exit the mutex 
lock while you process the message, and then reaquire the mutex 
to use the message queue. This is so that you can keep adding 
messages to the queue without having to wait until the last 
message is consumed.

There are many ways to implement a queue for thread message 
passing, some are more advanced and require less locking. But you 
can start with a fixed array.

--jm
May 15 2013
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 15 May 2013 at 15:35:05 UTC, Juan Manuel Cabo wrote:

 It sounds like you need to:
     1) use a Message Queue.
     2) Copy the message while you work on it with the consumer, 
 so that you can exit the mutex.


At which point I'll suggest considering std.concurrency instead 
of rolling everything yourself.
May 15 2013
prev sibling next sibling parent Sean Kelly <sean invisibleduck.org> writes:
On May 14, 2013, at 10:59 AM, Heinz <thor587 gmail.com> wrote:


 Guys, this is a precise example of what i'm trying to do. You'll =


notice that there're 2 ways of waking up the consumer:

=20
 /////////////////////////////////////////////////////////////////////
 Condition cond; // Previously instantiated.
 bool loop =3D false; // This variable determine if the consumer should =


take the next iteration or wait until a thread calls SetLoop(true).

 Object my_variable; // A value used for comunicating producer and =


consumer. It's not a prerequisite to be set for an iteration to happen.

Let's back up a minute.  Can you be more specific about what you're =
trying to do?  I think you shouldn't need to use the "loop" var at all, =
but I'm not entirely sure.  Also, loop and my_variable will be =
thread-local given how they're declared.=
May 14 2013
prev sibling parent "Juan Manuel Cabo" <juanmanuel.cabo gmail.com> writes:
On Tuesday, 14 May 2013 at 17:59:55 UTC, Heinz wrote:

 Guys, this is a precise example of what i'm trying to do. 
 You'll notice that there're 2 ways of waking up the consumer:

 /////////////////////////////////////////////////////////////////////
 Condition cond; // Previously instantiated.
 bool loop = false; // This variable determine if the consumer 
 should take the next iteration or wait until a thread calls 
 SetLoop(true).
 Object my_variable; // A value used for comunicating producer 
 and consumer. It's not a prerequisite to be set for an 
 iteration to happen.


You also need to declare Condition as __gshared. If you don't,
each thread will see a different Condition object or null,
because by default, every D global variable goes to Thread Local
Storage.

Any variables meant to be shared between threads need to be
either '__gshared' or 'shared'. You could also declare other
shared data as 'shared', but I don't think that's what you want
here.

To list all global variables not declared as __gshared, you can
compile using the -vtls switch.

--jm
May 14 2013
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On May 14, 2013, at 9:09 AM, Heinz <thor587 gmail.com> wrote:


 On Monday, 13 May 2013 at 21:04:23 UTC, Juan Manuel Cabo wrote:
=20

 There is one thing that should definitely added to the documentation, =




and that is what happens when one issues a notify while the thread =
hasn't yet called Condition.wait().

=20
 I can confirm that under Win32 calling notify() before wait()
 internally signals the condition and then calling wait() returns
 immediately and actually does not wait. This is the expected
 behavior and is actually how Win32 events work.


A Win32 event can be simulated basically like so:

class Event {
    Condition c;
    bool signaled =3D false;

    this() {
        c =3D new Condition(new Mutex);
    }

    void wait() {
        synchronized (c.mutex) {
            while (!signaled)
                c.wait();
            signaled =3D false;
        }
    }

    void notify() {
        synchronized (c.mutex) {
            signaled =3D true;
            c.notify();
        }
    }
}

auto e =3D new Event;

T get() {
    while (true) {
        e.wait();
        // A -- race here
        synchronized (m) {
            if (!m.isEmpty())
                return m.take();
        }
    }
}

void put(T val) {
    synchronized(m) {
        m.add(val);
    }
    e.notify();
}

You'll notice the redundancy here though to combat the race at point A.  =
Generally, you really want to avoid the Win32 model and use the =
mutex/condition pair directly with your container or whatever.



 On Tuesday, 14 May 2013 at 08:58:31 UTC, Dmitry Olshansky wrote:
=20

 Have to lock it otherwise you have a race condition on a condition =




variable (wow!).

=20
 Ok, i'll lock it just in case. It also makes me feel my code is
 more robust. This will do right?
=20
 ...
 synchronized(cond.mutex)
     cond.notify();
 =85


Yep.



 My internal bool variable that affects the condition (the one
 that decides if the consumer thread should wait) must be setable
 at any moment by any thread so i leave it outside the lock. Also,
 after setting this variable i immediately call notify() with
 mutex unlocked. That's why it is working i think.


I don't understand what you mean here.  If the variable is protected by =
a lock it can still be set by any thread at any time.  Just only one =
thread at a time.  Doing a lock-free write of the variable basically =
just means that the variable will probably be set eventually, which is =
rarely what you actually want.=
May 14 2013
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
I'm doing this from my phone so please bear with me.

You use a mutex in combination with a condition variable so you 
can check the state of something to determine if waiting is 
necessary. So the classic producer/consumer would be something 
like:

T get() {
     shnchronized(c.mutex) {
         while (q.isEmpty)
             c.wait();
         return q.take();
     }
}

void put(T val) {
     synchronized (c.mutex) {
         q.add(val);
         c.notify();
     }
}

You can emulate a Win32 event that stays flipped by 
checking/setting a bool or int inside the mutex.
May 14 2013
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 14 May 2013 04:58:27 -0400, Dmitry Olshansky  =

<dmitry.olsh gmail.com> wrote:


 14-May-2013 08:33, Heinz =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

 BTW, given recent discussion on memory barriers, I think my previous=







 statement that the mutex does not need to be locked to call notify i=






s

 probably incorrect.




 Have to lock it otherwise you have a race condition on a condition  =



 variable (wow!).


No, the issue would be reordering (is that possible in this case?).  The=
  =

actual signaling of the condition would not require the lock, but you  =

still need to lock to send the message (e.g. set the boolean) or you wil=
l  =

definitely have issues.

But since you have to lock anyway, signaling while holding the lock, or =
 =

while being outside the lock isn't really a difference.

Maybe I'm wrong...


 Haven't had any issues calling notify outside a synchronized statemen=




t,

 even from multiple threads. At least this works under Win32 with my
 producer thread all wrapped inside synchronized(). Only a single
 consumer thread is inside synchronized() but then i have 2 more threa=




ds

 making naked calls to notify(). Not a single crash.


 Doesn't prove anything, it could happen that you just miss a  =



 notification, for instance. Another common case is that it so happens =


 =


 that wait will (with luck) always happen before any of notify and  =



 notifications come spaced out in time.


I don't see how you could miss a notification, can you explain further?

-Steve
May 14 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
14-May-2013 21:02, Steven Schveighoffer пишет:

 On Tue, 14 May 2013 04:58:27 -0400, Dmitry Olshansky
 <dmitry.olsh gmail.com> wrote:


 14-May-2013 08:33, Heinz пишет:

 BTW, given recent discussion on memory barriers, I think my previous
 statement that the mutex does not need to be locked to call notify is
 probably incorrect.




 Have to lock it otherwise you have a race condition on a condition
 variable (wow!).


 No, the issue would be reordering (is that possible in this case?).  The
 actual signaling of the condition would not require the lock, but you
 still need to lock to send the message (e.g. set the boolean) or you
 will definitely have issues.


Never had to notify without sending some message (otherwise how you'd 
affect the thread on the other side of fence?). I'd have to dig into how 
D's condition variables are done to say for sure. But I thought that 
notify is not re-entrant that is it needs to be locked (as there is a 
queue of threads to manage).

I just had fresh experience in Java where you basically can't notify 
threads waiting on the monitor without grabbing the lock, same with call 
to wait. It's a clean-cut run-time error (when it can detect it).


 But since you have to lock anyway, signaling while holding the lock, or
 while being outside the lock isn't really a difference.


On the level of gut feeling there must be something about it as you 
don't see:

synchronized(m){
	// ... send message
}
notify();

anytime of day. And hosting work out of mutex seems natural isn't it?
Not in condition to analyze just yet but one clue is that it seems like 
you may be notifying a thread that would wake up only to block on mutex 
somebody else holds.


 Maybe I'm wrong...


Maybe I am.


 Haven't had any issues calling notify outside a synchronized statement,
 even from multiple threads. At least this works under Win32 with my
 producer thread all wrapped inside synchronized(). Only a single
 consumer thread is inside synchronized() but then i have 2 more threads
 making naked calls to notify(). Not a single crash.


 Doesn't prove anything, it could happen that you just miss a
 notification, for instance. Another common case is that it so happens
 that wait will (with luck) always happen before any of notify and
 notifications come spaced out in time.


 I don't see how you could miss a notification, can you explain further?


Not particularly well put. In general I was implying that program not 
crushing doesn't mean anything useful. It might work either due to 
happenstance. Depending on the implementation of notify, if it's not 
re-entrant then technically anything fishy can happen.

-- 
Dmitry Olshansky
May 14 2013
parent reply Sean Kelly <sean invisibleduck.org> writes:
On May 14, 2013, at 12:02 PM, Dmitry Olshansky <dmitry.olsh gmail.com> =
wrote:


 14-May-2013 21:02, Steven Schveighoffer =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

=20
 But since you have to lock anyway, signaling while holding the lock, =




or

 while being outside the lock isn't really a difference.
=20


=20
 On the level of gut feeling there must be something about it as you =


don't see:

=20
 synchronized(m){
 	// ... send message
 }
 notify();
=20
 anytime of day. And hosting work out of mutex seems natural isn't it?



If you move the notify out of the mutex then you can end up with =
multiple threads competing for the same value.  Say the producer puts a =
value in the queue, leaves the mutex, and notifies a waiting thread.  =
Then consumer A enters the mutex, sees that there's something in the =
container and takes it, then consumer B receives the notification and =
wakes up to discover that the container is empty.  So long as your wait =
loops are done properly the only bad result will be pointless wakeups, =
but worst case you could have a crash or exception if you're removing =
data from the container without checking if it's empty.=
May 14 2013
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 14 May 2013 15:19:29 -0400, Sean Kelly <sean invisibleduck.org>  
wrote:


 If you move the notify out of the mutex then you can end up with  
 multiple threads competing for the same value.  Say the producer puts a  
 value in the queue, leaves the mutex, and notifies a waiting thread.   
 Then consumer A enters the mutex, sees that there's something in the  
 container and takes it, then consumer B receives the notification and  
 wakes up to discover that the container is empty.  So long as your wait  
 loops are done properly the only bad result will be pointless wakeups,  
 but worst case you could have a crash or exception if you're removing  
 data from the container without checking if it's empty.


Any thread that does not check to ensure his data is ready while waiting  
for a condition, I would argue is incorrectly implemented -- cond.notify  
is not directly tied to the data, and there's always cond.notifyAll which  
could wake up any number of threads.

I also don't think there's a requirement that threads waiting on a  
condition have priority over threads simply trying to lock.

But I see little reason to avoid putting the cond.notify inside the lock.   
You HAVE to lock to insert the data anyway.

And Dmitry seems to have some Java experience that indicates it's at least  
required there.  I'd say go with notifying only while locked.

I learn something every day :)

-Steve
May 14 2013