www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Threading Questions

reply bitwise <bitwise.pvt gmail.com> writes:
Hey, I've got a few questions if anybody's got a minute.

I'm trying to wrap my head around the threading situation in D. 
So far, things seem to be working as expected, but I want to 
verify my solutions.

1) Are the following two snippets exactly equivalent(not just in 
observable behaviour)?
a)

Mutex mut;
mut.lock();
scope(exit) mut.unlock();

b)
Mutex mut;
synchronized(mut) { }

Will 'synchronized' call 'lock' on the Mutex, or do something 
else(possibly related to the interface Object.Monitor)?

2) Phobos has 'Condition' which takes a Mutex in the constructor. 
The documentation doesn't exactly specify this, but should I 
assume it works the same as std::condition_variable in C++?

For example, is this correct?

Mutex mut;
Condition cond = new Condition(mut);

// mut must be locked before calling Condition.wait
synchronized(mut)  // depends on answer to (1)
{
     // wait() unlocks the mutex and enters wait state
     // wait() must re-acquire the mutex before returning when 
cond is signalled
     cond.wait();
}

3) Why do I have to pass a "Mutex" to "Condition"? Why can't I 
just pass an "Object"?

4) Will D's Condition ever experience spurious wakeups?

5) Why doesn't D's Condition.wait take a predicate? I assume this 
is because the answer to (4) is no.

6) Does 'shared' actually have any effect on non-global variables 
beside the syntactic regulations?

I know that all global variables are TLS unless explicitly marked 
as 'shared', but someone once told me something about 'shared' 
affecting member variables in that accessing them from a separate 
thread would return T.init instead of the actual value... or 
something like that. This seems to be wrong(thankfully).

For example, I have created this simple Worker class which seems 
to work fine without a 'shared' keyword in sight(thankfully). I'm 
wondering though, if there would be any unexpected consequences 
of doing things this way.

http://dpaste.com/2ZG2QZV




Thanks!
     Bit
Sep 25 2015
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
Pretty please? :)
Sep 25 2015
prev sibling next sibling parent ponce <contact gam3sfrommars.fr> writes:
Sorry I don't know the answers but these questions are 
interesting so BUMP ;)

On Friday, 25 September 2015 at 15:19:27 UTC, bitwise wrote:
 1) Are the following two snippets exactly equivalent(not just 
 in observable behaviour)?
 a)

 Mutex mut;
 mut.lock();
 scope(exit) mut.unlock();

 b)
 Mutex mut;
 synchronized(mut) { }

 Will 'synchronized' call 'lock' on the Mutex, or do something 
 else(possibly related to the interface Object.Monitor)?
Don't know. Is this Object monitor a mutex or something else?
 6) Does 'shared' actually have any effect on non-global 
 variables beside the syntactic regulations?
Don't think so.
Sep 26 2015
prev sibling next sibling parent reply Russel Winder via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
I hadn't answered as I do not have answers to the questions you ask. My
reason: people should not be doing their codes using these low-level
shared memory techniques. Data parallel things should be using the
std.parallelism module. Dataflow-style things should be using spawn and
channels =E2=80=93 akin to the way you do things in Go.

So to give you an answer I would go back a stage, forget threads,
mutexes, synchronized, etc. and ask what do you want you workers to do?
If they are to do something and return a result then spawn and channel
is exactly the right abstraction to use. Think "farmer=E2=80=93worker", the
farmer spawns the workers and then collects their results. No shared
memory anywyere =E2=80=93 at least not mutable.

On Fri, 2015-09-25 at 15:19 +0000, bitwise via Digitalmars-d-learn
wrote:
 Hey, I've got a few questions if anybody's got a minute.
=20
 I'm trying to wrap my head around the threading situation in D.=20
 So far, things seem to be working as expected, but I want to=20
 verify my solutions.
=20
 1) Are the following two snippets exactly equivalent(not just in=20
 observable behaviour)?
 a)
=20
 Mutex mut;
 mut.lock();
 scope(exit) mut.unlock();
=20
 b)
 Mutex mut;
 synchronized(mut) { }
=20
 Will 'synchronized' call 'lock' on the Mutex, or do something=20
 else(possibly related to the interface Object.Monitor)?
=20
 2) Phobos has 'Condition' which takes a Mutex in the constructor.=20
 The documentation doesn't exactly specify this, but should I=20
 assume it works the same as std::condition_variable in C++?
=20
 For example, is this correct?
=20
 Mutex mut;
 Condition cond =3D new Condition(mut);
=20
 // mut must be locked before calling Condition.wait
 synchronized(mut)  // depends on answer to (1)
 {
      // wait() unlocks the mutex and enters wait state
      // wait() must re-acquire the mutex before returning when=20
 cond is signalled
      cond.wait();
 }
=20
 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I=20
 just pass an "Object"?
=20
 4) Will D's Condition ever experience spurious wakeups?
=20
 5) Why doesn't D's Condition.wait take a predicate? I assume this=20
 is because the answer to (4) is no.
=20
 6) Does 'shared' actually have any effect on non-global variables=20
 beside the syntactic regulations?
=20
 I know that all global variables are TLS unless explicitly marked=20
 as 'shared', but someone once told me something about 'shared'=20
 affecting member variables in that accessing them from a separate=20
 thread would return T.init instead of the actual value... or=20
 something like that. This seems to be wrong(thankfully).
=20
 For example, I have created this simple Worker class which seems=20
 to work fine without a 'shared' keyword in sight(thankfully). I'm=20
 wondering though, if there would be any unexpected consequences=20
 of doing things this way.
=20
 http://dpaste.com/2ZG2QZV
=20
=20
=20
=20
 Thanks!
      Bit
--=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 28 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Monday, 28 September 2015 at 11:47:38 UTC, Russel Winder wrote:
 I hadn't answered as I do not have answers to the questions you 
 ask. My reason: people should not be doing their codes using 
 these low-level shared memory techniques. Data parallel things 
 should be using the std.parallelism module. Dataflow-style 
 things should be using spawn and channels – akin to the way you 
 do things in Go.

 So to give you an answer I would go back a stage, forget 
 threads, mutexes, synchronized, etc. and ask what do you want 
 you workers to do? If they are to do something and return a 
 result then spawn and channel is exactly the right abstraction 
 to use. Think "farmer–worker", the farmer spawns the workers 
 and then collects their results. No shared memory anywyere – at 
 least not mutable.
https://www.youtube.com/watch?v=S7pGs7JU7eM Bit
Sep 28 2015
parent Russel Winder via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Tue, 2015-09-29 at 03:05 +0000, bitwise via Digitalmars-d-learn
wrote:
 On Monday, 28 September 2015 at 11:47:38 UTC, Russel Winder wrote:
 I hadn't answered as I do not have answers to the questions you=20
 ask. My reason: people should not be doing their codes using=20
 these low-level shared memory techniques. Data parallel things=20
 should be using the std.parallelism module. Dataflow-style=20
 things should be using spawn and channels =E2=80=93 akin to the way you=
=20
 do things in Go.
=20
 So to give you an answer I would go back a stage, forget=20
 threads, mutexes, synchronized, etc. and ask what do you want=20
 you workers to do? If they are to do something and return a=20
 result then spawn and channel is exactly the right abstraction=20
 to use. Think "farmer=E2=80=93worker", the farmer spawns the workers=
=20
 and then collects their results. No shared memory anywyere =E2=80=93 at=
=20
 least not mutable.
=20 https://www.youtube.com/watch?v=3DS7pGs7JU7eM =20 Bit
What's the tl;dr as text, I very, very rarely watch videos. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 29 2015
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/25/15 11:19 AM, bitwise wrote:
 Hey, I've got a few questions if anybody's got a minute.

 I'm trying to wrap my head around the threading situation in D. So far,
 things seem to be working as expected, but I want to verify my solutions.

 1) Are the following two snippets exactly equivalent(not just in
 observable behaviour)?
 a)

 Mutex mut;
 mut.lock();
 scope(exit) mut.unlock();

 b)
 Mutex mut;
 synchronized(mut) { }

 Will 'synchronized' call 'lock' on the Mutex, or do something
 else(possibly related to the interface Object.Monitor)?
Yes. A mutex object has it's internal lock as its monitor.
 2) Phobos has 'Condition' which takes a Mutex in the constructor. The
 documentation doesn't exactly specify this, but should I assume it works
 the same as std::condition_variable in C++?
I am not sure about std::condition_variable. core.sync.condition works like a standard condition (https://en.wikipedia.org/wiki/Monitor_%28synchronization%29)
 For example, is this correct?

 Mutex mut;
 Condition cond = new Condition(mut);

 // mut must be locked before calling Condition.wait
 synchronized(mut)  // depends on answer to (1)
 {
      // wait() unlocks the mutex and enters wait state
      // wait() must re-acquire the mutex before returning when cond is
 signalled
      cond.wait();
 }
Yes, I believe it is.
 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass
 an "Object"?
An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object. I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.
 4) Will D's Condition ever experience spurious wakeups?
What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied. In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock. I think there are cases with multiple threads where you can potentially wake up the thread waiting on a condition AFTER the condition was already reset by another.
 5) Why doesn't D's Condition.wait take a predicate? I assume this is
 because the answer to (4) is no.
The actual "condition" that you are waiting on is up to you to check/define.
 6) Does 'shared' actually have any effect on non-global variables beside
 the syntactic regulations?
I believe shared doesn't alter code generation at all. It only prevents certain things and affects the type.
 I know that all global variables are TLS unless explicitly marked as
 'shared', but someone once told me something about 'shared' affecting
 member variables in that accessing them from a separate thread would
 return T.init instead of the actual value... or something like that.
 This seems to be wrong(thankfully).
No, this isn't true.
 For example, I have created this simple Worker class which seems to work
 fine without a 'shared' keyword in sight(thankfully). I'm wondering
 though, if there would be any unexpected consequences of doing things
 this way.

 http://dpaste.com/2ZG2QZV
Some errors: 1. When calling notifyAll, you should ALWAYS have the mutex locked. 2. Since the mutex is protecting _run, it should only be checked/modified with the lock held. 3. After you have been woken up, you should check that the condition is satisfied. 4. Technically, you shouldn't access member variables that are GC allocated from a dtor. I know it's a struct, but structs can be GC allocated as well. I would replace your if(tasks.empty) with while(tasks.empty && _run) to fix issue 3. -Steve
Sep 29 2015
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 29 Sep 2015 15:10:58 -0400
schrieb Steven Schveighoffer <schveiguy yahoo.com>:

 
 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just
 pass an "Object"?
An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object. I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.
But you'll need access to the Mutex in user code as well. And often you use multiple Conditions with one Mutex so a Condition doesn't really own the Mutex.
 4) Will D's Condition ever experience spurious wakeups?
What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied. In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock.
Spurious wakeup is a common term when talking about posix conditions and it does indeed mean a wait() call can return without ever calling notify(): https://en.wikipedia.org/wiki/Spurious_wakeup http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups And yes, this does happen for core.sync.condition as well. As a result you'll always have to check in a loop: synchronized(mutex) { while(some_flag_or_expression) { cond.wait(); } } ----------------- synchronized(mutex) { some_flag_or_expression = true; cond.notify(); }
 
 I think there are cases with multiple threads where you can
 potentially wake up the thread waiting on a condition AFTER the
 condition was already reset by another.
 
 5) Why doesn't D's Condition.wait take a predicate? I assume this is
 because the answer to (4) is no.
The actual "condition" that you are waiting on is up to you to check/define.
He probably means that you could pass an expression to wait and wait would do the looping / check internally. That's probably a nicer API but not implemented.
 6) Does 'shared' actually have any effect on non-global variables
 beside the syntactic regulations?
I believe shared doesn't alter code generation at all. It only prevents certain things and affects the type.
It shouldn't. I think in GDC it does generate different code, but that's an implementation detail that needs to be fixed.
Sep 29 2015
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/29/15 4:38 PM, Johannes Pfau wrote:
 Am Tue, 29 Sep 2015 15:10:58 -0400
 schrieb Steven Schveighoffer <schveiguy yahoo.com>:

 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just
 pass an "Object"?
An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object. I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.
But you'll need access to the Mutex in user code as well.
synchronized(condition.mutex)
 And often you
 use multiple Conditions with one Mutex so a Condition doesn't really
 own the Mutex.
It's just a different option. Often times, you have a condition variable, and a mutex variable. It's not super-important, you can always do: new Condition(new Mutex);
 4) Will D's Condition ever experience spurious wakeups?
What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied. In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock.
Spurious wakeup is a common term when talking about posix conditions and it does indeed mean a wait() call can return without ever calling notify(): https://en.wikipedia.org/wiki/Spurious_wakeup http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups
OK thanks.
 5) Why doesn't D's Condition.wait take a predicate? I assume this is
 because the answer to (4) is no.
The actual "condition" that you are waiting on is up to you to check/define.
He probably means that you could pass an expression to wait and wait would do the looping / check internally. That's probably a nicer API but not implemented.
yeah, that could probably be done. One thing to note is that these classes are from ages ago (probably close to 10 years). New API suggestions may be allowed. I just wanted to stress that there isn't some sort of built-in condition predicate (like a boolean). -Steve
Sep 29 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 29 September 2015 at 23:20:31 UTC, Steven 
Schveighoffer wrote:
 yeah, that could probably be done. One thing to note is that 
 these classes are from ages ago (probably close to 10 years). 
 New API suggestions may be allowed.


 -Steve
I'm still thinking about my last rant, here... So by new API, do you mean just adding a couple of new functions, or rewriting a new Condition class(as is the plan for streams)? Since D is moving towards a phobos with no GC, what will happen to things that are classes like Condition and Mutex? If DIP74 were implemented, Condition and Mutex could be made ref counted, but DIP74 seems like something that will be very complicated, and may not happen for a long time. So the only other alternative is to make it a struct, but for a Mutex, that would prevent you from doing this: Mutex m = new Mutex(); synchronized(m) { } I also don't mind the way that the current streams are made up of a class hierarchy. Although inheritance is overused sometimes, I don't think it's bad. But, if I'm correct about the current trend in D, it seems any new stream stuff will end up getting flattened into some template/struct solution. Any comments on this? Thanks, Bit
Oct 04 2015
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Sunday, October 04, 2015 14:42:48 bitwise via Digitalmars-d-learn wrote:
 Since D is moving towards a phobos with no GC, what will happen
 to things that are classes like Condition and Mutex?
Phobos and druntime will always use the GC for some things, and some things just plain need classes. Rather, we're trying to make it so that Phobos does not use the GC when it doesn't need to use the GC as well reduce how much the GC is required for stuff like string processing where lazy ranges can be used instead in many cases. As for Condition and Mutex specifically, I don't know whey they were ever classes except perhaps to take advantage of the monitor in Object. Maybe they'll get changed to structs, maybe they won't, but most D code is thread-local, and most of the code that isn't is going to use message passing, which means that explicit mutexes and conditions are unnecessary. So, most code won't be impacted regardless of what we do with Condition and Mutex. Regardless, I doubt that anything will be done with Condition or Mutex until shared is revisted, which is supposed to happen sometime soon but hasn't happened yet. What happens with shared could completely change how Condition and Mutex are handled (e.g. they don't support shared directly even though they should probably have most of their members marked with shared, because Sean Kelly didn't want to be doing anything with shared that he'd have to change later). - Jonathan M Davis
Oct 04 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Monday, 5 October 2015 at 00:23:21 UTC, Jonathan M Davis wrote:
 On Sunday, October 04, 2015 14:42:48 bitwise via 
 Digitalmars-d-learn wrote:
 Since D is moving towards a phobos with no GC, what will 
 happen to things that are classes like Condition and Mutex?
Phobos and druntime will always use the GC for some things, and some things just plain need classes. Rather, we're trying to make it so that Phobos does not use the GC when it doesn't need to use the GC as well reduce how much the GC is required for stuff like string processing where lazy ranges can be used instead in many cases.
I was under the impression that the idea was to _completely_ eliminate the GC. It says in Andre's 2015H1 vision statement: "We aim to make the standard library usable in its entirety without a garbage collector." I understand the allocation/freeing of memory is expensive, but I thought the actual sweep of the GC was a problem too, and that disabling the GC to avoid the sweep was the plan for some people. I don't know how long D's GC takes to sweep, but even a 5ms pause would be unacceptable for a performance intensive game. I guess if you use nogc properly though, you could still safely turn off the GC, right?
 As for Condition and Mutex specifically, I don't know whey they 
 were ever classes except perhaps to take advantage of the 
 monitor in Object. Maybe they'll get changed to structs, maybe 
 they won't, but most D code is thread-local, and most of the 
 code that isn't is going to use message passing, which means 
 that explicit mutexes and conditions are unnecessary. So, most 
 code won't be impacted regardless of what we do with Condition 
 and Mutex.
You may be right. I wrote a simple download manager in D using message passing. It was a little awkward at first, but in general, the spawn/send/receive API seems very intuitive. It feels awkward because the data you're working with is out of reach, but I guess it's safer that way.
 Regardless, I doubt that anything will be done with Condition 
 or Mutex until shared is revisted, which is supposed to happen 
 sometime soon but hasn't happened yet. What happens with shared 
 could completely change how Condition and Mutex are handled 
 (e.g. they don't support shared directly even though they 
 should probably have most of their members marked with shared, 
 because Sean Kelly didn't want to be doing anything with shared 
 that he'd have to change later).

 - Jonathan M Davis
I'm not sure what's going to be done with shared, but I do think it's annoying that you can't do this: shared Array!int numbers; someThread... { numbers.clear(); // 'clear' is not shared } So this means that on top of the already ridiculous number of attributes D has, now you have to mark everything as shared too =/ Bit
Oct 05 2015
next sibling parent reply Laeeth Isharc <spamnolaeeth nospamlaeeth.com> writes:
On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote:
 You may be right. I wrote a simple download manager in D using 
 message passing. It was a little awkward at first, but in 
 general, the spawn/send/receive API seems very intuitive. It 
 feels awkward because the data you're working with is out of 
 reach, but I guess it's safer that way.
Any possibility of a blog post on your experience of doing so ? ;) [I should start writing some directly, but for time being, until I have my blog up and running again, I write from time to time on Quora]. A few minutes of writing now and then can have a remarkably big impact as well as clarifying your own thoughts, and the time invested is amply repaid, even viewed from a narrowly self-interested perspective. I had same experience with learning message passing. Feels like learning to eat with chopsticks in the beginning, but soon enough it feels much more civilised when it's the right tool for the job.
Oct 05 2015
parent bitwise <bitwise.pvt gmail.com> writes:
On Monday, 5 October 2015 at 20:18:18 UTC, Laeeth Isharc wrote:
 On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote:
 You may be right. I wrote a simple download manager in D using 
 message passing. It was a little awkward at first, but in 
 general, the spawn/send/receive API seems very intuitive. It 
 feels awkward because the data you're working with is out of 
 reach, but I guess it's safer that way.
Any possibility of a blog post on your experience of doing so ? ;) [I should start writing some directly, but for time being, until I have my blog up and running again, I write from time to time on Quora]. A few minutes of writing now and then can have a remarkably big impact as well as clarifying your own thoughts, and the time invested is amply repaid, even viewed from a narrowly self-interested perspective.
Unfortunately, my time is limited right now. I do have another project, which I've decided will either be finished or discarded by the dawn of 2016. So in the near future, I should have more time for other things.
 I had same experience with learning message passing.  Feels 
 like learning to eat with chopsticks in the beginning, but soon 
 enough it feels much more civilised when it's the right tool 
 for the job.
I like the way my Worker class works because when I don't need the thread anymore, I can simply discard the object that represents the thread. As long as the Worker object is higher up on the stack than anything it's working on, all is well, and the concept of spawn/join is not visible while programming. This works out ok, because while the jobs I'm doing are slow enough to make a UI thread lag, they aren't long-running enough to where waiting for the Worker's thread to join in the destructor becomes a problem. There may be a small lag as the Worker's destructor waits for the last job to finish and the thread to join, but it's only happens once in the lifetime of the worker, so it's not a big deal. If care is not taken, the above could be subject to these problems: 1) shared memory corruption 2) worker accessing dead memory if it's placed on the stack below what it's working on 3) queueing a long running task could freeze the program on ~Worker() If you're moving or copying data into a thread, then returning the result(which can be ignored) I think most of the above can be solved. It's still a bit foreign to me though, and C++ has no such construct yet afaik. I read a bit about std::future and so on, but I'm not sure if they're standard yet. The biggest blocker though, is that the project I'm using that Worker class in is a Unity3D plugin. They only very recently updated their iOS libs to allow libc++ > 98.... Bit
Oct 05 2015
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 10/5/15 1:40 PM, bitwise wrote:
 On Monday, 5 October 2015 at 00:23:21 UTC, Jonathan M Davis wrote:
 On Sunday, October 04, 2015 14:42:48 bitwise via Digitalmars-d-learn
 wrote:
 Since D is moving towards a phobos with no GC, what will happen to
 things that are classes like Condition and Mutex?
Phobos and druntime will always use the GC for some things, and some things just plain need classes. Rather, we're trying to make it so that Phobos does not use the GC when it doesn't need to use the GC as well reduce how much the GC is required for stuff like string processing where lazy ranges can be used instead in many cases.
I was under the impression that the idea was to _completely_ eliminate the GC. It says in Andre's 2015H1 vision statement: "We aim to make the standard library usable in its entirety without a garbage collector."
No, the plan is to allow the user to choose how he wants to allocate. Many pieces of phobos make the assumption that the GC is fair game. I think the plan is to make those pieces instead allocate on the stack and provide a mechanism to move that allocation to the GC (Walter's Dconf talk was about this), or accept an allocator to use as a mechanism for allocating memory. -Steve
Oct 05 2015
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, September 29, 2015 22:38:42 Johannes Pfau via Digitalmars-d-learn
wrote:
 Am Tue, 29 Sep 2015 15:10:58 -0400
 schrieb Steven Schveighoffer <schveiguy yahoo.com>:

 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just
 pass an "Object"?
An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object. I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done.
But you'll need access to the Mutex in user code as well. And often you use multiple Conditions with one Mutex so a Condition doesn't really own the Mutex.
 4) Will D's Condition ever experience spurious wakeups?
What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied. In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock.
Spurious wakeup is a common term when talking about posix conditions and it does indeed mean a wait() call can return without ever calling notify(): https://en.wikipedia.org/wiki/Spurious_wakeup http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups And yes, this does happen for core.sync.condition as well. As a result you'll always have to check in a loop: synchronized(mutex) { while(some_flag_or_expression) { cond.wait(); } } ----------------- synchronized(mutex) { some_flag_or_expression = true; cond.notify(); }
What I took from the answers to that SO question was that in general, it really doesn't matter whether a condition variable has spurious wakeups. You're going to have to check that the associated bool is true when you wake up anyway. Maybe without spurious wakeups, it wouldn't be required if only one thread was waiting for the signal, but you'd almost certainly still need an associated bool in case it becomes true prior to waiting. In addition, if you want to avoid locking up your program, it's ferquently the case that you want a timed wait so that you can check whether the program is trying to exit (or at least that the thread in question is being terminated), and you'd need a separate bool in that case as well so that you can check whether the condition has actually been signaled. So, ultimately, while spurious wakeups do seem wrong from a correctness perspective, when you look at what a condition variable needs to do, it usually doesn't matter that spurious wakeups exist, and a correctly used condition variable will just handle spurious wakeups as a side effect of how it's used. - Jonathan M Davis
Sep 30 2015
parent bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 30 September 2015 at 10:32:01 UTC, Jonathan M Davis 
wrote:
 On Tuesday, September 29, 2015 22:38:42 Johannes Pfau via 
 Digitalmars-d-learn wrote:
 [...]
What I took from the answers to that SO question was that in general, it really doesn't matter whether a condition variable has spurious wakeups. You're going to have to check that the associated bool is true when you wake up anyway. Maybe without spurious wakeups, it wouldn't be required if only one thread was waiting for the signal, but you'd almost certainly still need an associated bool in case it becomes true prior to waiting. In addition, if you want to avoid locking up your program, it's ferquently the case that you want a timed wait so that you can check whether the program is trying to exit (or at least that the thread in question is being terminated), and you'd need a separate bool in that case as well so that you can check whether the condition has actually been signaled. So, ultimately, while spurious wakeups do seem wrong from a correctness perspective, when you look at what a condition variable needs to do, it usually doesn't matter that spurious wakeups exist, and a correctly used condition variable will just handle spurious wakeups as a side effect of how it's used. - Jonathan M Davis
Yea, I guess you're right. The class in the example I posted was a crude reproduction of something I'm using right now in another project: http://codepad.org/M4fVyiXf I don't think it would make a difference whether it woke up randomly or not. I've been using this code regularly with no problems. Bit
Oct 04 2015
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 29 September 2015 at 19:10:58 UTC, Steven 
Schveighoffer wrote:
 An object that implements the Monitor interface may not 
 actually be a mutex. For example, a pthread_cond_t requires a 
 pthread_mutex_t to operate properly.
Right! I feel like I should have caught the fact that ConditionVariable still has to use pthread_cond_t under the hood, and adopts all of it's behaviour and requirements as a result.
 4. Technically, you shouldn't access member variables that are 
 GC allocated from a dtor. I know it's a struct, but structs can 
 be GC allocated as well.
Right.... forgot about that. GC's are really beginning to get on my nerves.. IMO, RAII for GC is a horrible tradeoff. I'm still not sure I would like Rust, but their memory model is making it a very enticing proposition. I'm almost at the point where I just don't care how much convenience, or familiarity D can offer in other areas.. Its starting to seem like none of it is worth it with a GC-based memory model standing in the way. Maybe this is an exageration...D has a lot of great features..but it's the net benefit that will ultimately determine whether or not people use D. I use C#(garbage collected) for making apps/games, and while, _in_theory_, the GC is supposed to protect you from leaks, memory is not the only thing that can leak. Threads need to be stopped, graphics resources need to be released, etc.. So when I can't rely on RAII to free these things, I need to free them explicitly, which basically puts me right back where I started. Anyways, I realize this will probably be buried 3 pages deep in D-Learn by Monday, but at least I feel better :) Bit
Oct 03 2015
parent reply Kagamin <spam here.lot> writes:
On Sunday, 4 October 2015 at 04:24:55 UTC, bitwise wrote:
 I use C#(garbage collected) for making apps/games, and while, 
 _in_theory_, the GC is supposed to protect you from leaks, 
 memory is not the only thing that can leak. Threads need to be 
 stopped, graphics resources need to be released, etc.
XNA doesn't manage graphics resources? On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote:
 I'm not sure what's going to be done with shared, but I do 
 think it's annoying that you can't do this:

 shared Array!int numbers;

 someThread... {
     numbers.clear(); // 'clear' is not shared
 }

 So this means that on top of the already ridiculous number of 
 attributes D has, now you have to mark everything as shared too 
 =/
That's illegal in other languages too except that they allow you to do it. If you want concurrent collections, you must code them separately: https://msdn.microsoft.com/en-us/library/system.collections.concurrent%28v=vs.110%29.aspx
Oct 07 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Wednesday, 7 October 2015 at 09:09:36 UTC, Kagamin wrote:
 On Sunday, 4 October 2015 at 04:24:55 UTC, bitwise wrote:
 I use C#(garbage collected) for making apps/games, and while, 
 _in_theory_, the GC is supposed to protect you from leaks, 
 memory is not the only thing that can leak. Threads need to be 
 stopped, graphics resources need to be released, etc.
XNA doesn't manage graphics resources? On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote:
 I'm not sure what's going to be done with shared, but I do 
 think it's annoying that you can't do this:

 shared Array!int numbers;

 someThread... {
     numbers.clear(); // 'clear' is not shared
 }

 So this means that on top of the already ridiculous number of 
 attributes D has, now you have to mark everything as shared 
 too =/
That's illegal in other languages too except that they allow you to do it. If you want concurrent collections, you must code them separately: https://msdn.microsoft.com/en-us/library/system.collections.concurrent%28v=vs.110%29.aspx
I'm not sure what you mean by illegal. AFAIK 'shared' is unique to D. As far as simply locking and then accessing a global variable(class static member) in C#, there is no problem doing that from multiple threads. If you have System.Collections.Generic.List(T) static class member, there is nothing wrong with using it from multiple threads like this: class Foo { static List<int> numbers = new List<int>(); void bar() { new Thread(()=>{ lock(numbers) { numbers.Add(1); }).Start(); } } Bit
Oct 07 2015
parent reply Kagamin <spam here.lot> writes:
On Thursday, 8 October 2015 at 02:31:24 UTC, bitwise wrote:
 If you have System.Collections.Generic.List(T) static class 
 member, there is nothing wrong with using it from multiple 
 threads like this:
The equivalent of your D example would be class Foo { static List<int> numbers = new List<int>(); void bar() { new Thread(()=>{ numbers.Add(1); }).Start(); } }
Oct 08 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 8 October 2015 at 10:11:38 UTC, Kagamin wrote:
 On Thursday, 8 October 2015 at 02:31:24 UTC, bitwise wrote:
 If you have System.Collections.Generic.List(T) static class 
 member, there is nothing wrong with using it from multiple 
 threads like this:
The equivalent of your D example would be class Foo { static List<int> numbers = new List<int>(); void bar() { new Thread(()=>{ numbers.Add(1); }).Start(); } }
That still doesn't explain what you mean about it being illegal in other languages or why you brought up C# in the first place. Bit
Oct 08 2015
parent reply Kagamin <spam here.lot> writes:
On Thursday, 8 October 2015 at 13:44:46 UTC, bitwise wrote:
 That still doesn't explain what you mean about it being illegal 
 in other languages or why you brought up C# in the first place.
Illegal means the resulting program behaves incorrectly, potentially leading to silent failures and data corruption. C# is a language that allows such bugs, and D disallows them - treats such code as invalid and rejects.
Oct 08 2015
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Thursday, 8 October 2015 at 20:42:46 UTC, Kagamin wrote:
 On Thursday, 8 October 2015 at 13:44:46 UTC, bitwise wrote:
 That still doesn't explain what you mean about it being 
 illegal in other languages or why you brought up C# in the 
 first place.
Illegal means the resulting program behaves incorrectly, potentially leading to silent failures and data corruption. C# is a language that allows such bugs, and D disallows them - treats such code as invalid and rejects.
Ah, I see. I thought you meant illegal meant it won't compile. Wouldn't it be more correct to say that it's undefined behaviour? Bit
Oct 08 2015
parent Kagamin <spam here.lot> writes:
On Friday, 9 October 2015 at 04:04:42 UTC, bitwise wrote:
 Ah, I see. I thought you meant illegal meant it won't compile. 
 Wouldn't it be more correct to say that it's undefined 
 behaviour?
I's probably not as undefined as in C case, i.e. it doesn't break safety guarantees, only the application's high-level business logic gets confused by data races.
Oct 09 2015
prev sibling parent Kagamin <spam here.lot> writes:
On Friday, 25 September 2015 at 15:19:27 UTC, bitwise wrote:
 I know that all global variables are TLS unless explicitly 
 marked as 'shared', but someone once told me something about 
 'shared' affecting member variables in that accessing them from 
 a separate thread would return T.init instead of the actual 
 value... or something like that. This seems to be 
 wrong(thankfully).
T.init is returned for TLS variable when accessed from a thread for which it wasn't initialized.
Oct 01 2015