www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is there any threadsafe queue?

reply John Burton <john.burton jbmail.com> writes:
Is there any threadsafe queue in the standard library?
I've not been able to find anything but thought I'd check before 
making my own.

I want to be able to assemble messages (Which are just streams of 
bytes) in one thread into a struct and push them to the queue, 
and then have a another thread be able to read and process the 
messages. Single producer and consumer.
Sep 13 2017
next sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, September 13, 2017 07:51:19 John Burton via Digitalmars-d-
learn wrote:
 Is there any threadsafe queue in the standard library?
 I've not been able to find anything but thought I'd check before
 making my own.

 I want to be able to assemble messages (Which are just streams of
 bytes) in one thread into a struct and push them to the queue,
 and then have a another thread be able to read and process the
 messages. Single producer and consumer.
You could probably do what you want with std.concurrency, since most of what it does is deal with sending and receiving data across threads, and that should be queuing up messages as part of what it does. But if you're looking for a data structure that you can mark as shared and have it handle all of the locking for you so that it will work safely across threads, then no, the standard library does not currently have anything like that. It's rather lacking in containers in general at this point, let alone ones designed with concurrency in mind. There may be something on code.dlang.org, but I don't know. Regardless, I'd suggest that you first try and see if you can get std.concurrency to work for your use case rather than jumping into implementing any containers yourself. - Jonathan M Davis
Sep 13 2017
parent John Burton <john.burton jbmail.com> writes:
On Wednesday, 13 September 2017 at 09:49:46 UTC, Jonathan M Davis 
wrote:
 On Wednesday, September 13, 2017 07:51:19 John Burton via 
 Digitalmars-d- learn wrote:
 [...]
You could probably do what you want with std.concurrency, since most of what it does is deal with sending and receiving data across threads, and that should be queuing up messages as part of what it does. But if you're looking for a data structure that you can mark as shared and have it handle all of the locking for you so that it will work safely across threads, then no, the standard library does not currently have anything like that. It's rather lacking in containers in general at this point, let alone ones designed with concurrency in mind. There may be something on code.dlang.org, but I don't know. Regardless, I'd suggest that you first try and see if you can get std.concurrency to work for your use case rather than jumping into implementing any containers yourself.
Thanks, I took a better look. I was wanting to port some c++ code which is why I was looking for this. Actually it looks like "send" and "receive" functions from this package do exactly what I need (and are in fact exactly a thread safe queue...) :)
Sep 13 2017
prev sibling next sibling parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 13 September 2017 at 07:51:19 UTC, John Burton 
wrote:
 Is there any threadsafe queue in the standard library?
 I've not been able to find anything but thought I'd check 
 before making my own.

 I want to be able to assemble messages (Which are just streams 
 of bytes) in one thread into a struct and push them to the 
 queue, and then have a another thread be able to read and 
 process the messages. Single producer and consumer.
Not sure what you are looking for, but a really cool topic are lock-free data structures. They are inherently thread-safe. One popular lib in D: https://code.dlang.org/packages/lock-free
Sep 13 2017
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 13 September 2017 at 22:54:06 UTC, Seb wrote:
 https://code.dlang.org/packages/lock-free
That package has a single-reader single-writer queue, exactly what the OP asked for.
Sep 14 2017
prev sibling parent Bienlein <jeti789 web.de> writes:
On Wednesday, 13 September 2017 at 07:51:19 UTC, John Burton 
wrote:
 Is there any threadsafe queue in the standard library?
 I've not been able to find anything but thought I'd check 
 before making my own.

 I want to be able to assemble messages (Which are just streams 
 of bytes) in one thread into a struct and push them to the 
 queue, and then have a another thread be able to read and 
 process the messages. Single producer and consumer.
You can take some existing container and overwrite the add and remove methods to be synchronized, e.g. public void add(T t) { synchronized { super.add(t); } } However, this could cause some lock contention depending on your use case. This is why class Vector in Java is basically discontinued. In that class every method is synchronized which has led to bad timing behavior. For the new concurrent collections in Java since Java 5 some work has been done to replace synchronized with some CAS approach. For example, class ConcurrentLinkedQueue in Java does some tricks with CAS algorithms to get around this.
Sep 14 2017