www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Does D programming language have work steal queue?

reply zoujiaqing <zoujiaqing gmail.com> writes:
Does D language have task steal queue?
The requirements are high-performance, lock-free, and thread-safe.
May 22 2022
next sibling parent reply zoujiaqing <zoujiaqing gmail.com> writes:
On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


I think the final usage scenario is similar to this C++ project:
https://github.com/ConorWilliams/Forkpool
May 22 2022
parent zoujiaqing <zoujiaqing gmail.com> writes:
On Sunday, 22 May 2022 at 21:23:24 UTC, zoujiaqing wrote:

 On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


 I think the final usage scenario is similar to this C++ project:
 https://github.com/ConorWilliams/Forkpool


This project implements many of the ideas in (available in 
reference/):

  * F. Schmaus et al., “Nowa: A Wait-Free Continuation-Stealing 
Concurrency Platform”. In: 2021 IEEE International Parallel and 
Distributed Processing Symposium (IPDPS). 2021.
  * C. -X. Lin, T. -W. Huang and M. D. F. Wong, "An Efficient 
Work-Stealing Scheduler for Task Dependency Graph," 2020 IEEE 
26th International Conference on Parallel and Distributed Systems 
(ICPADS), 2020, pp. 64-71, doi: 10.1109/ICPADS51040.2020.00018.
May 22 2022
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


I have one called fluffy:
https://github.com/UplinkCoder/fluffy

I am not 100% sure about the performance I did try to make it 
reasonable but in the absence of anything else it might be 
jumping off point for you.
May 22 2022
next sibling parent reply mw <mingwu gmail.com> writes:
On Sunday, 22 May 2022 at 22:37:43 UTC, Stefan Koch wrote:

 On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


 I have one called fluffy:
 https://github.com/UplinkCoder/fluffy



The git link 404?


 I am not 100% sure about the performance I did try to make it 
 reasonable but in the absence of anything else it might be 
 jumping off point for you.
May 22 2022
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 22 May 2022 at 23:35:11 UTC, mw wrote:

 On Sunday, 22 May 2022 at 22:37:43 UTC, Stefan Koch wrote:

 On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


 I have one called fluffy:
 https://github.com/UplinkCoder/fluffy



 The git link 404?


 I am not 100% sure about the performance I did try to make it 
 reasonable but in the absence of anything else it might be 
 jumping off point for you.




Looks like I had the repo set to private.
It should work now.
May 22 2022
prev sibling parent zoujiaqing <zoujiaqing gmail.com> writes:
On Sunday, 22 May 2022 at 22:37:43 UTC, Stefan Koch wrote:

 On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


 I have one called fluffy:
 https://github.com/UplinkCoder/fluffy

 I am not 100% sure about the performance I did try to make it 
 reasonable but in the absence of anything else it might be 
 jumping off point for you.


Thanks ;)
May 23 2022
prev sibling parent reply mw <mingwu gmail.com> writes:
On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


I have a C's liblfds D wrapper:

https://github.com/mw66/liblfdsd

right now only bmm and bss queue are wrapped.

It's not in dub yet, but I have been using it for a while, feel 
free to give it a try, or even send PRs :-)
May 22 2022
parent reply zoujiaqing <zoujiaqing gmail.com> writes:
On Sunday, 22 May 2022 at 23:34:19 UTC, mw wrote:

 On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


 I have a C's liblfds D wrapper:

 https://github.com/mw66/liblfdsd

 right now only bmm and bss queue are wrapped.

 It's not in dub yet, but I have been using it for a while, feel 
 free to give it a try, or even send PRs :-)


Thanks :)
I will try it.
May 23 2022
next sibling parent mw <mingwu gmail.com> writes:
On Monday, 23 May 2022 at 23:07:00 UTC, zoujiaqing wrote:

 On Sunday, 22 May 2022 at 23:34:19 UTC, mw wrote:

 On Sunday, 22 May 2022 at 21:07:19 UTC, zoujiaqing wrote:

 Does D language have task steal queue?
 The requirements are high-performance, lock-free, and 
 thread-safe.


 I have a C's liblfds D wrapper:

 https://github.com/mw66/liblfdsd

 right now only bmm and bss queue are wrapped.

 It's not in dub yet, but I have been using it for a while, 
 feel free to give it a try, or even send PRs :-)





BTW, some very simple performance comparison with other D queues 
are here:

https://github.com/mw66/liblfdsd/tree/master/comparison
May 23 2022
prev sibling parent mw <mingwu gmail.com> writes:
 I will try it.


It's in dub now:

https://code.dlang.org/packages/liblfdsd


Also added queue_umm: unbounded,manyproducer,many_consumer, 
lock-free queue
Jun 04 2022