www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Oh, my GoD! Goroutines on D

reply Jin <nin-jin ya.ru> writes:
DUB module: http://code.dlang.org/packages/jin-go
GIT repo: https://github.com/nin-jin/go.d

Function "go" starts coroutines (vibe.d tasks) in thread pool 
like in Go language:

	unittest
	{
		import core.time;
		import jin.go;

		__gshared static string[] log;

		static void saying( string message )
		{
			for( int i = 0 ; i < 3 ; ++i ) {
				sleep( 100.msecs );
				log ~= message;
			}
		}

		go!saying( "hello" );
		sleep( 50.msecs );
		saying( "world" );

		log.assertEq([ "hello" , "world" , "hello" , "world" , "hello" 
, "world" ]);
	}

Class "Channel" is wait-free one-consumer-one-provider channel. 
By default, Channel blocks thread on receive from clear channel 
or send to full channel:

unittest
{
	static void summing( Channel!int output ) {
		foreach( i ; ( output.size * 2 ).iota ) {
			output.next = 1;
		}
		output.close();
	}

	auto input = go!summing;
	while( !input.full ) yield;

	input.sum.assertEq( input.size * 2 );
}

You can no wait if you do not want:

	unittest
	{
		import core.time;
		import jin.go;

		static auto after( Channel!bool channel , Duration dur )
		{
			sleep( dur );
			if( !channel.closed ) channel.next = true;
		}

		static auto tick( Channel!bool channel , Duration dur )
		{
			while( !channel.closed ) after( channel , dur );
		}

		auto ticks = go!tick( 101.msecs );
		auto booms = go!after( 501.msecs );

		string log;

		while( booms.clear )
		{
			while( !ticks.clear ) {
				log ~= "tick";
				ticks.popFront;
			}
			log ~= ".";
			sleep( 51.msecs );
		}
		log ~= "BOOM!";

		log.assertEq( "..tick..tick..tick..tick..BOOM!" );
	}

Channel are InputRange and OutputRange compatible. Structs 
"Inputs" and "Outputs" are round-robin facade to array of 
channels.

More examples in unit tests: 
https://github.com/nin-jin/go.d/blob/master/source/jin/go.d#L293

Current problems:

1. You can give channel to more than two thread. I'm going to 
play with unique pointers to solve this problem. Any hints?

2. Sometimes you must close channel to notify partner to break 
range cycle. Solving (1) problem can solve and this.

3. API may be better. Advices?
Mar 27 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 27 March 2016 at 18:17:55 UTC, Jin wrote:
 1. You can give channel to more than two thread. I'm going to 
 play with unique pointers to solve this problem. Any hints?
Note that this is also the case in go. Yes, contrary to what is usually said, go is dead usnafe when it come to threads.
Mar 27 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/27/2016 11:17 AM, Jin wrote:
 [...]
Nice! Please write an article about this!
Mar 27 2016
parent reply Jin <nin-jin ya.ru> writes:
On Sunday, 27 March 2016 at 20:39:57 UTC, Walter Bright wrote:
 On 3/27/2016 11:17 AM, Jin wrote:
 [...]
Nice! Please write an article about this!
My english is too bad to write articles, sorry :-(
Mar 28 2016
next sibling parent Lass Safin <lasssafin gmail.com> writes:
On Monday, 28 March 2016 at 13:10:45 UTC, Jin wrote:
 On Sunday, 27 March 2016 at 20:39:57 UTC, Walter Bright wrote:
 On 3/27/2016 11:17 AM, Jin wrote:
 [...]
Nice! Please write an article about this!
My english is too bad to write articles, sorry :-(
Just use engrish, we won't care. Really.
Mar 28 2016
prev sibling next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2016-03-28 at 13:10 +0000, Jin via Digitalmars-d wrote:
=C2=A0
 My english is too bad to write articles, sorry :-(
Write the article content and then get someone who is a person good at writing in English to ghostwrite the article from your content. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Mar 28 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/28/2016 6:10 AM, Jin wrote:
 My english is too bad to write articles, sorry :-(
Baloney, your english is very good. Besides, I'm sure there will be many volunteers here to help you touch it up.
Mar 28 2016
parent reply Jin <nin-jin ya.ru> writes:
On Monday, 28 March 2016 at 19:29:55 UTC, Walter Bright wrote:
 On 3/28/2016 6:10 AM, Jin wrote:
 My english is too bad to write articles, sorry :-(
Baloney, your english is very good. Besides, I'm sure there will be many volunteers here to help you touch it up.
I just wrote the article on russin: https://habrahabr.ru/post/280378/ Translation to english: https://translate.google.com/translate?hl=ru&sl=ru&tl=en&u=https%3A%2F%2Fhabrahabr.ru%2Fpost%2F280378%2F
Mar 28 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/28/2016 3:35 PM, Jin wrote:
 On Monday, 28 March 2016 at 19:29:55 UTC, Walter Bright wrote:
 On 3/28/2016 6:10 AM, Jin wrote:
 My english is too bad to write articles, sorry :-(
Baloney, your english is very good. Besides, I'm sure there will be many volunteers here to help you touch it up.
I just wrote the article on russin: https://habrahabr.ru/post/280378/ Translation to english: https://translate.google.com/translate?hl=ru&sl=ru&tl=en&u=https%3A%2F%2Fhabrahabr.ru%2Fpost%2F280378%2F
Awesome! Who wants to help with the English?
Mar 28 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 3/28/2016 5:07 PM, Walter Bright wrote:
 On 3/28/2016 3:35 PM, Jin wrote:
 On Monday, 28 March 2016 at 19:29:55 UTC, Walter Bright wrote:
 On 3/28/2016 6:10 AM, Jin wrote:
 My english is too bad to write articles, sorry :-(
Baloney, your english is very good. Besides, I'm sure there will be many volunteers here to help you touch it up.
I just wrote the article on russin: https://habrahabr.ru/post/280378/ Translation to english: https://translate.google.com/translate?hl=ru&sl=ru&tl=en&u=https%3A%2F%2Fhabrahabr.ru%2Fpost%2F280378%2F
Awesome! Who wants to help with the English?
I'll start with the first paragraph (I don't know any Russian): Google: Multitasking - this is what is implemented in Go of the good, though not perfect. Nice syntax with a tart aftertaste, simple and powerful abstraction, bribe its elegance compared to other imperative languages. And taste the best, so do not want to have to slide to mediocrity. Therefore, if and switch to another language, it must be even more expressive and with no less sensible multitasking. English: Multitasking - Go's multitasking capabilities are good, though not perfect. It has a nice syntax with a sweet aftertaste, simple and powerful abstraction, elegant compared to other imperative languages. It exhibits good taste, so one does not wish to compromise into mediocrity. Therefore, to switch to another language, it must be even more expressive and with no less sensible multitasking.
Mar 28 2016
prev sibling parent sigod <sigod.mail gmail.com> writes:
On Monday, 28 March 2016 at 22:35:12 UTC, Jin wrote:
 On Monday, 28 March 2016 at 19:29:55 UTC, Walter Bright wrote:
 On 3/28/2016 6:10 AM, Jin wrote:
 My english is too bad to write articles, sorry :-(
Baloney, your english is very good. Besides, I'm sure there will be many volunteers here to help you touch it up.
I just wrote the article on russin: https://habrahabr.ru/post/280378/ Translation to english: https://translate.google.com/translate?hl=ru&sl=ru&tl=en&u=https%3A%2F%2Fhabrahabr.ru%2Fpost%2F280378%2F
Create repository on GitHub. So, it will be easier for others to help with translation.
Mar 29 2016
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-03-27 20:17, Jin wrote:
 DUB module: http://code.dlang.org/packages/jin-go
 GIT repo: https://github.com/nin-jin/go.d

 Function "go" starts coroutines (vibe.d tasks) in thread pool like in Go
 language:
It would be useful with a wiki page, or similar, that describes and compares different ways of doing concurrency in D, both built-in and third party libraries like this and vibe.d. Also compare against other languages like Go, Erlang, Scala and so on. -- /Jacob Carlborg
Mar 28 2016
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 03/28/2016 03:49 AM, Jacob Carlborg wrote:
 On 2016-03-27 20:17, Jin wrote:
 DUB module: http://code.dlang.org/packages/jin-go
 GIT repo: https://github.com/nin-jin/go.d

 Function "go" starts coroutines (vibe.d tasks) in thread pool like in Go
 language:
It would be useful with a wiki page, or similar, that describes and compares different ways of doing concurrency in D, both built-in and third party libraries like this and vibe.d. Also compare against other languages like Go, Erlang, Scala and so on.
And make sure to tell me about it so that my DConf presentation will be more complete: :) http://dconf.org/2016/talks/cehreli.html That abstract is awfully short but I've posted a pull request to improve it a little bit: <quote> D provides support for multitasking in the form of language features and standard library modules. D makes it easy for your programs to perform multiple tasks at the same time. This kind of support is especially important in order to take advantage of the multiple CPU cores that are available on modern computing systems. Multitasking is one of the most difficult computing concepts to implement correctly. This talk will introduce different kinds of multitasking, as well as parallelism, a concept which is in fact unrelated to, but is often confused with, multitasking. The talk will conclude with fibers (aka co-routines), a powerful tool that is often overlooked despite its convenience. </quote> Seriously, I appreciate any documentation links that you can give to complete my "homework" before DConf. :) Ali
Mar 28 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-03-29 01:53, Ali Çehreli wrote:

 Seriously, I appreciate any documentation links that you can give to
 complete my "homework" before DConf. :)
I was hoping someone could give _me_ the links, that's why I wrote the post ;) -- /Jacob Carlborg
Mar 28 2016
prev sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Monday, 28 March 2016 at 10:49:28 UTC, Jacob Carlborg wrote:
 It would be useful with a wiki page, or similar, that describes 
 and compares different ways of doing concurrency in D, both 
 built-in and third party libraries like this and vibe.d. Also 
 compare against other languages like Go, Erlang, Scala and so 
 on.
+1 Wiki is absolutely the best solution to this, I agree. Plus, we already have a wiki so he should just go there and start writing. The community will incorrect grammar/syntax and typos.
Mar 29 2016
parent reply Jin <nin-jin ya.ru> writes:
On Tuesday, 29 March 2016 at 12:30:24 UTC, Dejan Lekic wrote:
 +1
 Wiki is absolutely the best solution to this, I agree. Plus, we 
 already have a wiki so he should just go there and start 
 writing. The community will incorrect grammar/syntax and typos.
http://wiki.dlang.org/Go_to_D
Mar 29 2016
parent H. S. Teoh <hsteoh quickfur.ath.cx> writes:
On Tuesday, 29 March 2016 at 17:10:02 UTC, Jin wrote:
 On Tuesday, 29 March 2016 at 12:30:24 UTC, Dejan Lekic wrote:
 +1
 Wiki is absolutely the best solution to this, I agree. Plus, 
 we already have a wiki so he should just go there and start 
 writing. The community will incorrect grammar/syntax and typos.
http://wiki.dlang.org/Go_to_D
Since I know a bit of Russian, I took a shot at improving this article, and got partway through the "Channels" section. But now I need to get back to work... so hopefully somebody else can work on improving the English text. :-) --T
Mar 29 2016
prev sibling parent reply Casey Sybrandy <sybrandy gmail.com> writes:
On Sunday, 27 March 2016 at 18:17:55 UTC, Jin wrote:
 DUB module: http://code.dlang.org/packages/jin-go
 GIT repo: https://github.com/nin-jin/go.d

 [...]
Have you considered using a Disrupter (http://lmax-exchange.github.io/disruptor/) for the channels? Not sure how it compares to what you're using from Vibe.d, but it's not a hard data structure to implement and, IIRC, it allows for multiple producers and consumers.
Mar 30 2016
parent reply Casey Sybrandy <sybrandy gmail.com> writes:
On Wednesday, 30 March 2016 at 14:28:50 UTC, Casey Sybrandy wrote:
 On Sunday, 27 March 2016 at 18:17:55 UTC, Jin wrote:
 DUB module: http://code.dlang.org/packages/jin-go
 GIT repo: https://github.com/nin-jin/go.d

 [...]
Have you considered using a Disrupter (http://lmax-exchange.github.io/disruptor/) for the channels? Not sure how it compares to what you're using from Vibe.d, but it's not a hard data structure to implement and, IIRC, it allows for multiple producers and consumers.
Oh, and yes, I know that it would have to be rewritten in D unless there's a C version somewhere. I actually did it once and it wasn't too bad. I don't think I have a copy anymore, but if I do find it, I can put it up somewhere.
Mar 30 2016
parent reply Jin <nin-jin ya.ru> writes:
On Wednesday, 30 March 2016 at 15:22:26 UTC, Casey Sybrandy wrote:
 Have you considered using a Disrupter 
 (http://lmax-exchange.github.io/disruptor/) for the channels?  
 Not sure how it compares to what you're using from Vibe.d, but 
 it's not a hard data structure to implement and, IIRC, it 
 allows for multiple producers and consumers.
Oh, and yes, I know that it would have to be rewritten in D unless there's a C version somewhere. I actually did it once and it wasn't too bad. I don't think I have a copy anymore, but if I do find it, I can put it up somewhere.
This is java bloatware. :-(
Mar 30 2016
next sibling parent reply Casey Sybrandy <sybrandy gmail.com> writes:
On Wednesday, 30 March 2016 at 15:50:47 UTC, Jin wrote:
 This is java bloatware. :-(
I've never used the library so I can't comment on that, but the actual data structure/algorithm is really pretty simple. The core components are atomic counters and a static array. I think it would be a good data structure for channels.
Mar 30 2016
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2016-03-30 at 17:01 +0000, Casey Sybrandy via Digitalmars-d
wrote:
 On Wednesday, 30 March 2016 at 15:50:47 UTC, Jin wrote:
=20
 This is java bloatware. :-(
I've never used the library so I can't comment on that, but the=C2=A0 actual data structure/algorithm is really pretty simple.=C2=A0=C2=A0The=
=C2=A0
 core components are atomic counters and a static array.=C2=A0=C2=A0I thin=
k=C2=A0
 it would be a good data structure for channels.
If I recollect correctly, the core data structure is a lock-free ring buffer, and the parallelism "trick" the use of multicast with atomic indexes. This works fine for the problem of creating a trading framework, but I suspect the architecture is just too big for realizing channels. In particular, the really important thing about channels over (thread|process)-safe queues is the ability to select. I have no idea how select is implemented on Windows but the classic Posix approach is to use file descriptors to represent the queues and select or epoll system calls to get the kernel to realize the select. As to how JCSP does select on the JVM, I shall have to go and delve into the source code=E2=80=A6 =C2=A0 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 01 2016
prev sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2016-03-30 at 15:50 +0000, Jin via Digitalmars-d wrote:
 On Wednesday, 30 March 2016 at 15:22:26 UTC, Casey Sybrandy wrote:
=20
=20
 Have you considered using a Disrupter=C2=A0
 (http://lmax-exchange.github.io/disruptor/) for the channels?=C2=A0=
=C2=A0
 Not sure how it compares to what you're using from Vibe.d, but=C2=A0
 it's not a hard data structure to implement and, IIRC, it=C2=A0
 allows for multiple producers and consumers.
Oh, and yes, I know that it would have to be rewritten in D=C2=A0 unless there's a C version somewhere.=C2=A0=C2=A0I actually did it once=
=C2=A0
 and it wasn't too bad.=C2=A0=C2=A0I don't think I have a copy anymore,=
=C2=A0
 but if I do find it, I can put it up somewhere.
This is java bloatware. :-(
I think that is probably just slander. Whilst there are some known problems with The Disruptor, blithely labelling it "java bloatware" is most likely an uneducated, and probably ill-judged, comment. =C2=A0=C2=A0 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.n= et 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 01 2016