www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [Semi-OT] Fibers vs. Async / Await

reply =?UTF-8?B?UmVuw6k=?= Zwanenburg <renezwanenburg gmail.com> writes:
The suggestion of adding async / await to the language comes up 
with some frequency, with fibers being seen as a lackluster 
alternative.

Now, I feel like I must be missing something obvious, as fibers 
seem like the better option in almost every aspect to me. I'll 
compare them to async / await in .Net since that's the 
implementation I'm most familiar with.

The issues I have with a/a:
1. When a function wants to do something asynchronous every 
function up the call stack needs to be in on it. This is 
especially problematic with generic code and higher-order 
functions. Even with D's powerful metaprogramming facilities I 
don't see a way to handle this nicely. As an example: the whole 
IEnumerable / IAsyncEnumerable situation in .Net is a 
manifestation of this problem.

2. It messes up your callstack. This leads to problems with:
- Stack traces
- Debugging
- Memory dumps
- Sampling profilers
- And probably more that I haven't run into yet.

Now, some tooling can kinda sorta reconstruct a usable stack, but 
all the efforts I've seen so far are still harder to use than 
when you have a proper stack.

3. I don't like the Task\<T> and await noise everywhere. It 
clutters up the code.

4. It's too easy to make a mistake and forget an await somewhere. 
Yes, an IDE will likely give a warning. No, I still don't like it 
;)

5. It can put too much pressure on the GC. This won't be a 
problem when doing an HTTP request or something like that. It is 
a problem when there's an allocation for reading every single 
value in a DB query result.

All the above problems don't exist when using fibers. The only 
downside of fibers I'm aware of is that you have a full stack 
allocated for every fiber. This can cost a lot of (virtual) 
memory, but on the other hand it makes for very effective 
pooling. And if you're in a situation where this really is a 
problem you can tweak the stack size. Seems like a small price to 
pay to me.

Am I missing something? Thank you for your time.
May 11 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 12/05/2022 1:27 AM, René Zwanenburg wrote:

 All the above problems don't exist when using fibers. The only downside 
 of fibers I'm aware of is that you have a full stack allocated for every 
 fiber. This can cost a lot of (virtual) memory, but on the other hand it 
 makes for very effective pooling. And if you're in a situation where 
 this really is a problem you can tweak the stack size. Seems like a 
 small price to pay to me.


Don't forget the extra cost of having to scan those stacks regardless of 
those fibers state.

The context state required for a task will be a lot smaller (and hence 
cheaper) than what a single fiber stack will cost to scan.
May 11 2022
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/11/22 06:49, rikki cattermole wrote:


 The context state required for a task will be a lot smaller (and hence
 cheaper) than what a single fiber stack will cost to scan.


While were on it, what are the blockers for stackless fibers for D?

Ali
May 11 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 12/05/2022 8:15 AM, Ali Çehreli wrote:

 On 5/11/22 06:49, rikki cattermole wrote:
 
  > The context state required for a task will be a lot smaller (and hence
  > cheaper) than what a single fiber stack will cost to scan.
 
 While were on it, what are the blockers for stackless fibers for D?
 
 Ali
 


I don't know.

Whatever solution we come up with, it must support yielding in say a 
database driver, while an ORM returns say a Future and have the event 
loop automatically complete it.
May 11 2022
prev sibling parent reply =?UTF-8?B?UmVuw6k=?= Zwanenburg <renezwanenburg gmail.com> writes:
On Wednesday, 11 May 2022 at 13:49:28 UTC, rikki cattermole wrote:

 Don't forget the extra cost of having to scan those stacks 
 regardless of those fibers state.

 The context state required for a task will be a lot smaller 
 (and hence cheaper) than what a single fiber stack will cost to 
 scan.


That's a good point. As an improvement, could we mark the unused 
part of the fiber's stack as NO_SCAN when it yields? And ofc undo 
it when the fiber switches back in. Or would that be too heavy an 
operation?
May 12 2022
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 12/05/2022 11:37 PM, René Zwanenburg wrote:

 On Wednesday, 11 May 2022 at 13:49:28 UTC, rikki cattermole wrote:

 Don't forget the extra cost of having to scan those stacks regardless 
 of those fibers state.

 The context state required for a task will be a lot smaller (and hence 
 cheaper) than what a single fiber stack will cost to scan.


 
 That's a good point. As an improvement, could we mark the unused part of 
 the fiber's stack as NO_SCAN when it yields? And ofc undo it when the 
 fiber switches back in. Or would that be too heavy an operation?


I don't think it would help, and could potentially do the wrong thing.

GC's like all memory allocators like to work with blocks of memory, that 
could easily overshoot the bounds of unused that were specified.
May 12 2022
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/12/22 7:37 AM, René Zwanenburg wrote:

 On Wednesday, 11 May 2022 at 13:49:28 UTC, rikki cattermole wrote:

 Don't forget the extra cost of having to scan those stacks regardless 
 of those fibers state.

 The context state required for a task will be a lot smaller (and hence 
 cheaper) than what a single fiber stack will cost to scan.


 
 That's a good point. As an improvement, could we mark the unused part of 
 the fiber's stack as NO_SCAN when it yields? And ofc undo it when the 
 fiber switches back in. Or would that be too heavy an operation?


Fibers already do this.

https://github.com/dlang/druntime/blob/392c528924d0ce4db57a03bfdaa6587169568493/src/core/thread/fiber.d#L429-L436

-Steve
May 12 2022
prev sibling next sibling parent reply bauss <jj_1337 live.dk> writes:
On Wednesday, 11 May 2022 at 13:27:48 UTC, René Zwanenburg wrote:

 3. I don't like the Task\<T> and await noise everywhere. It 
 clutters up the code.


You don't have to have a "Task" type that you declare, it could 
simply be T and then the function is just marked async and T 
automatically becomes Task!T.


envisioned the ability to add your own Task types etc. but that 
has never been implemented or anything, so it's really just 
verbose.


 4. It's too easy to make a mistake and forget an await 
 somewhere. Yes, an IDE will likely give a warning. No, I still 
 don't like it ;)


It's easy to fix, don't allow implicit conversions between Task!T 
and T and thus whenever you attempt to use the result you're 
forced to await it because otherwise you'd get an error from the 
compiler due to mismatching types.

Like the below would definitely fail:

```
async int getNumberAsync();

...

auto number = getNumberAsync();

int otherNumber = 10 * number; // number cannot be used here due 
to it not being implicitly convertible to int. The fix would be 
to await getNumberAsync().
```


 5. It can put too much pressure on the GC. This won't be a 
 problem when doing an HTTP request or something like that. It 
 is a problem when there's an allocation for reading every 
 single value in a DB query result.


It doesn't really put anymore pressure on the GC. The compiler 
converts your code to a state machine and that hardly adds any 
overhead. It makes your functions somewhat linear, that's it. It 
doesn't require many more allocations, at least not more than 
fibers.
May 11 2022
parent reply =?UTF-8?B?UmVuw6k=?= Zwanenburg <renezwanenburg gmail.com> writes:
On Thursday, 12 May 2022 at 06:04:06 UTC, bauss wrote:

 You don't have to have a "Task" type that you declare, it could 
 simply be T and then the function is just marked async and T 
 automatically becomes Task!T.


That would help a little. Doesn't get rid of the await though.


 It's easy to fix, don't allow implicit conversions between 
 Task!T and T and thus whenever you attempt to use the result 
 you're forced to await it because otherwise you'd get an error 
 from the compiler due to mismatching types.


Right. .Net doesn't do implicit conversion either, I was thinking 
of functions that are void / just Task, like writing to a 
database. Assuming you use exceptions to report problems.


 It doesn't really put anymore pressure on the GC. The compiler 
 converts your code to a state machine and that hardly adds any 
 overhead. It makes your functions somewhat linear, that's it. 
 It doesn't require many more allocations, at least not more 
 than fibers.


Are you sure about this? The state machine needs to be stored 
somewhere. I'd think we would need something similar to a 
delegate: a fixed-size structure that can be passed around, with 
a pointer to a variable sized context / state machine living on 
the heap.
May 12 2022
parent bauss <jj_1337 live.dk> writes:
On Thursday, 12 May 2022 at 11:47:48 UTC, René Zwanenburg wrote:

 On Thursday, 12 May 2022 at 06:04:06 UTC, bauss wrote:

 You don't have to have a "Task" type that you declare, it 
 could simply be T and then the function is just marked async 
 and T automatically becomes Task!T.


 That would help a little. Doesn't get rid of the await though.


The await is necessary, just like yield is necessary for fibers. 
They function much the same. The reason why you can leave out the 
await is because you might want to pass on the task to somewhere 
else, you might have a list of tasks that you have to wait for, 
so you need to be able to control when to wait for a task to 
finish and when not to wait.


 It's easy to fix, don't allow implicit conversions between 
 Task!T and T and thus whenever you attempt to use the result 
 you're forced to await it because otherwise you'd get an error 
 from the compiler due to mismatching types.


 Right. .Net doesn't do implicit conversion either, I was 
 thinking of functions that are void / just Task, like writing 
 to a database. Assuming you use exceptions to report problems.



You can do it, but it's highly adviced against. There are only 
very few cases where you want to do it.

For just Task there is no type and no result to await, so you 
just await the execution.

I'm confused about why this is confusing to you and how you'd 
solve that?


 It doesn't really put anymore pressure on the GC. The compiler 
 converts your code to a state machine and that hardly adds any 
 overhead. It makes your functions somewhat linear, that's it. 
 It doesn't require many more allocations, at least not more 
 than fibers.


 Are you sure about this? The state machine needs to be stored 
 somewhere. I'd think we would need something similar to a 
 delegate: a fixed-size structure that can be passed around, 
 with a pointer to a variable sized context / state machine 
 living on the heap.


The state machine can be optimized away pretty much and all you 
have is the current state which is like a couple bytes maybe and 
just tells what part of the machine is currently executing. This 
is not much different from storing information about what task is 
currently executing in a fiber.

A statemachine is pretty much just a switch with a case for each 
state and then each step just changes to the next state.
May 13 2022
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Wednesday, 11 May 2022 at 13:27:48 UTC, René Zwanenburg wrote:

 Am I missing something? Thank you for your time.


Don't forget that fibers aren't supported on all platforms.

They are definitely interesting but I don't think they should be 
the basis of everything.
May 12 2022
next sibling parent reply IGotD- <nise nise.com> writes:
On Thursday, 12 May 2022 at 08:49:18 UTC, Sebastiaan Koppe wrote:

 Don't forget that fibers aren't supported on all platforms.

 They are definitely interesting but I don't think they should 
 be the basis of everything.


That was one of my observations when I looked in the druntime 
source code. The D runtime has basically implemented all of the 
scheduling itself, including all context switching. This means 
that it has to be implemented for all architectures, and variants 
of all architectures which is a lot. Many platforms like Windows 
has its own fiber API and I was surprised that druntime didn't 
use that one.

I'm not saying that it is wrong to do the entire implementation 
in druntime but it is a lot of work and only the most common CPU 
architectures will be supported. There is also the risk that it 
becomes outdated as CPU vendors add stuff to their CPUs.

I think since the D project has such limited resources, it should 
go for as generic solutions and reuse existing APIs. Async/Await 
is becoming accepted as a programming model and I think that D 
should put its effort to support that.
May 12 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 12/05/2022 9:16 PM, IGotD- wrote:

 I'm not saying that it is wrong to do the entire implementation in 
 druntime but it is a lot of work and only the most common CPU 
 architectures will be supported. There is also the risk that it becomes 
 outdated as CPU vendors add stuff to their CPUs.
 
 I think since the D project has such limited resources, it should go for 
 as generic solutions and reuse existing APIs. Async/Await is becoming 
 accepted as a programming model and I think that D should put its effort 
 to support that.


The cost to maintain our fiber implementation is very minimal.

Prior to the refactor 3 years ago, a whole pile of the context switch 
assembly was last touched like 8-13 years ago.

https://github.com/dlang/druntime/blame/3ead62a9bf4e0f866af10fdd3bc4edeb87237305/src/core/thread.d#L3754

ABI's are generally stable for this stuff. If they weren't a lot of 
userland would break and there would be no way to fix things.
May 12 2022
prev sibling parent reply =?UTF-8?B?UmVuw6k=?= Zwanenburg <renezwanenburg gmail.com> writes:
On Thursday, 12 May 2022 at 08:49:18 UTC, Sebastiaan Koppe wrote:

 Don't forget that fibers aren't supported on all platforms.


Didn't think of that. I only run D stuff on X86-64 at the moment 
but cross-platform support is definitely important.
May 12 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 12/05/2022 11:52 PM, René Zwanenburg wrote:

 On Thursday, 12 May 2022 at 08:49:18 UTC, Sebastiaan Koppe wrote:

 Don't forget that fibers aren't supported on all platforms.


 
 Didn't think of that. I only run D stuff on X86-64 at the moment but 
 cross-platform support is definitely important.


If you have phobos linked in, you are pretty much guaranteed to have fibers.

You will either get a compile time error, or a link error if you didn't.
May 12 2022