www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Proposal for porting D runtime to WebAssembly

reply Sebastiaan Koppe <mail skoppe.eu> writes:
This is my proposal for porting D runtime to WebAssembly. I would 
like to ask you to review it. You can find it here: 
https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Nov 23 2019
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


While I can't say anything on the details, the document looks 
well prepared. Thanks a lot for your work, it is very good 
starting point.

Kind regards
Andre
Nov 23 2019
prev sibling next sibling parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


Perhaps you can explicitly clarify that "port" in this context 
means that you will add the required version(WebAssembly) blocks 
in the official druntime, rather than in a fork of druntime.
(WebAssembly predefined version now explicitly mentions that it 
is for 32bit. Do you want to broaden this to 64bit aswell, or add 
a new version identifier?)

I read that Clang uses a triple with explicit mention of WASI: 
--target wasm32-wasi
Are you planning for the same with LDC? Will you need a new 
predefined version identifier for WASI-libc? Perhaps group all 
required compiler features in a section (and move the `real` 
story there).

Can you elaborate on how you envision CI testing?
Do you want to add that to LDC testing? (this may also mean that 
you first add a new change to LDC's druntime, confirming 
functionality with LDC CI, and then upstreaming the change)

I'm assuming you already started some work in this area? Where 
can we track it?

Great initiative!
   Johan
Nov 23 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 10:29:24 UTC, Johan Engelen 
wrote:

 Perhaps you can explicitly clarify that "port" in this context 
 means that you will add the required version(WebAssembly) 
 blocks in the official druntime, rather than in a fork of 
 druntime.


Indeed. It will not be a fork, but the changes will be upstreamed 
into the official druntime.


 (WebAssembly predefined version now explicitly mentions that it 
 is for 32bit. Do you want to broaden this to 64bit aswell, or 
 add a new version identifier?)


I haven't seen anybody working on wasm64. I know it exists, but 
that is about it.

I do not know what the future of wasm64 will hold. Probably there 
will come a time somebody needs it, but as of yet everybody 
focuses on wasm32, and I don't see that changing anytime soon.

Still, I think it is a good idea to be prepared. Personally I 
would add wasm32 and wasm64 and also define WebAssembly whenever 
one of them is. Don't know if that is the smart thing to do.


 I read that Clang uses a triple with explicit mention of WASI: 
 --target wasm32-wasi
 Are you planning for the same with LDC? Will you need a new 
 predefined version identifier for WASI-libc? Perhaps group all 
 required compiler features in a section (and move the `real` 
 story there).


Rust uses that as well. It would make sense for us to use that as 
well. Good idea.

The ultimate goal is to not use libc, but directly call the wasi 
api. In the mean, yes, we should introduce the WASI-libc version. 
I have now put all that under the WebAssembly version, but that 
is conflating things. (although it is not a big deal, since the 
linker will strip them out if unused.)

Will add to a separate compiler section in the gist.


 Can you elaborate on how you envision CI testing?


We can use any of the WASI runtimes. I personally use Wasmer 
(written in rust, uses cranelift which is also used in Firefox). 
Another option (or in parallel) would be using the V8 in either 
node or an headless browser (although that would be better suited 
for testing JavaScript interoperability).

I would go with wasmer first.


 Do you want to add that to LDC testing? (this may also mean 
 that you first add a new change to LDC's druntime, confirming 
 functionality with LDC CI, and then upstreaming the change)


Yes, in fact, I am already targetting LDC's druntime.


 I'm assuming you already started some work in this area? Where 
 can we track it?


Will post the link here after some clean up. A few days.


 Great initiative!
   Johan


Thanks, these are some very good points.
Nov 23 2019
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 10:29:24 UTC, Johan Engelen 
wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 I'm assuming you already started some work in this area? Where 
 can we track it?

 Great initiative!
   Johan


You can track the work here: 
https://github.com/skoppe/druntime/tree/wasm

Almost all unittests pass.

I am in the process of getting `ldc-build-druntime` to build it, 
as well as hooking into main().

I really wanted to make a pr, so that others can build it as 
well, but I am pressed for time due to family weekend trip. It is 
on my list once I get back, as well as incorpareting all info 
from this thread back into the proposal.

Some things to tackle before going beta:

- AA unittests fail
- reals (probably are going to be unsupported)
- wasi libc needs to be distributed (either in source and 
compiled into wasm druntime) or statically linked
- CI (but should be doable once ldc-build-druntime works)
- hooking into main() (I thought about making a  weak _start() in 
druntime so that users can still override it when they want) 
(_start is the wasm's equivalent of _Dmain)
- probably need help from LDC to spill i32 pointer on the shadow 
stack
Jan 03 2020
next sibling parent reply kinke <noone nowhere.com> writes:
On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe wrote:

 You can track the work here: 
 https://github.com/skoppe/druntime/tree/wasm


I gave it a quick glance; looks pretty good, and like pretty much 
work. ;) - Thx.

The compiler should probably help a bit by firstly predefining a 
version `CRuntime_WASI` (either for all wasm targets, or for 
triples like wasm32-unknown-unknown-wasi) and secondly emitting 
TLS globals as regular globals for now, so that you don't have to 
add `__gshared` everywhere.


 - reals (probably are going to be unsupported)


It's probably just a matter of checking which type clang uses for 
C `long double` when targeting wasm, and making LDC use the same 
type.


 - wasi libc needs to be distributed (either in source and 
 compiled into wasm druntime) or statically linked


I'd prefer a static lib (and referencing that one via 
`-defaultlib=druntime-ldc,phobos2-ldc,wasi` in ldc2.conf's wasm 
section).
Building it via LDC CI for inclusion in (some?) prebuilt LDC 
packages is probably not that much of a hassle with a clang host 
compiler.


 once ldc-build-druntime works


If you need some CMake help (excluding C files etc.), 
https://github.com/ldc-developers/ldc/pull/2787 might have 
something useful.


 (_start is the wasm's equivalent of _Dmain)


Not really; _start (in libc) is used on Linux too, which sets up 
the C runtime, then calls C main, which calls druntime's 
_d_run_main which in turn calls _Dmain.
Jan 04 2020
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 4 January 2020 at 16:28:24 UTC, kinke wrote:

 On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe 
 wrote:

 You can track the work here: 
 https://github.com/skoppe/druntime/tree/wasm


 I gave it a quick glance; looks pretty good, and like pretty 
 much work. ;) - Thx.


Great. Thanks for looking.


 The compiler should probably help a bit by firstly predefining 
 a version `CRuntime_WASI` (either for all wasm targets, or for 
 triples like wasm32-unknown-unknown-wasi) and secondly emitting 
 TLS globals as regular globals for now, so that you don't have 
 to add `__gshared` everywhere.


Yes. I will probably manage to do the first, but for the second 
one I definitely need some pointers.


 - reals (probably are going to be unsupported)


 It's probably just a matter of checking which type clang uses 
 for C `long double` when targeting wasm, and making LDC use the 
 same type.


Could be. I personally prefer to avoid them because wasm only 
supports f32/f64, which I guess means they will be emulated (I 
have no idea though, maybe some wasm hosts do the right thing). 
But some people might need them, so if fixing the ABI is not a 
big deal, we could include them.


 - wasi libc needs to be distributed (either in source and 
 compiled into wasm druntime) or statically linked


 I'd prefer a static lib (and referencing that one via 
 `-defaultlib=druntime-ldc,phobos2-ldc,wasi` in ldc2.conf's wasm 
 section).


Good.


 Building it via LDC CI for inclusion in (some?) prebuilt LDC 
 packages is probably not that much of a hassle with a clang 
 host compiler.


I don't think so either. I have already got it building, so I 
just need to go over my notes.


 once ldc-build-druntime works


 If you need some CMake help (excluding C files etc.), 
 https://github.com/ldc-developers/ldc/pull/2787 might have 
 something useful.


Thanks.


 (_start is the wasm's equivalent of _Dmain)


 Not really; _start (in libc) is used on Linux too, which sets 
 up the C runtime, then calls C main, which calls druntime's 
 _d_run_main which in turn calls _Dmain.


Ahh, fumbling as I go along. Thanks for the correction.
Jan 07 2020
prev sibling parent Elronnd <elronnd elronnd.net> writes:
 (_start is the wasm's equivalent of _Dmain)


 Not really; _start (in libc) is used on Linux too, which sets 
 up the C runtime, then calls C main, which calls druntime's 
 _d_run_main which in turn calls _Dmain.


Small correction: _start generally calls __libc_start_main() or 
similar, with the addresses of main, argc, argv, envp, module ini 
and fini, and possibly some other stuff I forgot about.
Jan 15 2020
prev sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe wrote:


 - reals (probably are going to be unsupported)


It seems to me for now they can be threated as double without any 
problems
Jan 05 2020
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 5 January 2020 at 08:24:21 UTC, Denis Feklushkin wrote:

 On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe 
 wrote:


 - reals (probably are going to be unsupported)


 It seems to me for now they can be threated as double without 
 any problems


Yeah, that is what I have done so far.
Jan 07 2020
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 7 January 2020 at 08:17:37 UTC, Sebastiaan Koppe 
wrote:

 On Sunday, 5 January 2020 at 08:24:21 UTC, Denis Feklushkin 
 wrote:

 On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe 
 wrote:


 - reals (probably are going to be unsupported)


 It seems to me for now they can be threated as double without 
 any problems


 Yeah, that is what I have done so far.


I believe that's the best choice even long term. `real` is 
supposed to represent the largest natively supported FP type by 
the underlying ISA. In WebAssembly that's f64, so there's no need 
emulate anything. Of course, people who need wider 
integer/fixed/floating types can use third-party libraries for 
that.
There are other platforms where D's real type is the same as 
double, so I don't see a reason to worry.
Jan 07 2020
prev sibling next sibling parent reply Ola Fosheim Gr <ola.fosheim.grostad gmail.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


Yes, if I read this right the plan is to keep the runtime small. 
That is good, small footprint is important.

Also, if applicable, structure the object file in way that 
compress well (gzip). E.g. the layout of compiler emitted data 
structures and constants on the heap.
Nov 23 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 12:40:20 UTC, Ola Fosheim Gr 
wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 Yes, if I read this right the plan is to keep the runtime 
 small. That is good, small footprint is important.


Small footprint is super important, especially when targeting the 
browser.

The first stage is getting something to work though, but I will 
definitely chisel bytes off afterwards.


 Also, if applicable, structure the object file in way that 
 compress well (gzip). E.g. the layout of compiler emitted data 
 structures and constants on the heap.


I don't know how much control we have (or want) over this. In the 
end LLVM and wasm-lld do that and we just piggyback that.
Nov 23 2019
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


I was wondering whats your position on Fibers?
Can they be implemented in current WebAssembly?

If so I'd guess they would be a nice match for async related 
functionality javascript is known for.

Best regards,
Alexandru.
Nov 23 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru Ermicioi 
wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 I was wondering whats your position on Fibers?


I am not going to support them in this initial port. And to be 
honest I rather see us moving towards stackless coroutines.


 Can they be implemented in current WebAssembly?


I haven't looked into it. I suppose they could be, since go has 
their goroutines supported in wasm as well.

But I don't think it is easy. WebAssembly uses the Harvard 
architecture, which means code and data is separate and code 
isn't addressable. That is why wasm uses a function table and 
indexes instead of function pointer addresses. So things like 
moving the instruction pointer are out.


 If so I'd guess they would be a nice match for async related 
 functionality javascript is known for.


You can still use the JavaScript eventloop, either browser or 
node.
Nov 23 2019
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/23/19 3:48 PM, Sebastiaan Koppe wrote:

 On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru Ermicioi wrote:

 I was wondering whats your position on Fibers?


 
 I am not going to support them in this initial port. And to be honest I 
 rather see us moving towards stackless coroutines.


I really hope you're right. I've been pushing for those for years, but 

them for eons and D still seems to have no interest in coroutines that 
*don't* involve the overhead of fibers bothers me to no end.

I did started working on a couple DIPs for them, though. Interestingly, 
I just found out today about C++'s proposed coroutines and was shocked 
by how similar they are to what I was designing; even right down to 
details like how the existence of a yield instruction is what triggers 
the compiler to treat the function as a coroutine, and the requirement 
that a coroutine's return type be a special type that includes the state 
information.

Still, a few differences, though. For example, unlike the C++ proposal, 
I'm hoping to avoid the need for additional keywords and heap 
allocation. And I also started a secondary DIP that builds on the 
coroutine foundation to make a much cleaner user-experience using the 
coroutines to generate ranges (what I would expect to be the most common 
use-case).
Nov 23 2019
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 23 November 2019 at 23:21:49 UTC, Nick Sabalausky 
(Abscissa) wrote:

 On 11/23/19 3:48 PM, Sebastiaan Koppe wrote:
 years, but never got the impression anyone else cared. The fact 

 interest in coroutines that *don't* involve the overhead of 
 fibers bothers me to no end.


Fun fact: Simula had stackless coroutines in the 1960s... :-)

Well, I guess I have to add that they were stackless because the 
language was implemented with closure-like-objects, so there was 
no stack, only activation records on the heap. Actually, I 
believe the MIPS architecture had this as their default too (or 
maybe it was another CPU, anyway, it has been a thing.)
Nov 24 2019
prev sibling next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 23 November 2019 at 23:21:49 UTC, Nick Sabalausky 
(Abscissa) wrote:

 I did started working on a couple DIPs for them, though.


Can you share a link to DIP draft?
I'd like to read how it would work.

Thank you,
Alexandru.
Nov 24 2019
prev sibling parent reply Georgi D <georgid outlook.com> writes:
On Saturday, 23 November 2019 at 23:21:49 UTC, Nick Sabalausky 
(Abscissa) wrote:

 On 11/23/19 3:48 PM, Sebastiaan Koppe wrote:

 On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru 
 Ermicioi wrote:

 I was wondering whats your position on Fibers?


 
 I am not going to support them in this initial port. And to be





 I did started working on a couple DIPs for them, though. 
 Interestingly, I just found out today about C++'s proposed 
 coroutines and was shocked by how similar they are to what I 
 was designing; even right down to details like how the 
 existence of a yield instruction is what triggers the compiler 
 to treat the function as a coroutine, and the requirement that 
 a coroutine's return type be a special type that includes the 
 state information.

 Still, a few differences, though. For example, unlike the C++ 
 proposal, I'm hoping to avoid the need for additional keywords 
 and heap allocation. And I also started a secondary DIP that 
 builds on the coroutine foundation to make a much cleaner 
 user-experience using the coroutines to generate ranges (what I 
 would expect to be the most common use-case).


Hi Sebastiaan,

If you are looking at the C++ coroutines I would recommend 
looking into the  proposal for "First-class symmetric coroutines 
in C++".

The official paper can be found here: 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1430r1.pdf

There is also a presentation with some nice animations explaining 
the proposal here:
https://docs.google.com/presentation/d/1B5My9nh-P2HLGI8Dtfm6Q7ZfHD9hJ27kJUgnfA2syv4/edit?usp=sharing

There paper is still in early development, for example the syntax 
has changed since then as well as some other pieces.

If you are interested I can connect you with the author of the 
paper who can explain it with more details.

Georgi
Nov 25 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 13:50:20 UTC, Georgi D wrote:

 Hi Sebastiaan,

 If you are looking at the C++ coroutines I would recommend 
 looking into the  proposal for "First-class symmetric 
 coroutines in C++".

 The official paper can be found here: 
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1430r1.pdf

 There is also a presentation with some nice animations 
 explaining the proposal here:
 https://docs.google.com/presentation/d/1B5My9nh-P2HLGI8Dtfm6Q7ZfHD9hJ27kJUgnfA2syv4/edit?usp=sharing

 There paper is still in early development, for example the 
 syntax has changed since then as well as some other pieces.

 If you are interested I can connect you with the author of the 
 paper who can explain it with more details.

 Georgi


Thanks for that. It would be great, but I don't have time for 
that at the moment.
Nov 26 2019
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2019-11-23 10:51, Sebastiaan Koppe wrote:

 This is my proposal for porting D runtime to WebAssembly. I would like 
 to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


What will happen to code that uses TLS? Will it be promoted to a global 
variable or will it fail to compile?

-- 
/Jacob Carlborg
Nov 24 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 24 November 2019 at 18:46:04 UTC, Jacob Carlborg wrote:

 On 2019-11-23 10:51, Sebastiaan Koppe wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 What will happen to code that uses TLS? Will it be promoted to 
 a global variable or will it fail to compile?


LLVM errors out saying it can't select tls for wasm. We could 
modify ldc to not emit TLS instructions under WebAssembly.

But yeah, right now, you need to __gshared everything.

I know.
Nov 24 2019
parent Dukc <ajieskola gmail.com> writes:
On Sunday, 24 November 2019 at 20:42:24 UTC, Sebastiaan Koppe 
wrote:

 LLVM errors out saying it can't select tls for wasm. We could 
 modify ldc to not emit TLS instructions under WebAssembly.


No need do make that rule WASM-specific. Do this for all programs 
that have thearding disabled.
Nov 25 2019
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


This proposal is so perfectly balanced between value and 
implementability that I can find nothing to add or remove.

I'm interested, what's your motivation in doing all this? If I 
understood correctly, your primary motivation to write Spasm was 
to write better optimized front-end programs than you get with JS 
frameworks. But wouldn't it be easier to just use Rust since it 
has already implemented all this?
Nov 25 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 09:01:15 UTC, Dukc wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 This proposal is so perfectly balanced between value and 
 implementability that I can find nothing to add or remove.


Thanks!


 I'm interested, what's your motivation in doing all this? If I 
 understood correctly, your primary motivation to write Spasm 
 was to write better optimized front-end programs than you get 
 with JS frameworks.


That is a fair question. Spasm has been very successful if you 
look at rendering speed. It (almost) beats everything else out 
there [1]. Well, that is not surprising since everything is known 
at compile time; it literally compiles down to the same code as 
if you issued low-level dom calls manually. I am very happy about 
that.

With regards to developer experience it is behind. First of all 
you have to deal with betterC. This alone is already a hurdle for 
many. Second is the DSL, or lack of it. It doesn't come close to 
something like e.g. SwiftUI. In fact, I wrote a (unfinished) 
material-ui component library on top of spasm and I was 
struggling at times.

So it became clear to me I need to have druntime available. It 
will allow people to use the (almost) complete set of D features 
and it opens up some metaprogramming avenues that are closed off 
right now. With that I will be able to create some nice DSL, in 
line with JSX/SwiftUI or 
<insert-your-favorite-declarative-framework>.

There are plenty of opportunities here. It is not unfeasible to 
connect spasm to Qt, or dlangui, and create a cross-platform UI 
library, something like flutter.

On the other hand, I am very excited about WebAssembly in 
general. It is certainly at the beginning of the hype curve and I 
suspect some very exciting things will appear in the future. Some 
of them are already here right now. For instance, you can target 
ARM by compiling D code to wasm and then use wasmer to compile it 
to ARM. With D connecting itself to the wasm world it exposes 
itself to a lot of cool things, which we mostly get for free.

As an example, it is just a matter of time before a PaaS provider 
fully embraces wasm. Instead of having docker containers you just 
compile to wasm, which will be pretty small and can boot in (sub) 
milli-seconds (plus they don't necessarily need a linux host 
kernel running and can run it closer to the hypervisor.)

There are tons of possibilities here, and I want D to be a viable 
option when that day comes.

So it is not just about frontends anymore.


 But wouldn't it be easier to just use Rust since it has already 
 implemented all this?


All the rust frameworks for web apps that I have seen rely on 
runtime techniques like the virtual dom. As a consequence they 
spend more cpu time and result in bigger files. That may be 
perfectly fine for most (and it probably is), but I wanted to 
squeeze it as much as I could. Maybe it is possible to do that in 
rust as well, I don't know. D's metaprogramming seemed a more 
natural fit.

[1] except Svelte, which is a little bit smaller in code size, 
and a tiny bit faster. But they build a whole compiler just for 
that. Lets wait for host bindings support in wasm and measure 
again.
Nov 25 2019
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 November 2019 at 12:52:46 UTC, Sebastiaan Koppe 
wrote:

 As an example, it is just a matter of time before a PaaS 
 provider fully embraces wasm.


This sounds interesting, I've been pondering about serverless 
FaaS (function as a service), where you basically (hopefully) get 
functions triggered by NoSQL database updates and not have to 
bother with your own webserver.

I see that CloudFlare has support for webassembly in their 
workers, but for Google Functions I only see Node10, but maybe 
they can run webassembly as well? I haven't found anything 
definitive on it though...

https://blog.cloudflare.com/webassembly-on-cloudflare-workers/
https://cloud.google.com/functions/docs/


 Instead of having docker containers you just compile to wasm, 
 which will be pretty small and can boot in (sub) milli-seconds 
 (plus they don't necessarily need a linux host kernel running 
 and can run it closer to the hypervisor.)


Yes, but the biggest potential I see is when you don't have to 
set up servers to process data.

Just throw the data into the distributed database, which triggers 
a Function that updates other parts of the database and then 
triggers another function that push the resulting PDF (or 
whatever) to a service that serves the files directly (i.e. 
cached close to the user like CloudFlare).

Seems like it could be less hassle, but not sure if will catch on 
or fizzle out... I think I'll wait and see what happens. :-)
Nov 25 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 13:28:17 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 25 November 2019 at 12:52:46 UTC, Sebastiaan Koppe 
 wrote:

 As an example, it is just a matter of time before a PaaS 
 provider fully embraces wasm.


 This sounds interesting, I've been pondering about serverless 
 FaaS (function as a service), where you basically (hopefully) 
 get functions triggered by NoSQL database updates and not have 
 to bother with your own webserver.


This is already doable with dynamodb, or kinesis streams. Or 
google's dataflow.

Using wasm just makes that more seamless (and faster).


 I see that CloudFlare has support for webassembly in their 
 workers, but for Google Functions I only see Node10, but maybe 
 they can run webassembly as well? I haven't found anything 
 definitive on it though...


Node has good wasm support, I don't know how you would get the 
wasm binary in, but it probably can be done.


 Instead of having docker containers you just compile to wasm, 
 which will be pretty small and can boot in (sub) milli-seconds 
 (plus they don't necessarily need a linux host kernel running 
 and can run it closer to the hypervisor.)


 Yes, but the biggest potential I see is when you don't have to 
 set up servers to process data.


I rather not setup servers for anything.


 Just throw the data into the distributed database, which 
 triggers a Function that updates other parts of the database 
 and then triggers another function that push the resulting PDF 
 (or whatever) to a service that serves the files directly (i.e. 
 cached close to the user like CloudFlare).


You don't have to wait for that. That future is already here. The 
in and output could also be distributed storage, event streams or 
some queue.

The problem, however, is often when using those tools you get 
pushed into a small set of supported programming languages. Like 
AWS' glue that focuses on Scala or Python, or google's functions 
that only support js/python and go. Understandable, but I rather 
choose my own language. Wasm makes that possible.
Nov 25 2019
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 November 2019 at 13:52:29 UTC, Sebastiaan Koppe 
wrote:

 You don't have to wait for that. That future is already here. 
 The in and output could also be distributed storage, event 
 streams or some queue.


Yes, I am most familiar with Google Cloud. Earlier this year 
Google Functions was not available in European datacenters IIRC, 
but now it is at least available in London and Belgium. So things 
are moving in that direction, somewhat slowly. It is annoying to 
not have Google Functions when working with Google Firebase, so 
if webworkers is possible then that could make things much better 
(even for simple things like generating thumbnail images).


 Like AWS' glue that focuses on Scala or Python, or google's 
 functions that only support js/python and go. Understandable, 
 but I rather choose my own language. Wasm makes that possible.


Let's hope there is a way for other services than CloudFlare. 
CloudFlare Workers look cool, but their KV store has very low 
propagation guarantees on updates (60 seconds).
Nov 25 2019
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/25/19 7:52 AM, Sebastiaan Koppe wrote:

 So it became clear to me I need to have druntime available. It will 
 allow people to use the (almost) complete set of D features and it opens 
 up some metaprogramming avenues that are closed off right now. With that 
 I will be able to create some nice DSL, in line with JSX/SwiftUI or 
 <insert-your-favorite-declarative-framework>.
 
 There are plenty of opportunities here. It is not unfeasible to connect 
 spasm to Qt, or dlangui, and create a cross-platform UI library, 
 something like flutter.
 
 On the other hand, I am very excited about WebAssembly in general. It is 
 certainly at the beginning of the hype curve and I suspect some very 
 exciting things will appear in the future. Some of them are already here 
 right now. For instance, you can target ARM by compiling D code to wasm 
 and then use wasmer to compile it to ARM. With D connecting itself to 
 the wasm world it exposes itself to a lot of cool things, which we 
 mostly get for free.
 
 As an example, it is just a matter of time before a PaaS provider fully 
 embraces wasm. Instead of having docker containers you just compile to 
 wasm, which will be pretty small and can boot in (sub) milli-seconds 
 (plus they don't necessarily need a linux host kernel running and can 
 run it closer to the hypervisor.)


As someone who does web application development, all of this sounds 
awesome. I would LOVE to have a real programming language to do the 
client-side stuff.

-Steve
Nov 25 2019
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


Thanks for putting this together, it looks very carefully thought 
out.

On this particular part:


 Exceptions can be thrown but not catched. A thrown exception 
 will
 terminate the program. Exceptions are still in the proposal 
 phase.
 When the proposal is accepted exceptions can be fully supported.


This would suggest that there may be some benefit in D providing 
improved support for return-type-based error propagation (as with 
`Result` from Rust), no ... ?
Nov 25 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 12:19:30 UTC, Joseph Rushton 
Wakeling wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 Thanks for putting this together, it looks very carefully 
 thought out.


Thanks!


 Exceptions can be thrown but not catched. A thrown exception 
 will
 terminate the program. Exceptions are still in the proposal 
 phase.
 When the proposal is accepted exceptions can be fully 
 supported.


 This would suggest that there may be some benefit in D 
 providing improved support for return-type-based error 
 propagation (as with `Result` from Rust), no ... ?


Yes, definitely. But what do you mean with improved support? Like 
better pattern matching over either types?
Nov 25 2019
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 25 November 2019 at 13:00:23 UTC, Sebastiaan Koppe 
wrote:

 Yes, definitely. But what do you mean with improved support? 
 Like better pattern matching over either types?


Yes, that sort of thing.  And maybe a move towards trying to use 
this kind of error handling in newer editions of the standard 
library (I'm reluctant to push too strongly on that, but I get 
the impression there is some inclination to move in this 
direction, as a reflection of wider design trends).
Nov 25 2019
prev sibling next sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


On the GC part. It says "The only unknown part is how to dump the 
registers to the stack to ensure no pointers are held in the 
registers only."

Please correct me where I'm wrong, but on the level of 
WebAssembly there are no registers, there is an operand stack 
outside the address space, there are local variables to the 
current function, again outside the accessible address space of 
program's linear memory, and there is the linear memory itself. 
So scanning the stack becomes a really hard (should I say 
impossible?) part. What some compilers do is they organize 
another stack manually in the linear memory and store the values 
that would otherwise be on the normal stack, there. Which means 
in case of D you'll have to seriously change the codegen, to 
change how local variables are stored, and to use a kind of 
shadow stack for temporaries in expressions that may be pointers. 
Do you really have a plan about it?
Nov 25 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 18:44:01 UTC, thedeemon wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 Please correct me where I'm wrong, but on the level of 
 WebAssembly there are no registers, there is an operand stack 
 outside the address space, there are local variables to the 
 current function, again outside the accessible address space of 
 program's linear memory, and there is the linear memory itself. 
 So scanning the stack becomes a really hard (should I say 
 impossible?) part. What some compilers do is they organize 
 another stack manually in the linear memory and store the 
 values that would otherwise be on the normal stack, there.


Yeah, that accurately describes the situation. I will update the 
wording in the document to use 'stack', 'shadow stack' (also 
sometimes called 'user stack') and the local variable. Thanks.

One solution that I employed in spasm's experimental gc is to 
only run it directly from javascript. This way there can't be 
anything hiding in the stack or in a local variable. Although 
that approach doesn't work for all use cases.


 Which means in case of D you'll have to seriously change the 
 codegen, to change how local variables are stored, and to use a 
 kind of shadow stack for temporaries in expressions that may be 
 pointers. Do you really have a plan about it?


Well, no, not fully. That is why I said 'unknown'. But there must 
be a solution somewhere.

LLVM already puts pointers to stack or local variables in the 
shadow stack. As well as for structs-by-val that don't fit the 
stack. We could adjust LDC to nudge LLVM to maintain live roots 
on the shadow stack as well.

Go's approach is to put everything on the shadow stack. (see: 
https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4/preview#heading=h.mjo1bish3xni)

There is also the possibility of a code transformation. Binaryen 
has a spill-the-pointer pass that effectively gets you go's 
solution (but only for i32's) (see: 
https://github.com/WebAssembly/binaryen/blob/master/src/passes/pass.cpp#L310)

I am favoring the first option, but I don't know how hard that 
would be. Will update the document with this info.

Thank you for questioning this.
Nov 26 2019
prev sibling next sibling parent reply Thomas Brix <brix brix-verden.dk> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


An alternative idea, would be to use emscriptens fork of musl to 
have a full C-library. AFAIK this includes threading.

LLVM is supposed to support TLS in wasm since version 9.
Nov 26 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 26 November 2019 at 09:18:05 UTC, Thomas Brix wrote:

 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:

 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


 An alternative idea, would be to use emscriptens fork of musl 
 to have a full C-library. AFAIK this includes threading.

 LLVM is supposed to support TLS in wasm since version 9.


Yes, indeed. https://reviews.llvm.org/D64537 gives a good 
overview.

I believe it is best to first actually have a version of druntime 
on wasm, rather than eagerly pulling in all the latest features. 
I find the scope I set in the proposal to be quite reasonable.

Adding tls, threading and exception handling would be much easier 
after this work is done and merged. And it would also be 
something others might want to contribute to.
Nov 26 2019
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2019-11-23 09:51:13 +0000, Sebastiaan Koppe said:


 This is my proposal for porting D runtime to WebAssembly. I would like 
 to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d


Not sure if you are aware of this:

https://wasmtime.dev/

Maybe it helps or gives some inspiration.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Nov 27 2019