www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Proposal for porting D runtime to WebAssembly

reply Sebastiaan Koppe <mail skoppe.eu> writes:
This is my proposal for porting D runtime to WebAssembly. I would 
like to ask you to review it. You can find it here: 
https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Nov 23 2019
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
While I can't say anything on the details, the document looks well prepared. Thanks a lot for your work, it is very good starting point. Kind regards Andre
Nov 23 2019
prev sibling next sibling parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Perhaps you can explicitly clarify that "port" in this context means that you will add the required version(WebAssembly) blocks in the official druntime, rather than in a fork of druntime. (WebAssembly predefined version now explicitly mentions that it is for 32bit. Do you want to broaden this to 64bit aswell, or add a new version identifier?) I read that Clang uses a triple with explicit mention of WASI: --target wasm32-wasi Are you planning for the same with LDC? Will you need a new predefined version identifier for WASI-libc? Perhaps group all required compiler features in a section (and move the `real` story there). Can you elaborate on how you envision CI testing? Do you want to add that to LDC testing? (this may also mean that you first add a new change to LDC's druntime, confirming functionality with LDC CI, and then upstreaming the change) I'm assuming you already started some work in this area? Where can we track it? Great initiative! Johan
Nov 23 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 10:29:24 UTC, Johan Engelen 
wrote:
 Perhaps you can explicitly clarify that "port" in this context 
 means that you will add the required version(WebAssembly) 
 blocks in the official druntime, rather than in a fork of 
 druntime.
Indeed. It will not be a fork, but the changes will be upstreamed into the official druntime.
 (WebAssembly predefined version now explicitly mentions that it 
 is for 32bit. Do you want to broaden this to 64bit aswell, or 
 add a new version identifier?)
I haven't seen anybody working on wasm64. I know it exists, but that is about it. I do not know what the future of wasm64 will hold. Probably there will come a time somebody needs it, but as of yet everybody focuses on wasm32, and I don't see that changing anytime soon. Still, I think it is a good idea to be prepared. Personally I would add wasm32 and wasm64 and also define WebAssembly whenever one of them is. Don't know if that is the smart thing to do.
 I read that Clang uses a triple with explicit mention of WASI: 
 --target wasm32-wasi
 Are you planning for the same with LDC? Will you need a new 
 predefined version identifier for WASI-libc? Perhaps group all 
 required compiler features in a section (and move the `real` 
 story there).
Rust uses that as well. It would make sense for us to use that as well. Good idea. The ultimate goal is to not use libc, but directly call the wasi api. In the mean, yes, we should introduce the WASI-libc version. I have now put all that under the WebAssembly version, but that is conflating things. (although it is not a big deal, since the linker will strip them out if unused.) Will add to a separate compiler section in the gist.
 Can you elaborate on how you envision CI testing?
We can use any of the WASI runtimes. I personally use Wasmer (written in rust, uses cranelift which is also used in Firefox). Another option (or in parallel) would be using the V8 in either node or an headless browser (although that would be better suited for testing JavaScript interoperability). I would go with wasmer first.
 Do you want to add that to LDC testing? (this may also mean 
 that you first add a new change to LDC's druntime, confirming 
 functionality with LDC CI, and then upstreaming the change)
Yes, in fact, I am already targetting LDC's druntime.
 I'm assuming you already started some work in this area? Where 
 can we track it?
Will post the link here after some clean up. A few days.
 Great initiative!
   Johan
Thanks, these are some very good points.
Nov 23 2019
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 10:29:24 UTC, Johan Engelen 
wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
I'm assuming you already started some work in this area? Where can we track it? Great initiative! Johan
You can track the work here: https://github.com/skoppe/druntime/tree/wasm Almost all unittests pass. I am in the process of getting `ldc-build-druntime` to build it, as well as hooking into main(). I really wanted to make a pr, so that others can build it as well, but I am pressed for time due to family weekend trip. It is on my list once I get back, as well as incorpareting all info from this thread back into the proposal. Some things to tackle before going beta: - AA unittests fail - reals (probably are going to be unsupported) - wasi libc needs to be distributed (either in source and compiled into wasm druntime) or statically linked - CI (but should be doable once ldc-build-druntime works) - hooking into main() (I thought about making a weak _start() in druntime so that users can still override it when they want) (_start is the wasm's equivalent of _Dmain) - probably need help from LDC to spill i32 pointer on the shadow stack
Jan 03
next sibling parent reply kinke <noone nowhere.com> writes:
On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe wrote:
 You can track the work here: 
 https://github.com/skoppe/druntime/tree/wasm
I gave it a quick glance; looks pretty good, and like pretty much work. ;) - Thx. The compiler should probably help a bit by firstly predefining a version `CRuntime_WASI` (either for all wasm targets, or for triples like wasm32-unknown-unknown-wasi) and secondly emitting TLS globals as regular globals for now, so that you don't have to add `__gshared` everywhere.
 - reals (probably are going to be unsupported)
It's probably just a matter of checking which type clang uses for C `long double` when targeting wasm, and making LDC use the same type.
 - wasi libc needs to be distributed (either in source and 
 compiled into wasm druntime) or statically linked
I'd prefer a static lib (and referencing that one via `-defaultlib=druntime-ldc,phobos2-ldc,wasi` in ldc2.conf's wasm section). Building it via LDC CI for inclusion in (some?) prebuilt LDC packages is probably not that much of a hassle with a clang host compiler.
 once ldc-build-druntime works
If you need some CMake help (excluding C files etc.), https://github.com/ldc-developers/ldc/pull/2787 might have something useful.
 (_start is the wasm's equivalent of _Dmain)
Not really; _start (in libc) is used on Linux too, which sets up the C runtime, then calls C main, which calls druntime's _d_run_main which in turn calls _Dmain.
Jan 04
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 4 January 2020 at 16:28:24 UTC, kinke wrote:
 On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe 
 wrote:
 You can track the work here: 
 https://github.com/skoppe/druntime/tree/wasm
I gave it a quick glance; looks pretty good, and like pretty much work. ;) - Thx.
Great. Thanks for looking.
 The compiler should probably help a bit by firstly predefining 
 a version `CRuntime_WASI` (either for all wasm targets, or for 
 triples like wasm32-unknown-unknown-wasi) and secondly emitting 
 TLS globals as regular globals for now, so that you don't have 
 to add `__gshared` everywhere.
Yes. I will probably manage to do the first, but for the second one I definitely need some pointers.
 - reals (probably are going to be unsupported)
It's probably just a matter of checking which type clang uses for C `long double` when targeting wasm, and making LDC use the same type.
Could be. I personally prefer to avoid them because wasm only supports f32/f64, which I guess means they will be emulated (I have no idea though, maybe some wasm hosts do the right thing). But some people might need them, so if fixing the ABI is not a big deal, we could include them.
 - wasi libc needs to be distributed (either in source and 
 compiled into wasm druntime) or statically linked
I'd prefer a static lib (and referencing that one via `-defaultlib=druntime-ldc,phobos2-ldc,wasi` in ldc2.conf's wasm section).
Good.
 Building it via LDC CI for inclusion in (some?) prebuilt LDC 
 packages is probably not that much of a hassle with a clang 
 host compiler.
I don't think so either. I have already got it building, so I just need to go over my notes.
 once ldc-build-druntime works
If you need some CMake help (excluding C files etc.), https://github.com/ldc-developers/ldc/pull/2787 might have something useful.
Thanks.
 (_start is the wasm's equivalent of _Dmain)
Not really; _start (in libc) is used on Linux too, which sets up the C runtime, then calls C main, which calls druntime's _d_run_main which in turn calls _Dmain.
Ahh, fumbling as I go along. Thanks for the correction.
Jan 07
prev sibling parent Elronnd <elronnd elronnd.net> writes:
 (_start is the wasm's equivalent of _Dmain)
Not really; _start (in libc) is used on Linux too, which sets up the C runtime, then calls C main, which calls druntime's _d_run_main which in turn calls _Dmain.
Small correction: _start generally calls __libc_start_main() or similar, with the addresses of main, argc, argv, envp, module ini and fini, and possibly some other stuff I forgot about.
Jan 15
prev sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe wrote:

 - reals (probably are going to be unsupported)
It seems to me for now they can be threated as double without any problems
Jan 05
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 5 January 2020 at 08:24:21 UTC, Denis Feklushkin wrote:
 On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe 
 wrote:

 - reals (probably are going to be unsupported)
It seems to me for now they can be threated as double without any problems
Yeah, that is what I have done so far.
Jan 07
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 7 January 2020 at 08:17:37 UTC, Sebastiaan Koppe 
wrote:
 On Sunday, 5 January 2020 at 08:24:21 UTC, Denis Feklushkin 
 wrote:
 On Friday, 3 January 2020 at 10:34:40 UTC, Sebastiaan Koppe 
 wrote:

 - reals (probably are going to be unsupported)
It seems to me for now they can be threated as double without any problems
Yeah, that is what I have done so far.
I believe that's the best choice even long term. `real` is supposed to represent the largest natively supported FP type by the underlying ISA. In WebAssembly that's f64, so there's no need emulate anything. Of course, people who need wider integer/fixed/floating types can use third-party libraries for that. There are other platforms where D's real type is the same as double, so I don't see a reason to worry.
Jan 07
prev sibling next sibling parent reply Ola Fosheim Gr <ola.fosheim.grostad gmail.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Yes, if I read this right the plan is to keep the runtime small. That is good, small footprint is important. Also, if applicable, structure the object file in way that compress well (gzip). E.g. the layout of compiler emitted data structures and constants on the heap.
Nov 23 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 12:40:20 UTC, Ola Fosheim Gr 
wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Yes, if I read this right the plan is to keep the runtime small. That is good, small footprint is important.
Small footprint is super important, especially when targeting the browser. The first stage is getting something to work though, but I will definitely chisel bytes off afterwards.
 Also, if applicable, structure the object file in way that 
 compress well (gzip). E.g. the layout of compiler emitted data 
 structures and constants on the heap.
I don't know how much control we have (or want) over this. In the end LLVM and wasm-lld do that and we just piggyback that.
Nov 23 2019
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
I was wondering whats your position on Fibers? Can they be implemented in current WebAssembly? If so I'd guess they would be a nice match for async related functionality javascript is known for. Best regards, Alexandru.
Nov 23 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru Ermicioi 
wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
I was wondering whats your position on Fibers?
I am not going to support them in this initial port. And to be honest I rather see us moving towards stackless coroutines.
 Can they be implemented in current WebAssembly?
I haven't looked into it. I suppose they could be, since go has their goroutines supported in wasm as well. But I don't think it is easy. WebAssembly uses the Harvard architecture, which means code and data is separate and code isn't addressable. That is why wasm uses a function table and indexes instead of function pointer addresses. So things like moving the instruction pointer are out.
 If so I'd guess they would be a nice match for async related 
 functionality javascript is known for.
You can still use the JavaScript eventloop, either browser or node.
Nov 23 2019
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 11/23/19 3:48 PM, Sebastiaan Koppe wrote:
 On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru Ermicioi wrote:
 I was wondering whats your position on Fibers?
I am not going to support them in this initial port. And to be honest I rather see us moving towards stackless coroutines.
I really hope you're right. I've been pushing for those for years, but them for eons and D still seems to have no interest in coroutines that *don't* involve the overhead of fibers bothers me to no end. I did started working on a couple DIPs for them, though. Interestingly, I just found out today about C++'s proposed coroutines and was shocked by how similar they are to what I was designing; even right down to details like how the existence of a yield instruction is what triggers the compiler to treat the function as a coroutine, and the requirement that a coroutine's return type be a special type that includes the state information. Still, a few differences, though. For example, unlike the C++ proposal, I'm hoping to avoid the need for additional keywords and heap allocation. And I also started a secondary DIP that builds on the coroutine foundation to make a much cleaner user-experience using the coroutines to generate ranges (what I would expect to be the most common use-case).
Nov 23 2019
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 23 November 2019 at 23:21:49 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 11/23/19 3:48 PM, Sebastiaan Koppe wrote:
 years, but never got the impression anyone else cared. The fact 

 interest in coroutines that *don't* involve the overhead of 
 fibers bothers me to no end.
Fun fact: Simula had stackless coroutines in the 1960s... :-) Well, I guess I have to add that they were stackless because the language was implemented with closure-like-objects, so there was no stack, only activation records on the heap. Actually, I believe the MIPS architecture had this as their default too (or maybe it was another CPU, anyway, it has been a thing.)
Nov 24 2019
prev sibling next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 23 November 2019 at 23:21:49 UTC, Nick Sabalausky 
(Abscissa) wrote:
 I did started working on a couple DIPs for them, though.
Can you share a link to DIP draft? I'd like to read how it would work. Thank you, Alexandru.
Nov 24 2019
prev sibling parent reply Georgi D <georgid outlook.com> writes:
On Saturday, 23 November 2019 at 23:21:49 UTC, Nick Sabalausky 
(Abscissa) wrote:
 On 11/23/19 3:48 PM, Sebastiaan Koppe wrote:
 On Saturday, 23 November 2019 at 15:23:41 UTC, Alexandru 
 Ermicioi wrote:
 I was wondering whats your position on Fibers?
I am not going to support them in this initial port. And to be
 I did started working on a couple DIPs for them, though. 
 Interestingly, I just found out today about C++'s proposed 
 coroutines and was shocked by how similar they are to what I 
 was designing; even right down to details like how the 
 existence of a yield instruction is what triggers the compiler 
 to treat the function as a coroutine, and the requirement that 
 a coroutine's return type be a special type that includes the 
 state information.

 Still, a few differences, though. For example, unlike the C++ 
 proposal, I'm hoping to avoid the need for additional keywords 
 and heap allocation. And I also started a secondary DIP that 
 builds on the coroutine foundation to make a much cleaner 
 user-experience using the coroutines to generate ranges (what I 
 would expect to be the most common use-case).
Hi Sebastiaan, If you are looking at the C++ coroutines I would recommend looking into the proposal for "First-class symmetric coroutines in C++". The official paper can be found here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1430r1.pdf There is also a presentation with some nice animations explaining the proposal here: https://docs.google.com/presentation/d/1B5My9nh-P2HLGI8Dtfm6Q7ZfHD9hJ27kJUgnfA2syv4/edit?usp=sharing There paper is still in early development, for example the syntax has changed since then as well as some other pieces. If you are interested I can connect you with the author of the paper who can explain it with more details. Georgi
Nov 25 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 13:50:20 UTC, Georgi D wrote:
 Hi Sebastiaan,

 If you are looking at the C++ coroutines I would recommend 
 looking into the  proposal for "First-class symmetric 
 coroutines in C++".

 The official paper can be found here: 
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1430r1.pdf

 There is also a presentation with some nice animations 
 explaining the proposal here:
 https://docs.google.com/presentation/d/1B5My9nh-P2HLGI8Dtfm6Q7ZfHD9hJ27kJUgnfA2syv4/edit?usp=sharing

 There paper is still in early development, for example the 
 syntax has changed since then as well as some other pieces.

 If you are interested I can connect you with the author of the 
 paper who can explain it with more details.

 Georgi
Thanks for that. It would be great, but I don't have time for that at the moment.
Nov 26 2019
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2019-11-23 10:51, Sebastiaan Koppe wrote:
 This is my proposal for porting D runtime to WebAssembly. I would like 
 to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
What will happen to code that uses TLS? Will it be promoted to a global variable or will it fail to compile? -- /Jacob Carlborg
Nov 24 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 24 November 2019 at 18:46:04 UTC, Jacob Carlborg wrote:
 On 2019-11-23 10:51, Sebastiaan Koppe wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
What will happen to code that uses TLS? Will it be promoted to a global variable or will it fail to compile?
LLVM errors out saying it can't select tls for wasm. We could modify ldc to not emit TLS instructions under WebAssembly. But yeah, right now, you need to __gshared everything. I know.
Nov 24 2019
parent Dukc <ajieskola gmail.com> writes:
On Sunday, 24 November 2019 at 20:42:24 UTC, Sebastiaan Koppe 
wrote:
 LLVM errors out saying it can't select tls for wasm. We could 
 modify ldc to not emit TLS instructions under WebAssembly.
No need do make that rule WASM-specific. Do this for all programs that have thearding disabled.
Nov 25 2019
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
This proposal is so perfectly balanced between value and implementability that I can find nothing to add or remove. I'm interested, what's your motivation in doing all this? If I understood correctly, your primary motivation to write Spasm was to write better optimized front-end programs than you get with JS frameworks. But wouldn't it be easier to just use Rust since it has already implemented all this?
Nov 25 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 09:01:15 UTC, Dukc wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
This proposal is so perfectly balanced between value and implementability that I can find nothing to add or remove.
Thanks!
 I'm interested, what's your motivation in doing all this? If I 
 understood correctly, your primary motivation to write Spasm 
 was to write better optimized front-end programs than you get 
 with JS frameworks.
That is a fair question. Spasm has been very successful if you look at rendering speed. It (almost) beats everything else out there [1]. Well, that is not surprising since everything is known at compile time; it literally compiles down to the same code as if you issued low-level dom calls manually. I am very happy about that. With regards to developer experience it is behind. First of all you have to deal with betterC. This alone is already a hurdle for many. Second is the DSL, or lack of it. It doesn't come close to something like e.g. SwiftUI. In fact, I wrote a (unfinished) material-ui component library on top of spasm and I was struggling at times. So it became clear to me I need to have druntime available. It will allow people to use the (almost) complete set of D features and it opens up some metaprogramming avenues that are closed off right now. With that I will be able to create some nice DSL, in line with JSX/SwiftUI or <insert-your-favorite-declarative-framework>. There are plenty of opportunities here. It is not unfeasible to connect spasm to Qt, or dlangui, and create a cross-platform UI library, something like flutter. On the other hand, I am very excited about WebAssembly in general. It is certainly at the beginning of the hype curve and I suspect some very exciting things will appear in the future. Some of them are already here right now. For instance, you can target ARM by compiling D code to wasm and then use wasmer to compile it to ARM. With D connecting itself to the wasm world it exposes itself to a lot of cool things, which we mostly get for free. As an example, it is just a matter of time before a PaaS provider fully embraces wasm. Instead of having docker containers you just compile to wasm, which will be pretty small and can boot in (sub) milli-seconds (plus they don't necessarily need a linux host kernel running and can run it closer to the hypervisor.) There are tons of possibilities here, and I want D to be a viable option when that day comes. So it is not just about frontends anymore.
 But wouldn't it be easier to just use Rust since it has already 
 implemented all this?
All the rust frameworks for web apps that I have seen rely on runtime techniques like the virtual dom. As a consequence they spend more cpu time and result in bigger files. That may be perfectly fine for most (and it probably is), but I wanted to squeeze it as much as I could. Maybe it is possible to do that in rust as well, I don't know. D's metaprogramming seemed a more natural fit. [1] except Svelte, which is a little bit smaller in code size, and a tiny bit faster. But they build a whole compiler just for that. Lets wait for host bindings support in wasm and measure again.
Nov 25 2019
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 November 2019 at 12:52:46 UTC, Sebastiaan Koppe 
wrote:
 As an example, it is just a matter of time before a PaaS 
 provider fully embraces wasm.
This sounds interesting, I've been pondering about serverless FaaS (function as a service), where you basically (hopefully) get functions triggered by NoSQL database updates and not have to bother with your own webserver. I see that CloudFlare has support for webassembly in their workers, but for Google Functions I only see Node10, but maybe they can run webassembly as well? I haven't found anything definitive on it though... https://blog.cloudflare.com/webassembly-on-cloudflare-workers/ https://cloud.google.com/functions/docs/
 Instead of having docker containers you just compile to wasm, 
 which will be pretty small and can boot in (sub) milli-seconds 
 (plus they don't necessarily need a linux host kernel running 
 and can run it closer to the hypervisor.)
Yes, but the biggest potential I see is when you don't have to set up servers to process data. Just throw the data into the distributed database, which triggers a Function that updates other parts of the database and then triggers another function that push the resulting PDF (or whatever) to a service that serves the files directly (i.e. cached close to the user like CloudFlare). Seems like it could be less hassle, but not sure if will catch on or fizzle out... I think I'll wait and see what happens. :-)
Nov 25 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 13:28:17 UTC, Ola Fosheim Grøstad 
wrote:
 On Monday, 25 November 2019 at 12:52:46 UTC, Sebastiaan Koppe 
 wrote:
 As an example, it is just a matter of time before a PaaS 
 provider fully embraces wasm.
This sounds interesting, I've been pondering about serverless FaaS (function as a service), where you basically (hopefully) get functions triggered by NoSQL database updates and not have to bother with your own webserver.
This is already doable with dynamodb, or kinesis streams. Or google's dataflow. Using wasm just makes that more seamless (and faster).
 I see that CloudFlare has support for webassembly in their 
 workers, but for Google Functions I only see Node10, but maybe 
 they can run webassembly as well? I haven't found anything 
 definitive on it though...
Node has good wasm support, I don't know how you would get the wasm binary in, but it probably can be done.
 Instead of having docker containers you just compile to wasm, 
 which will be pretty small and can boot in (sub) milli-seconds 
 (plus they don't necessarily need a linux host kernel running 
 and can run it closer to the hypervisor.)
Yes, but the biggest potential I see is when you don't have to set up servers to process data.
I rather not setup servers for anything.
 Just throw the data into the distributed database, which 
 triggers a Function that updates other parts of the database 
 and then triggers another function that push the resulting PDF 
 (or whatever) to a service that serves the files directly (i.e. 
 cached close to the user like CloudFlare).
You don't have to wait for that. That future is already here. The in and output could also be distributed storage, event streams or some queue. The problem, however, is often when using those tools you get pushed into a small set of supported programming languages. Like AWS' glue that focuses on Scala or Python, or google's functions that only support js/python and go. Understandable, but I rather choose my own language. Wasm makes that possible.
Nov 25 2019
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 November 2019 at 13:52:29 UTC, Sebastiaan Koppe 
wrote:
 You don't have to wait for that. That future is already here. 
 The in and output could also be distributed storage, event 
 streams or some queue.
Yes, I am most familiar with Google Cloud. Earlier this year Google Functions was not available in European datacenters IIRC, but now it is at least available in London and Belgium. So things are moving in that direction, somewhat slowly. It is annoying to not have Google Functions when working with Google Firebase, so if webworkers is possible then that could make things much better (even for simple things like generating thumbnail images).
 Like AWS' glue that focuses on Scala or Python, or google's 
 functions that only support js/python and go. Understandable, 
 but I rather choose my own language. Wasm makes that possible.
Let's hope there is a way for other services than CloudFlare. CloudFlare Workers look cool, but their KV store has very low propagation guarantees on updates (60 seconds).
Nov 25 2019
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/25/19 7:52 AM, Sebastiaan Koppe wrote:
 So it became clear to me I need to have druntime available. It will 
 allow people to use the (almost) complete set of D features and it opens 
 up some metaprogramming avenues that are closed off right now. With that 
 I will be able to create some nice DSL, in line with JSX/SwiftUI or 
 <insert-your-favorite-declarative-framework>.
 
 There are plenty of opportunities here. It is not unfeasible to connect 
 spasm to Qt, or dlangui, and create a cross-platform UI library, 
 something like flutter.
 
 On the other hand, I am very excited about WebAssembly in general. It is 
 certainly at the beginning of the hype curve and I suspect some very 
 exciting things will appear in the future. Some of them are already here 
 right now. For instance, you can target ARM by compiling D code to wasm 
 and then use wasmer to compile it to ARM. With D connecting itself to 
 the wasm world it exposes itself to a lot of cool things, which we 
 mostly get for free.
 
 As an example, it is just a matter of time before a PaaS provider fully 
 embraces wasm. Instead of having docker containers you just compile to 
 wasm, which will be pretty small and can boot in (sub) milli-seconds 
 (plus they don't necessarily need a linux host kernel running and can 
 run it closer to the hypervisor.)
As someone who does web application development, all of this sounds awesome. I would LOVE to have a real programming language to do the client-side stuff. -Steve
Nov 25 2019
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Thanks for putting this together, it looks very carefully thought out. On this particular part:
 Exceptions can be thrown but not catched. A thrown exception 
 will
 terminate the program. Exceptions are still in the proposal 
 phase.
 When the proposal is accepted exceptions can be fully supported.
This would suggest that there may be some benefit in D providing improved support for return-type-based error propagation (as with `Result` from Rust), no ... ?
Nov 25 2019
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 12:19:30 UTC, Joseph Rushton 
Wakeling wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Thanks for putting this together, it looks very carefully thought out.
Thanks!
 Exceptions can be thrown but not catched. A thrown exception 
 will
 terminate the program. Exceptions are still in the proposal 
 phase.
 When the proposal is accepted exceptions can be fully 
 supported.
This would suggest that there may be some benefit in D providing improved support for return-type-based error propagation (as with `Result` from Rust), no ... ?
Yes, definitely. But what do you mean with improved support? Like better pattern matching over either types?
Nov 25 2019
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 25 November 2019 at 13:00:23 UTC, Sebastiaan Koppe 
wrote:
 Yes, definitely. But what do you mean with improved support? 
 Like better pattern matching over either types?
Yes, that sort of thing. And maybe a move towards trying to use this kind of error handling in newer editions of the standard library (I'm reluctant to push too strongly on that, but I get the impression there is some inclination to move in this direction, as a reflection of wider design trends).
Nov 25 2019
prev sibling next sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
On the GC part. It says "The only unknown part is how to dump the registers to the stack to ensure no pointers are held in the registers only." Please correct me where I'm wrong, but on the level of WebAssembly there are no registers, there is an operand stack outside the address space, there are local variables to the current function, again outside the accessible address space of program's linear memory, and there is the linear memory itself. So scanning the stack becomes a really hard (should I say impossible?) part. What some compilers do is they organize another stack manually in the linear memory and store the values that would otherwise be on the normal stack, there. Which means in case of D you'll have to seriously change the codegen, to change how local variables are stored, and to use a kind of shadow stack for temporaries in expressions that may be pointers. Do you really have a plan about it?
Nov 25 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 25 November 2019 at 18:44:01 UTC, thedeemon wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Please correct me where I'm wrong, but on the level of WebAssembly there are no registers, there is an operand stack outside the address space, there are local variables to the current function, again outside the accessible address space of program's linear memory, and there is the linear memory itself. So scanning the stack becomes a really hard (should I say impossible?) part. What some compilers do is they organize another stack manually in the linear memory and store the values that would otherwise be on the normal stack, there.
Yeah, that accurately describes the situation. I will update the wording in the document to use 'stack', 'shadow stack' (also sometimes called 'user stack') and the local variable. Thanks. One solution that I employed in spasm's experimental gc is to only run it directly from javascript. This way there can't be anything hiding in the stack or in a local variable. Although that approach doesn't work for all use cases.
 Which means in case of D you'll have to seriously change the 
 codegen, to change how local variables are stored, and to use a 
 kind of shadow stack for temporaries in expressions that may be 
 pointers. Do you really have a plan about it?
Well, no, not fully. That is why I said 'unknown'. But there must be a solution somewhere. LLVM already puts pointers to stack or local variables in the shadow stack. As well as for structs-by-val that don't fit the stack. We could adjust LDC to nudge LLVM to maintain live roots on the shadow stack as well. Go's approach is to put everything on the shadow stack. (see: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4/preview#heading=h.mjo1bish3xni) There is also the possibility of a code transformation. Binaryen has a spill-the-pointer pass that effectively gets you go's solution (but only for i32's) (see: https://github.com/WebAssembly/binaryen/blob/master/src/passes/pass.cpp#L310) I am favoring the first option, but I don't know how hard that would be. Will update the document with this info. Thank you for questioning this.
Nov 26 2019
prev sibling next sibling parent reply Thomas Brix <brix brix-verden.dk> writes:
On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
An alternative idea, would be to use emscriptens fork of musl to have a full C-library. AFAIK this includes threading. LLVM is supposed to support TLS in wasm since version 9.
Nov 26 2019
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Tuesday, 26 November 2019 at 09:18:05 UTC, Thomas Brix wrote:
 On Saturday, 23 November 2019 at 09:51:13 UTC, Sebastiaan Koppe 
 wrote:
 This is my proposal for porting D runtime to WebAssembly. I 
 would like to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
An alternative idea, would be to use emscriptens fork of musl to have a full C-library. AFAIK this includes threading. LLVM is supposed to support TLS in wasm since version 9.
Yes, indeed. https://reviews.llvm.org/D64537 gives a good overview. I believe it is best to first actually have a version of druntime on wasm, rather than eagerly pulling in all the latest features. I find the scope I set in the proposal to be quite reasonable. Adding tls, threading and exception handling would be much easier after this work is done and merged. And it would also be something others might want to contribute to.
Nov 26 2019
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2019-11-23 09:51:13 +0000, Sebastiaan Koppe said:

 This is my proposal for porting D runtime to WebAssembly. I would like 
 to ask you to review it. You can find it here: 
 https://gist.github.com/skoppe/7617ceba6afd67b2e20c6be4f922725d
Not sure if you are aware of this: https://wasmtime.dev/ Maybe it helps or gives some inspiration. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Nov 27 2019