www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What's the current status of WASM/WASI support?

reply ChrisG <joe hotmail.com> writes:
In 2019, there was a blog post that indicated a strong push for 
wasm support was afoot. I've had a hard time pulling together 
information since then. I was curious if anyone has a good 
general perspective on the state of D wasm support. I can't tell 
if it's a mostly dead subject. It looks like betterC support is 
generally there, but I'm not sure about runtime or library 
support. Specifically, I was interested in:

- What's the runtime support story (can I use GC)?
- What's the phobos support story?
- Are there alternate general purpose libraries folks have 
settled on for wasm development? (spasm?)
- Anything about WASI or Emscripten?

Thanks.
Mar 14 2023
next sibling parent reply Hipreme <msnmancini hotmail.com> writes:
On Tuesday, 14 March 2023 at 23:45:12 UTC, ChrisG wrote:

 In 2019, there was a blog post that indicated a strong push for 
 wasm support was afoot. I've had a hard time pulling together 
 information since then. I was curious if anyone has a good 
 general perspective on the state of D wasm support. I can't 
 tell if it's a mostly dead subject. It looks like betterC 
 support is generally there, but I'm not sure about runtime or 
 library support. Specifically, I was interested in:

 - What's the runtime support story (can I use GC)?
 - What's the phobos support story?
 - Are there alternate general purpose libraries folks have 
 settled on for wasm development? (spasm?)
 - Anything about WASI or Emscripten?

 Thanks.



Hipreme Engine has been ported to WASM.
There's a way to include the custom runtime I'm using right now. 
You can use almost the complete runtime, although it does not 
collect. If you're not leaking memory you're fine.

You can check the announcement post here. Just do your part and 
in the future we'll be able to have some official D runtime. For 
the time being it can work for 90% of the use cases.
https://forum.dlang.org/post/gtmlwwkovmuphrmfgvtf forum.dlang.org
Mar 14 2023
parent reply ChrisG <joe hotmail.com> writes:
On Wednesday, 15 March 2023 at 03:44:21 UTC, Hipreme wrote:


 Hipreme Engine has been ported to WASM.
 There's a way to include the custom runtime I'm using right 
 now. You can use almost the complete runtime, although it does 
 not collect. If you're not leaking memory you're fine.


Wow. That's an impressive feat.

This is the runtime you're using for wasm?
https://github.com/MrcSnm/webassembly/tree/f60b02feca726ab5c01d1bd0d888a9a72282ab9b
Mar 15 2023
parent Hipreme <msnmancini hotmail.com> writes:
On Wednesday, 15 March 2023 at 17:35:42 UTC, ChrisG wrote:

 On Wednesday, 15 March 2023 at 03:44:21 UTC, Hipreme wrote:


 Hipreme Engine has been ported to WASM.
 There's a way to include the custom runtime I'm using right 
 now. You can use almost the complete runtime, although it does 
 not collect. If you're not leaking memory you're fine.


 Wow. That's an impressive feat.

 This is the runtime you're using for wasm?
 https://github.com/MrcSnm/webassembly/tree/f60b02feca726ab5c01d1bd0d888a9a72282ab9b


Yes, you can follow the code from my engine as a guide into how 
to integrate the custom runtime in your project.

Also, if you wish to try dub libraries compiled with that custom 
runtime, you'll need to set the `DFLAGS` variable before calling 
`dub`. This is the reason you'll find plenty of shell and batch 
files.

Hipreme Engine has its own standard library as I prefer having 
this kind of control for testing on every platform, and also not 
even depending on libc. Although that happens, I still have a 
minimal libc port localized on `modules/d_std` of my engine.

If you wish to work with WASM, you'll need to deal with some 
problems such as loading files, getting directory files list, or 
those things which my engine has already dealt with as it needs a 
FS abstraction for every platform.

It is quite some work but well, everything has been done, 
specially since I used +- 1 month to get this runtime to that 
level. This custom runtime also works on PS Vita.
Mar 15 2023
prev sibling parent reply redthing1 <redthing1 alt.icu> writes:
On Tuesday, 14 March 2023 at 23:45:12 UTC, ChrisG wrote:

 In 2019, there was a blog post that indicated a strong push for 
 wasm support was afoot. I've had a hard time pulling together 
 information since then. I was curious if anyone has a good 
 general perspective on the state of D wasm support. I can't 
 tell if it's a mostly dead subject. It looks like betterC 
 support is generally there, but I'm not sure about runtime or 
 library support. Specifically, I was interested in:

 - What's the runtime support story (can I use GC)?
 - What's the phobos support story?
 - Are there alternate general purpose libraries folks have 
 settled on for wasm development? (spasm?)
 - Anything about WASI or Emscripten?

 Thanks.


I was wondering actually, has anyone experimented with WASI + D? 
I am interested in looking into it.
Oct 15 2023
next sibling parent reply Guillaume Piolat <first.name gmail.com> writes:
On Monday, 16 October 2023 at 04:11:19 UTC, redthing1 wrote:

 I was wondering actually, has anyone experimented with WASI + 
 D? I am interested in looking into it.


You can run WASI preview1 functions in LDC.
https://gist.github.com/p0nce/fccc816e3e98f5001d0015c916aaba3e

WASI preview1 has the POSIX functions needed to implement a 
"libc-like" layer on top.
Then the work begin of implementing C stdlib on top of WASI.

Either druntime has to provide that libc, either we need one 
built over WASI.
Else custom alternatives of druntime will do this to run on WASM.
Oct 16 2023
parent redthing1 <redthing1 alt.icu> writes:
On Monday, 16 October 2023 at 14:13:05 UTC, Guillaume Piolat 
wrote:

 On Monday, 16 October 2023 at 04:11:19 UTC, redthing1 wrote:

 I was wondering actually, has anyone experimented with WASI + 
 D? I am interested in looking into it.


 You can run WASI preview1 functions in LDC.
 https://gist.github.com/p0nce/fccc816e3e98f5001d0015c916aaba3e

 WASI preview1 has the POSIX functions needed to implement a 
 "libc-like" layer on top.
 Then the work begin of implementing C stdlib on top of WASI.

 Either druntime has to provide that libc, either we need one 
 built over WASI.
 Else custom alternatives of druntime will do this to run on 
 WASM.


That's awesome!

I think that providing that core library should happen in 
druntime. Not sure where to start.
Nov 03 2023
prev sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Monday, 16 October 2023 at 04:11:19 UTC, redthing1 wrote:

 On Tuesday, 14 March 2023 at 23:45:12 UTC, ChrisG wrote:

 In 2019, there was a blog post that indicated a strong push 
 for wasm support was afoot. I've had a hard time pulling 
 together information since then. I was curious if anyone has a 
 good general perspective on the state of D wasm support. I 
 can't tell if it's a mostly dead subject. It looks like 
 betterC support is generally there, but I'm not sure about 
 runtime or library support. Specifically, I was interested in:

 - What's the runtime support story (can I use GC)?
 - What's the phobos support story?
 - Are there alternate general purpose libraries folks have 
 settled on for wasm development? (spasm?)
 - Anything about WASI or Emscripten?

 Thanks.


 I was wondering actually, has anyone experimented with WASI + 
 D? I am interested in looking into it.


The d runtime port I did years ago used wasi and the wasi-libc. I 
ported the whole of druntime (aside from things like catching 
exception, fibers, etc.) but got stuck on the GC.

The trouble is that the GC is unable to scan the wasm stack (let 
alone the registers) so you have to spill pointers to the shadow 
stack. I know its possible, but never managed to find time to do 
it.
Nov 03 2023
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Friday, 3 November 2023 at 23:36:45 UTC, Sebastiaan Koppe 
wrote:

 The trouble is that the GC is unable to scan the wasm stack 
 (let alone the registers) so you have to spill pointers to the 
 shadow stack. I know its possible, but never managed to find 
 time to do it.


An even better way is to defer the scan until you return to the 
javascript event loop, since then you guarantee the wasm stack is 
empty.

I've long suspected that would work but never implemented it, and 
read a thing just earlier today confirming it would work (but now 
webasm has a new gc thing that works differently google is trying 
to push: https://v8.dev/blog/wasm-gc-porting )
Nov 03 2023
parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Friday, 3 November 2023 at 23:43:59 UTC, Adam D Ruppe wrote:

 On Friday, 3 November 2023 at 23:36:45 UTC, Sebastiaan Koppe 
 wrote:

 The trouble is that the GC is unable to scan the wasm stack 
 (let alone the registers) so you have to spill pointers to the 
 shadow stack. I know its possible, but never managed to find 
 time to do it.


 An even better way is to defer the scan until you return to the 
 javascript event loop, since then you guarantee the wasm stack 
 is empty.

 I've long suspected that would work but never implemented it, 
 and read a thing just earlier today confirming it would work 
 (but now webasm has a new gc thing that works differently 
 google is trying to push: https://v8.dev/blog/wasm-gc-porting )


It will work as long as there is a JS loop to return to where you 
can inject a call to collect. That is not always the case though. 
There are plenty of wasm runtimes where you have no such control.

The wasm gc is interesting. I have been following it since it's 
inception and it's great to see it arriving. Haven't had time to 
play with it, but I expect it to get traction.
Nov 04 2023
prev sibling parent reply Johan <j j.nl> writes:
On Friday, 3 November 2023 at 23:36:45 UTC, Sebastiaan Koppe 
wrote:

 On Monday, 16 October 2023 at 04:11:19 UTC, redthing1 wrote:

 On Tuesday, 14 March 2023 at 23:45:12 UTC, ChrisG wrote:

 In 2019, there was a blog post that indicated a strong push 
 for wasm support was afoot. I've had a hard time pulling 
 together information since then. I was curious if anyone has 
 a good general perspective on the state of D wasm support. I 
 can't tell if it's a mostly dead subject. It looks like 
 betterC support is generally there, but I'm not sure about 
 runtime or library support. Specifically, I was interested in:

 - What's the runtime support story (can I use GC)?
 - What's the phobos support story?
 - Are there alternate general purpose libraries folks have 
 settled on for wasm development? (spasm?)
 - Anything about WASI or Emscripten?

 Thanks.


 I was wondering actually, has anyone experimented with WASI + 
 D? I am interested in looking into it.


 The d runtime port I did years ago used wasi and the wasi-libc. 
 I ported the whole of druntime (aside from things like catching 
 exception, fibers, etc.) but got stuck on the GC.


It is really a pity that you (or someone else) never upstreamed 
this (or related) work. I feel it is good work going to waste :/
Let's not strive for perfect, strive for improvement. And the 
barrier for improvement is pretty low current with a completely 
non-working druntime with wasm (with or without wasi)...!

The separate minimum druntime work is valuable, but could be much 
more valuable if it became part of the standard druntime.

The WASM work always makes me excited, although I've never used 
it.
Thanks!
   Johan
Nov 05 2023
next sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Sunday, 5 November 2023 at 13:29:32 UTC, Johan wrote:

 On Friday, 3 November 2023 at 23:36:45 UTC, Sebastiaan Koppe 
 wrote:

 [...]


 It is really a pity that you (or someone else) never upstreamed 
 this (or related) work. I feel it is good work going to waste :/
 Let's not strive for perfect, strive for improvement. And the 
 barrier for improvement is pretty low current with a completely 
 non-working druntime with wasm (with or without wasi)...!

 The separate minimum druntime work is valuable, but could be 
 much more valuable if it became part of the standard druntime.

 The WASM work always makes me excited, although I've never used 
 it.
 Thanks!
   Johan


Agreed, wasm could be really useful for D
Nov 05 2023
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Sunday, 5 November 2023 at 13:36:00 UTC, Imperatorn wrote:

 On Sunday, 5 November 2023 at 13:29:32 UTC, Johan wrote:

 On Friday, 3 November 2023 at 23:36:45 UTC, Sebastiaan Koppe 
 wrote:

 [...]


 It is really a pity that you (or someone else) never 
 upstreamed this (or related) work. I feel it is good work 
 going to waste :/
 Let's not strive for perfect, strive for improvement. And the 
 barrier for improvement is pretty low current with a 
 completely non-working druntime with wasm (with or without 
 wasi)...!

 The separate minimum druntime work is valuable, but could be 
 much more valuable if it became part of the standard druntime.

 The WASM work always makes me excited, although I've never 
 used it.
 Thanks!
   Johan


 Agreed, wasm could be really useful for D


I might take a stab at it next week. I still have covid so 
currently I'm not so productive though.

If you're interested in working on it together next week:
https://discord.com/invite/s5VfkjXT
Nov 05 2023
prev sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Sunday, 5 November 2023 at 13:29:32 UTC, Johan wrote:

 On Friday, 3 November 2023 at 23:36:45 UTC, Sebastiaan Koppe 
 wrote:

 The d runtime port I did years ago used wasi and the 
 wasi-libc. I ported the whole of druntime (aside from things 
 like catching exception, fibers, etc.) but got stuck on the GC.


 It is really a pity that you (or someone else) never upstreamed 
 this (or related) work. I feel it is good work going to waste :/
 Let's not strive for perfect, strive for improvement. And the 
 barrier for improvement is pretty low current with a completely 
 non-working druntime with wasm (with or without wasi)...!

 The separate minimum druntime work is valuable, but could be 
 much more valuable if it became part of the standard druntime.

 The WASM work always makes me excited, although I've never used 
 it.
 Thanks!
   Johan


The work _is_ on github though 
https://github.com/skoppe/druntime/tree/wasm but I agree its 
unfortunate it never got finished.

I did notice binaryen added a spillpointers pass which might be a 
good way to get over the GC hump, 
https://github.com/WebAssembly/binaryen/blob/main/src/passes/SpillPointers.cpp
Nov 05 2023