www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Static ctors in wasm

reply "H. S. Teoh" <hsteoh qfbox.info> writes:
How are static ctors actually implemented in D?  I was under the
impression that there's some kind of table stored in the executable that
the druntime startup code traverses. But apparently this is
OS-dependent??  I'm trying to get static ctors to work in wasm, but
can't find any way of making it work. The druntime code that I looked
into all hook into stuff dependent on the executable format, and
obviously there isn't anything for wasm (yet).

Is this even possible in wasm?  Or am I missing something obvious?


T

-- 
A program should be written to model the concepts of the task it performs
rather than the physical world or a process because this maximizes the
potential for it to be applied to tasks that are conceptually similar and, more
important, to tasks that have not yet been conceived. -- Michael B. Allen
Feb 08
next sibling parent reply kinke <noone nowhere.com> writes:
On Friday, 9 February 2024 at 00:02:12 UTC, H. S. Teoh wrote:

 How are static ctors actually implemented in D?  I was under 
 the impression that there's some kind of table stored in the 
 executable that the druntime startup code traverses. But 
 apparently this is OS-dependent??  I'm trying to get static 
 ctors to work in wasm, but can't find any way of making it 
 work. The druntime code that I looked into all hook into stuff 
 dependent on the executable format, and obviously there isn't 
 anything for wasm (yet).


It primarily depends on the binary format, and is a cooperation 
between compiler and druntime. Usually, the compiler emits 
non-linker-strippable pointers to compiler-generated `ModuleInfo` 
structs into a special section (`__minfo`, `.minfo` or so), one 
for each .d module getting compiled into a specific object file. 
After linking to an executable/shared library, the ModuleInfo 
pointers of *all* .d modules of *each linked* object file are 
then stored consecutively in that named section of the data 
segment. This way, druntime can then reflect on all D 
modules/ModuleInfos linked into a binary, by getting that data 
range of ModuleInfo pointers (e.g., via linker-generated 
`__{start,stop}___minfo` bracketing symbols for ELF). And from 
there, module ctors, unittests etc. can be inferred.


 Is this even possible in wasm?  Or am I missing something 
 obvious?


LDC has a fallback for exotic platforms, where it uses a linked 
list, and compiler-generated CRT-constructor functions which 
insert the ModuleInfo pointers at program initialization (in 
undefined order), invoked by the C runtime before C main(). So if 
wasm doesn't support named sections/data ranges, but would 
'implicitly' support initializer functions (CRT constructors), 
that could be a potential route to take.
Feb 08
parent reply kinke <noone nowhere.com> writes:
On Friday, 9 February 2024 at 00:48:47 UTC, kinke wrote:

 So if wasm doesn't support named sections/data ranges


Oh well, looks like wasm-ld (i.e., lld) supports exactly the same 
magic `__{start,stop}_*` symbols as for ELF: 
https://github.com/llvm/llvm-project/issues/55839
Feb 08
parent reply kinke <noone nowhere.com> writes:
On Friday, 9 February 2024 at 01:17:26 UTC, kinke wrote:

 Oh well, looks like wasm-ld (i.e., lld) supports exactly the 
 same magic `__{start,stop}_*` symbols as for ELF: 
 https://github.com/llvm/llvm-project/issues/55839


Oh man, I've even added an LDC test for this, but almost 4 years 
ago and so forgot. :D

https://github.com/ldc-developers/ldc/blob/3c21924705aae83f0c16bfe54e673953671afe58/tests/codegen/wasi.d#L22-L39
Feb 08
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Friday, 9 February 2024 at 01:22:28 UTC, kinke wrote:

 Oh man, I've even added an LDC test for this, but almost 4 
 years ago and so forgot. :D

 https://github.com/ldc-developers/ldc/blob/3c21924705aae83f0c16bfe54e673953671afe58/tests/codegen/wasi.d#L22-L39


Yep, I remember asking you about for my druntime wasm port.
Feb 08
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Feb 09, 2024 at 01:22:28AM +0000, kinke via Digitalmars-d wrote:

 On Friday, 9 February 2024 at 01:17:26 UTC, kinke wrote:

 Oh well, looks like wasm-ld (i.e., lld) supports exactly the same
 magic `__{start,stop}_*` symbols as for ELF:
 https://github.com/llvm/llvm-project/issues/55839


 
 Oh man, I've even added an LDC test for this, but almost 4 years ago
 and so forgot. :D
 
 https://github.com/ldc-developers/ldc/blob/3c21924705aae83f0c16bfe54e673953671afe58/tests/codegen/wasi.d#L22-L39


Hmm, I'm running into a problem here.  Following the above link I
declared the symbols like this:

```d
extern(C) extern __gshared
{
    void* __start__minfo;
    void* __stop__minfo;
}
```

However, when I check their values at runtime:

```d
export void _wasm_start() // this is called from my JS bridge
{
    writefln("__start__minfo=0x%x", &__start__minfo);
    writefln("__end__minfo=0x%x", &__stop__minfo);
}
```

(I have a bare-bones version of writefln running for debugging purposes.)

The output says:

````
__start__minfo=0x0
__end__minfo=0x%0
````

What gives?

For reference, I'm using LDC 1.36.0 on x86_64--linux-gnu, with these
compile flags:

	-mtriple=wasm32-unknown-unknown-wasm -Iplatform/wasm -O2 -L-allow-undefined
-allinst -linkonce-templates --fvisibility=hidden

I had --fno-rtti before and it didn't work; I removed it but it still
made no difference. I guess it's unrelated to RTTI.  Also, I tried
removing --fvisibility-hidden but it also made no difference.

What am I doing wrong?


T

-- 
Give a man a fish, and he eats once. Teach a man to fish, and he will sit
forever.
Feb 15
next sibling parent reply David Gileadi <gileadisNOSPM gmail.com> writes:
On 2/15/24 2:45 PM, H. S. Teoh wrote:

 On Fri, Feb 09, 2024 at 01:22:28AM +0000, kinke via Digitalmars-d wrote:

 On Friday, 9 February 2024 at 01:17:26 UTC, kinke wrote:

 Oh well, looks like wasm-ld (i.e., lld) supports exactly the same
 magic `__{start,stop}_*` symbols as for ELF:
 https://github.com/llvm/llvm-project/issues/55839


 Oh man, I've even added an LDC test for this, but almost 4 years ago
 and so forgot. :D

 https://github.com/ldc-developers/ldc/blob/3c21924705aae83f0c16bfe54e673953671afe58/tests/codegen/wasi.d#L22-L39


 
 Hmm, I'm running into a problem here.  Following the above link I
 declared the symbols like this:
 
 ```d
 extern(C) extern __gshared
 {
      void* __start__minfo;
      void* __stop__minfo;
 }
 ```
 
 However, when I check their values at runtime:
 
 ```d
 export void _wasm_start() // this is called from my JS bridge
 {
      writefln("__start__minfo=0x%x", &__start__minfo);
      writefln("__end__minfo=0x%x", &__stop__minfo);
 }
 ```
 
 (I have a bare-bones version of writefln running for debugging purposes.)
 
 The output says:
 
 ````
 __start__minfo=0x0
 __end__minfo=0x%0
 ````
 
 What gives?
 
 For reference, I'm using LDC 1.36.0 on x86_64--linux-gnu, with these
 compile flags:
 
 	-mtriple=wasm32-unknown-unknown-wasm -Iplatform/wasm -O2 -L-allow-undefined
-allinst -linkonce-templates --fvisibility=hidden
 
 I had --fno-rtti before and it didn't work; I removed it but it still
 made no difference. I guess it's unrelated to RTTI.  Also, I tried
 removing --fvisibility-hidden but it also made no difference.
 
 What am I doing wrong?


While I have zero knowledge of how LDC handles WASM, I did notice one 
difference between your code and the example in the link: the definition 
at the link has three underscores before the `minfo` whereas your code 
has only two.
Feb 15
parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Feb 15, 2024 at 03:04:28PM -0700, David Gileadi via Digitalmars-d wrote:

 On 2/15/24 2:45 PM, H. S. Teoh wrote:


[...]

 ```d
 extern(C) extern __gshared
 {
      void* __start__minfo;
      void* __stop__minfo;
 }
 ```




[...]

 While I have zero knowledge of how LDC handles WASM, I did notice one
 difference between your code and the example in the link: the
 definition at the link has three underscores before the `minfo`
 whereas your code has only two.


Whoa... I totally missed that! :-O

However, correcting it to 3 underscores still produced 0. :-(   What's
going on??


T

-- 
I am not young enough to know everything. -- Oscar Wilde
Feb 15
prev sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
Start back at the basics, look at the object file dump, is ModuleInfo 
being generated?

If not, have you copied ModuleInfo over?
Feb 15
parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Feb 16, 2024 at 02:28:25PM +1300, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:

 Start back at the basics, look at the object file dump, is ModuleInfo
 being generated?


It's missing from the object file.



 If not, have you copied ModuleInfo over?


Does it need to be defined in object.d?


T

-- 
Doctor: "Sir, I’m afraid your DNA is backwards." / Patient: "And?"
Feb 15
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 16/02/2024 2:57 PM, H. S. Teoh wrote:

 On Fri, Feb 16, 2024 at 02:28:25PM +1300, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:

 Start back at the basics, look at the object file dump, is ModuleInfo
 being generated?


 
 It's missing from the object file.
 
 

 If not, have you copied ModuleInfo over?


 
 Does it need to be defined in object.d?
 
 
 T


Yes.

https://github.com/dlang/dmd/blob/a952831fdb2877199c8eda07292757a0c5c29a1a/compiler/src/dmd/dmsc.d#L82
Feb 15
parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Feb 16, 2024 at 03:08:06PM +1300, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:

 On 16/02/2024 2:57 PM, H. S. Teoh wrote:

 On Fri, Feb 16, 2024 at 02:28:25PM +1300, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:

 Start back at the basics, look at the object file dump, is
 ModuleInfo being generated?


 
 It's missing from the object file.
 
 

 If not, have you copied ModuleInfo over?


 
 Does it need to be defined in object.d?




[...]

 Yes.
 
 https://github.com/dlang/dmd/blob/a952831fdb2877199c8eda07292757a0c5c29a1a/compiler/src/dmd/dmsc.d#L82


Figures!  I was under the wrong impression that I only need to define it
for code that actually needs to traverse it.  Thanks for the tip!!

I copy-n-pasted ModuleInfo from the real (non-wasm) object.d, and now
I'm finally getting an address for __start___minfo.  However,
__end___minfo for some reason still shows up as 0x0?  I must be missing
something else?


T

-- 
Famous last words: I *think* this will work...
Feb 15
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 16/02/2024 5:45 PM, H. S. Teoh wrote:

 On Fri, Feb 16, 2024 at 03:08:06PM +1300, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:

 On 16/02/2024 2:57 PM, H. S. Teoh wrote:

 On Fri, Feb 16, 2024 at 02:28:25PM +1300, Richard (Rikki) Andrew Cattermole
via Digitalmars-d wrote:

 Start back at the basics, look at the object file dump, is
 ModuleInfo being generated?


 It's missing from the object file.



 If not, have you copied ModuleInfo over?


 Does it need to be defined in object.d?




 [...]

 Yes.

 https://github.com/dlang/dmd/blob/a952831fdb2877199c8eda07292757a0c5c29a1a/compiler/src/dmd/dmsc.d#L82


 
 Figures!  I was under the wrong impression that I only need to define it
 for code that actually needs to traverse it.  Thanks for the tip!!
 
 I copy-n-pasted ModuleInfo from the real (non-wasm) object.d, and now
 I'm finally getting an address for __start___minfo.  However,
 __end___minfo for some reason still shows up as 0x0?  I must be missing
 something else?
 
 
 T


Great!

Unfortunately I don't know how the lists created by the linker work.

But I do know that they are linker specific.
Feb 15
next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Feb 16, 2024 at 06:02:46PM +1300, Richard (Rikki) Andrew Cattermole via
Digitalmars-d wrote:

 
 On 16/02/2024 5:45 PM, H. S. Teoh wrote:


[...]

 I copy-n-pasted ModuleInfo from the real (non-wasm) object.d, and
 now I'm finally getting an address for __start___minfo.  However,
 __end___minfo for some reason still shows up as 0x0?  I must be
 missing something else?




[...]

 Great!
 
 Unfortunately I don't know how the lists created by the linker work.
 
 But I do know that they are linker specific.


Hmm, the module traversal code seems to depend on SectionGroup.  Does
this need to be imported by object.d too?  Currently I only have it in a
submodule in the runtime.


T

-- 
If you look at a thing nine hundred and ninety-nine times, you are perfectly
safe; if you look at it the thousandth time, you are in frightful danger of
seeing it for the first time. -- G. K. Chesterton
Feb 15
prev sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 16/02/2024 6:02 PM, Richard (Rikki) Andrew Cattermole wrote:

 Great!
 
 Unfortunately I don't know how the lists created by the linker work.
 
 But I do know that they are linker specific.


I did some searching.

``__start_section`` and ``__stop_section`` are generated by the linker 
automatically for a given section.

https://stackoverflow.com/a/48550485

I have been able to replicate what it should be doing using run.dlang.io.

```d
import std;
import core.attribute;

void main()
{
     writeln([var1, var2, var3], __start_sect_test, __stop_sect_test);
}

__gshared extern(C) extern {
     immutable int* __start_sect_test;
     immutable int* __stop_sect_test;
}

__gshared {
      (ldc.attributes.section("sect_test"))
     int var1 = 72;
      (ldc.attributes.section("sect_test"))
     int var2 = 43;
      (ldc.attributes.section("sect_test"))
     int var3 = 59;
}
```

Now you'll have something to compare the ``__minfo`` symbols to.

One thing to check is that your triple is for the binary wasm otherwise 
it might not be using sections.

https://github.com/ldc-developers/ldc/blob/6ede9a4fdfd04724fc28a60e6460993d8344136f/gen/modules.cpp#L105
Feb 15
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
https://github.com/llvm/llvm-project/issues/23280

https://reviews.llvm.org/D64148

This is consistently appearing to be related to GC'ing of sections.

I.e. try ``--no-gc-sections``.

Alternatively it may be better to swap over to the linked list approach, 
but you'll need to compile ldc to switch it over.
Feb 16
prev sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Feb 15, 2024 at 01:45:50PM -0800, H. S. Teoh via Digitalmars-d wrote:
[...]

 The output says:
 
 ````
 __start__minfo=0x0
 __end__minfo=0x%0
 ````


[...]

Ack, stray character got stuck in there somehow. Here's a pristine
copy-n-paste from the output log:

````
__start__minfo=0x0
__end__minfo=0x0

````


T

-- 
We are in class, we are supposed to be learning, we have a teacher... Is it too
much that I expect him to teach me??? -- RL
Feb 15
prev sibling next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Friday, 9 February 2024 at 00:02:12 UTC, H. S. Teoh wrote:

 Is this even possible in wasm?  Or am I missing something 
 obvious?


 T


Might I direct you to my past efforts of a druntime port? This is 
a good entry point https://github.com/skoppe/ldc/tree/wasm as it 
contains the changes for LDC and submodules for Phobos and 
druntime.

I got everything working, except for things that aren't supported 
like exceptions and fibers.

Its just 3 year behind on master.

The other issue is the GC not seeing all pointers hence freeing 
too much. As mentioned in another thread this can be solved by 
spilling the pointers to the shadow stack, which involves either 
changes to LDC/llvm or doing a post build step. Nowadays there is 
binaryen which reimplemented the required pass. I haven't been 
able to test it but others have used it with a similar Böhm GC.
Feb 08
prev sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Feb 09, 2024 at 01:17:26AM +0000, kinke via Digitalmars-d wrote:

 On Friday, 9 February 2024 at 00:48:47 UTC, kinke wrote:

 So if wasm doesn't support named sections/data ranges


 
 Oh well, looks like wasm-ld (i.e., lld) supports exactly the same magic
 `__{start,stop}_*` symbols as for ELF:
 https://github.com/llvm/llvm-project/issues/55839


Oh cool!

But is ModuleInfo emitted when compiling with --fno-rtti? I adapted
Sebastiaan Koppe's code to traverse module info at runtime, but I'm
getting an empty list... or maybe I missed something, as usual.


On Fri, Feb 09, 2024 at 07:21:04AM +0000, Sebastiaan Koppe via Digitalmars-d
wrote:
[...]

 Might I direct you to my past efforts of a druntime port? This is a
 good entry point https://github.com/skoppe/ldc/tree/wasm as it
 contains the changes for LDC and submodules for Phobos and druntime.


Thanks, that was very helpful!



 I got everything working, except for things that aren't supported like
 exceptions and fibers.


Hmm.  I wonder if it's possible to implement exceptions without stack
unwinding. Like using sumtypes or a dedicated error flag/register under
the hood, with the compiler transparently inserting the necessary
branches after calling a non-nothrow function.  Would require a major
ABI change though, may not be feasible.  But this could be a good
opportunity to experiment with alternative exception implementations.


[...]

 The other issue is the GC not seeing all pointers hence freeing too
 much. As mentioned in another thread this can be solved by spilling
 the pointers to the shadow stack, which involves either changes to
 LDC/llvm or doing a post build step. Nowadays there is binaryen which
 reimplemented the required pass. I haven't been able to test it but
 others have used it with a similar Böhm GC.


I'll look into it when I get to that point. :-D  Currently, for my
purposes, it's Good Enough(tm) to preallocate everything in a setup
function and then just use a bump-the-pointer allocator per callback,
resetting to last pointer location afterwards. Sorta like a poor man's
region allocator. As long as no pointers persist beyond the callback I
should be OK.


T

-- 
Ph.D. = Permanent head Damage
Feb 09