www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Leave GC collection to the user of the D library?

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
tl;dr I am scared of non-D programs calling my D library functions from 
foreign threads. So, I am planning on asking the user to trigger 
collection themselves by calling a collection function of my library. Crazy?

I've had serious issues bringing up a D library in a foreign 
environment: Python modules loading a C++ library, which in turn uses 
our D library. There were segmentation failures when loading this .so.

One workaround was to start the GC in disabled state with the following 
global variable defined in the library.

   extern(C) __gshared string[] rt_options = [ "gcopt=disable:1" ];

After that, the GC is enabled inside the library's initialization 
function with the following command.

     GC.enable();

That workaround seemed to be sufficient to load the library 
successfully. Unfortunately, that was not enough to weed out all issues 
related to libraries because this library itself loads other D 
libraries. All of this caused sporadic issues. (My brain is too fried to 
even remember what was a cause, what was a usable workaround, etc. 
Sometimes I wasted days chasing a solution while using a test, which had 
nothing to do with the solution. I would change the code, test, no go; 
repeat, no go. It turns out, my test was unrelated. Argh!)

So, we came up with a drastic solution: Since all this code works just 
fine in a pure D environment, make the library as thin as possible; the 
library starts a daemon that is written in D with all the functionality. 
The library merely dispatches the requests to that daemon.

The library starts the daemon with pipeProcess(); pipes are used for 
dispatching requests and shared memory is used for large data. This idea 
"worked like a charm." Phew!

However, dispatching of the requests to the daemon is performed by a 
single library thread in a blocked manner: When a request is written to 
the pipe, the response is read back (blocked) and the result is returned 
to the user of the library function.

So now we want to use this functionality from multiple threads. Yikes! I 
had so much trouble with foreign threads calling D libraries in the past 
that I get scared. (In one case it was Java threads.) There are so many 
dimensions to play with, hypothesizing a correct solution has been 
exhausting. I was never sure whether the issues were e.g. with 
threadAttachThis() or my misusing it.

Ok... How about this idea that would allow this library to be used from 
multiple threads: Leave the GC disabled with that 'rt_options' variable 
above and don't enable it in the library initialization function (this 
is not init(); rather, a function that the user calls explicitly). 
Instead, add yet another library API function for collecting garbage. I 
can document that no other thread is allowed to call any other function 
of the library when this collection function is called. They can do this 
either at strategic points that they know no other thread is using the 
library or they can use a mutex.

Another trivial function that I add can relay GC stats to the user so 
that they can decide to call the GC if the allocations have been high 
enough.

This would allow the user start as many foreign threads as possible. 
Right? Is this sane? Is collection the only issue here? Do foreign 
threads still need to call threadAttachThis()? What happens if they don't?

I feel so hopeless that in the past, I even thought about and 
experimented with banning the user from starting threads on their own. 
Rather, they would call my library on a posix compatible thread API and 
create their threads through me, which happens to be a D thread, so no 
thread would be a "foreign thread" and everything would work just fine. 
I haven't deployed this crazy idea (yet).

Ali
May 08
next sibling parent reply Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Sunday, 9 May 2021 at 03:25:06 UTC, Ali Çehreli wrote:
 The library starts the daemon with pipeProcess(); pipes are 
 used for dispatching requests and shared memory is used for 
 large data. This idea "worked like a charm." Phew!
Why don't you do this in a manner that works with multiple threads? Block calling threads with semaphores, wake them up when results are ready. (Sidenote, if gc was task limited then this would not have been an issue...)
May 09
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/9/21 12:59 AM, Ola Fosheim Grostad wrote:

 On Sunday, 9 May 2021 at 03:25:06 UTC, Ali =C3=87ehreli wrote:
 The library starts the daemon with pipeProcess(); pipes are used for
 dispatching requests and shared memory is used for large data. This
 idea "worked like a charm." Phew!
Why don't you do this in a manner that works with multiple threads?
That's what I want to do but those threads are created by the user,=20 unknown to the D GC. Although there are ways of dealing with that=20 case[1], I am questioning whether the library can disable GC and lets=20 the user manage GC collections explicitly (cooperatively?). Ali [1] https://dlang.org/phobos/core_memory.html
May 09
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 9 May 2021 at 10:27:04 UTC, Ali Çehreli wrote:
 That's what I want to do but those threads are created by the 
 user, unknown to the D GC. Although there are ways of dealing 
 with that case[1], I am questioning whether the library can 
 disable GC and lets the user manage GC collections explicitly 
 (cooperatively?).
Wouldn't this be annoying for the user of the library? (Or maybe you only have a handful users?). I have to admit I haven't read about IPC in a decade or so, but it seems to me that there are many options? Like, if you have N cores, create N worker threads in the daemon and create N pipes? Then set the count of the semaphore to N, so when the semaphore hits 0 all pipes are in use, and the calling thread will wait in the API stub (wrapper function) until a worker thread is available? Kinda clunky, but fits your model... I guess. But maybe you want to get rid of the daemon? In that case I probably would just use mailboxes and wrap the API in futures if the work isn't fine-grained, but I don't know what kind if library you are making...
May 09
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 9 May 2021 at 10:42:19 UTC, Ola Fosheim Grøstad wrote:
 semaphore hits 0 all pipes are in use, and the calling thread 
 will wait in the API stub (wrapper function) until a worker 
 thread is available? Kinda clunky, but fits your model... I 
 guess.
So I imagine you could structure your API wrapper something like this: semaphore.init(N) wrapped_api_call(){ semaphore.down(1) dostuff() api_call() semaphore.up(1) if (memory pressure){ semaphore.down(N) gc_collect() semaphore.up(N) } } I dunno.
May 09
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/9/21 3:42 AM, Ola Fosheim Gr=C3=B8stad wrote:

 On Sunday, 9 May 2021 at 10:27:04 UTC, Ali =C3=87ehreli wrote:
 whether the library can disable GC and lets
 the user manage GC collections explicitly (cooperatively?).
Wouldn't this be annoying for the user of the library? (Or maybe you only have a handful users?).
Yes, would be annoying; that's why I am asking what others would think=20 about this. (And yes, there are a handful of users.)
 if you have N cores, create N
 worker threads in the daemon and create N pipes?
That's my idea. The trouble is with the thin library layer: It cannot=20 allocate memory while another one is doing collection. This is handled=20 by D runtime for D threads but foreign threads (one that's created by=20 e.g. the C++ program) cannot be known to the D runtime by-default.
 Then set the count of
 the semaphore to N, so when the semaphore hits 0 all pipes are in use,=
 and the calling thread will wait in the API stub (wrapper function)
 until a worker thread is available?
That's an interesting idea! But it should be a little different: "I need = to collect garbage; I will wait till all others are quiet (and they=20 should know to wait until the collection is over); I collect garbage.=20 Everybody resume." Still, this would require keeping the D GC disabled=20 and my library handles collections at opportune moments that it creates.
 But maybe you want to get rid of the daemon?
I want the daemon because a full-fledged D library that loaded its own=20 libraries as neede caused issues. Perhaps they were all my mistakes but=20 now that it works this way; the daemon stays. :) Ali
May 09
parent reply Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Sunday, 9 May 2021 at 20:36:39 UTC, Ali Çehreli wrote:
 That's an interesting idea! But it should be a little 
 different: "I need to collect garbage; I will wait till all 
 others are quiet (and they should know to wait until the 
 collection is over); I collect garbage. Everybody resume." 
 Still, this would require keeping the D GC disabled and my 
 library handles collections at opportune moments that it 
 creates.
Yes, one semaphore may use an nonoptimal order when it wakes up threads... Getting this right takes some time.
May 09
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/9/21 6:58 PM, Ola Fosheim Grostad wrote:

 Yes, one semaphore may use an nonoptimal order when it wakes up threads...
 Getting this right takes some time.
ReadWriteMutex is promising: https://dlang.org/library/core/sync/rwmutex/read_write_mutex.html The writer would be the garbage collecting thread in this case. Ali
May 09
parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Monday, 10 May 2021 at 03:45:23 UTC, Ali Çehreli wrote:
 https://dlang.org/library/core/sync/rwmutex/read_write_mutex.html

 The writer would be the garbage collecting thread in this case.
The implementation seems to use a class monitor an syncronized, does that work with non D threads?
May 10
prev sibling next sibling parent reply Daniel N <no public.email> writes:
On Sunday, 9 May 2021 at 03:25:06 UTC, Ali Çehreli wrote:
 tl;dr I am scared of non-D programs calling my D library 
 functions from foreign threads. So, I am planning on asking the 
 user to trigger collection themselves by calling a collection 
 function of my library. Crazy?
Since this is a "thin" library, feels like the sane solution is to make it 100% nogc and keep the GC only in the server. But might not be possible if the design of those other libraries are all-in on GC "this library itself loads other D libraries"
May 09
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/9/21 1:09 AM, Daniel N wrote:

 On Sunday, 9 May 2021 at 03:25:06 UTC, Ali =C3=87ehreli wrote:
 tl;dr I am scared of non-D programs calling my D library functions
 from foreign threads. So, I am planning on asking the user to trigger=
 collection themselves by calling a collection function of my library.=
 Crazy?
Since this is a "thin" library, feels like the sane solution is to mak=
e
 it 100% nogc and keep the GC only in the server.
May not be possible because the library creates MmFile objects as needed = (the data length to be placed on shared memory is not known in advance). = Being a class, MmFile objects are naturally created with 'new' but=20 perhaps placement new could work. I haven't investigated that.
 But might not be possible if the design of those other libraries are
 all-in on GC
 "this library itself loads other D libraries"
I wasn't clear on that part: Now that the library is thin, loading other = D libraries has already been pushed to the backend daemon. So, my worry is based on my unconfidence in managing foreign threads=20 with thread_attachThis and thread_detachThis. Should I use a=20 thread-local Boolean to keep track? Should I call those inside 'static=20 this' and 'static ~this' blocks? What if the thread dies? I see that=20 even the name of the functions gained a "_tpl": =20 https://dlang.org/phobos/core_thread_threadbase.html#.thread_attachThis_t= pl I wonder what "_tpl" means. It mentions rt_moduleTlsCtor() there. Failing to find my way through all of that is the reason why I am hoping = that I can push a GC collection cycle to the caller. Although=20 embarrassing, it seems to be a more reliable design because the user is=20 in a better position to know they are not executing any of the D library = functions when they call the collection function. Ali
May 09
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Sunday, 9 May 2021 at 03:25:06 UTC, Ali Çehreli wrote:
 That workaround seemed to be sufficient to load the library 
 successfully. Unfortunately, that was not enough to weed out 
 all issues related to libraries because this library itself 
 loads other D libraries.
Can you statically link the library so that no other D library is needed? Might duplicate code but whatever. However, disable GC in D is difficult as soon some library function use an array for example you need the GC.
 This would allow the user start as many foreign threads as 
 possible. Right? Is this sane? Is collection the only issue 
 here? Do foreign threads still need to call threadAttachThis()? 
 What happens if they don't?

 I feel so hopeless that in the past, I even thought about and 
 experimented with banning the user from starting threads on 
 their own. Rather, they would call my library on a posix 
 compatible thread API and create their threads through me, 
 which happens to be a D thread, so no thread would be a 
 "foreign thread" and everything would work just fine. I haven't 
 deployed this crazy idea (yet).

 Ali
This is why I'm very opposed to any thread specific GC optimization. Allocated memory on GC and malloc/free must always be global and can operated on from any thread. Also, TLS must be removed from phobos/druntime so that there is basically no need for tracking the threads from a memory point of view, at least it shouldn't crash if D doesn't know about the thread unless using thread primitives. Highest priority should be removing TLS totally. Second, make sure there is no connection between memory management and threads.
May 09
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 9 May 2021 at 13:42:48 UTC, IGotD- wrote:
 Also, TLS must be removed from phobos/druntime so that there is 
 basically no need for tracking the threads from a memory point 
 of view, at least it shouldn't crash if D doesn't know about 
 the thread unless using thread primitives.
I don't think that would be enough. The garbage collector needs to know of all threads running D code, so that it can scan their stack and registers, so that it can know about objects the pointer to which exists only in that thread's stack or registers.
May 09
parent reply IGotD- <nise nise.com> writes:
On Sunday, 9 May 2021 at 13:53:30 UTC, Vladimir Panteleev wrote:
 I don't think that would be enough. The garbage collector needs 
 to know of all threads running D code, so that it can scan 
 their stack and registers, so that it can know about objects 
 the pointer to which exists only in that thread's stack or 
 registers.
That's the unfortunate case with D in that case. The otherwise good FF interoperability of D is hampered by this. Then we are back to if GC pointers should be an own type or not. Time for the D3 fork.
May 09
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 9 May 2021 at 14:10:48 UTC, IGotD- wrote:
 That's the unfortunate case with D in that case. The otherwise 
 good FF interoperability of D is hampered by this. Then we are 
 back to if GC pointers should be an own type or not. Time for 
 the D3 fork.
I wonder what you think about task-bound GC? https://forum.dlang.org/post/yqdwgbzkmutjzfdhotst forum.dlang.org
May 09
parent reply IGotD- <nise nise.com> writes:
On Sunday, 9 May 2021 at 16:08:07 UTC, Ola Fosheim Grøstad wrote:
 I wonder what you think about task-bound GC?

 https://forum.dlang.org/post/yqdwgbzkmutjzfdhotst forum.dlang.org
I see it as a special case where memory management is bounded to a certain primitive/ways of programming. Also a pool for each possible primitive will increase the meta data. Then you also mention that when the actor is destroyed no scan is needed because the pool is explicitly destroyed as well. This is basically a form of deterministic cleanup that you would expect in C++ when using unique_ptr for example. Some in that thread mention that D must head towards RC and that's what my opinion is as well. As Walter do not want to make a special fat pointer type, at least making Phobos/druntime using RC internally would be a step in the right direction.
May 09
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 9 May 2021 at 16:29:12 UTC, IGotD- wrote:
 I see it as a special case where memory management is bounded 
 to a certain primitive/ways of programming. Also a pool for 
 each possible primitive will increase the meta data.
What do you mean by meta data in this context? I think it should resolve at compile time? Yes, it does imply a programming model if you want GC, but it should allow the programmer to write his own scheduler for flexibility. You would still need RC between tasks...
 Some in that thread mention that D must head towards RC and 
 that's what my opinion is as well. As Walter do not want to 
 make a special fat pointer type, at least making 
 Phobos/druntime using RC internally would be a step in the 
 right direction.
The problem of not having a fat pointer type is when you have arrays of objects. Then you cannot have a RC counter on a negative offset, so you will be forced to embed it in all objects (whether RC-managed or not). It can be mitigated by requiring that all owning pointers to arrays must be a RC-slice and not a RC-pointer. Then you have a RC-slice of static length 1. (shrug)
May 09
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 9 May 2021 at 03:25:06 UTC, Ali Çehreli wrote:
 That workaround seemed to be sufficient to load the library 
 successfully. Unfortunately, that was not enough to weed out 
 all issues related to libraries because this library itself 
 loads other D libraries. All of this caused sporadic issues. 
 (My brain is too fried to even remember what was a cause, what 
 was a usable workaround, etc. Sometimes I wasted days chasing a 
 solution while using a test, which had nothing to do with the 
 solution. I would change the code, test, no go; repeat, no go. 
 It turns out, my test was unrelated. Argh!)
In my experience, calling D from C/C++ works fine as long as 1) the D runtime is allowed to initialize, and 2) all threads which execute D code are registered with the D runtime. If C/C++ code is allowed to hold the only reference to an object in the D GC heap, then the second rule needs to be extended to all threads which may hold a reference to said objects, but it may be practical to copy D objects at the C/D barrier either to caller-owned memory, or malloc-allocated memory that the caller can free by calling the standard C `free` function.
 So, we came up with a drastic solution: Since all this code 
 works just fine in a pure D environment, make the library as 
 thin as possible; the library starts a daemon that is written 
 in D with all the functionality. The library merely dispatches 
 the requests to that daemon.

 The library starts the daemon with pipeProcess(); pipes are 
 used for dispatching requests and shared memory is used for 
 large data. This idea "worked like a charm." Phew!

 However, dispatching of the requests to the daemon is performed 
 by a single library thread in a blocked manner: When a request 
 is written to the pipe, the response is read back (blocked) and 
 the result is returned to the user of the library function.
If the threads don't need to share state, you could just as well spawn one subprocess per thread, and let it do its own data processing. Another approach would be to listen on a UNIX socket instead of using a pipe, which allows using `accept` to open new communication channels on-demand.
 I feel so hopeless that in the past, I even thought about and 
 experimented with banning the user from starting threads on 
 their own. Rather, they would call my library on a posix 
 compatible thread API and create their threads through me, 
 which happens to be a D thread, so no thread would be a 
 "foreign thread" and everything would work just fine. I haven't 
 deployed this crazy idea (yet).
Perhaps it would be simpler to just write the library part in C / C++ / -betterC D. std.mmfile has many lines, but the work it has to do is actually quite simple. The same is true about std.socket. This will completely avoid your headache with getting the D runtime / GC to play well with the host process's threading model.
May 09
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/9/21 6:51 AM, Vladimir Panteleev wrote:

 In my experience, calling D from C/C++ works fine as long as 1) the D
 runtime is allowed to initialize, and 2) all threads which execute D
 code are registered with the D runtime.
I have the following old branch that tries to fix bugs with attaching and detaching: https://github.com/dlang/druntime/pull/1989 I also have this worry that if a foreign thread goes without the knowledge of D runtime, the program will crash because the runtime would try to stop a non-existing thread.
 If the threads don't need to share state, you could just as well spawn
 one subprocess per thread, and let it do its own data processing.
That can be achieved by some changes but the issue is not with the backend (the daemon); rather, the foreign threads that call the thin library layer. This layer does allocate (see below), which may trigger collection.
 Another approach would be to listen on a UNIX socket instead of using a
 pipe, which allows using `accept` to open new communication channels
 on-demand.
Aside: I've been under the impression that there couldn't be new pipes opened but I learned that a file descriptor can be passed to another process over unix domain sockets (only Linux is interesting to me here) and the file descriptor and its "passed copy" can be used like a pipe: https://stackoverflow.com/questions/2358684/can-i-share-a-file-descriptor-to-another-process-on-linux-or-are-they-local-to-t I haven't experimented with it yet but I think I will use it because the existing channel I wrote uses File objects of PipeProcess, which places large data on shared memory which is re-opened as more space is needed. (And thank you for your work in std.process and more! :) )
 Perhaps it would be simpler to just write the library part in C / C++ /
 -betterC D.
That's a great idea as well.
 std.mmfile has many lines, but the work it has to do is
 actually quite simple.
I know because I've already written the equivalent of mmfile myself by accident. :D When things didn't work about 2 years ago I suspected bugs in mmfile so I wrote my own wrapper. :)
 The same is true about std.socket. This will
 completely avoid your headache with getting the D runtime / GC to play
 well with the host process's threading model.
Makes sense. I've already turned into someone who uses only D's standard library or writes his own. :) The only third party dependency we have is cmake-d. Ali
May 09
next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Sunday, 9 May 2021 at 20:53:28 UTC, Ali Çehreli wrote:
 On 5/9/21 6:51 AM, Vladimir Panteleev wrote:

 In my experience, calling D from C/C++ works fine as long as
1) the D
 runtime is allowed to initialize, and 2) all threads which
execute D
 code are registered with the D runtime.
I have the following old branch that tries to fix bugs with attaching and detaching: https://github.com/dlang/druntime/pull/1989 I also have this worry that if a foreign thread goes without the knowledge of D runtime, the program will crash because the runtime would try to stop a non-existing thread.
That sounds like a bug? 🤔
May 09
prev sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 9 May 2021 at 20:53:28 UTC, Ali Çehreli wrote:
 That can be achieved by some changes but the issue is not with 
 the backend (the daemon); rather, the foreign threads that call 
 the thin library layer.

 [...]

 Aside: I've been under the impression that there couldn't be 
 new pipes opened but I learned that a file descriptor can be 
 passed to another process over unix domain sockets (only Linux 
 is interesting to me here) and the file descriptor and its 
 "passed copy" can be used like a pipe:


 https://stackoverflow.com/questions/2358684/can-i-share-a-file-descriptor-to-another-process-on-linux-or-are-they-local-to-t
Yes. But, all of this requires shared state, which requires synchronization and/or initialization. Alternatively, each thread would be entirely self-contained (and own its personal worker process), which avoids those issues entirely. (Perhaps with TLS, or better, by making the C++ caller own the state object.)
May 10