www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Foreign threads in D code.

reply Igor Shirkalin <mathsoft inbox.ru> writes:
Hello!

I have written some D code that I need to link to :C++ huge 
project. Let it be just one function that uses GC. The question 
is: if C++ code creates several threads and runs this :D function 
simultaneously, will GC work correctly?

p.s. Of course the druntime is initialized before it.

Igor Shirkalin
Jul 10
parent reply Biotronic <simen.kjaras gmail.com> writes:
On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:
 Hello!

 I have written some D code that I need to link to :C++ huge 
 project. Let it be just one function that uses GC. The question 
 is: if C++ code creates several threads and runs this :D 
 function simultaneously, will GC work correctly?

 p.s. Of course the druntime is initialized before it.

 Igor Shirkalin
If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory. Anything allocated by the GC would still be scanned. To inform DRuntime about your thread, you should call thread_attachThis: https://dlang.org/phobos/core_thread.html#.thread_attachThis As pointed out in the documentation of thread_attachThis, you might also want to call rt_moduleTlsCtor, to run thread local static constructors. Depending on your usage, this might not be necessary. -- Biotronic
Jul 10
next sibling parent reply Igor Shirkalin <mathsoft inbox.ru> writes:
On Tuesday, 11 July 2017 at 06:18:44 UTC, Biotronic wrote:
 On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:
 [...]
If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory. Anything allocated by the GC would still be scanned. To inform DRuntime about your thread, you should call thread_attachThis: https://dlang.org/phobos/core_thread.html#.thread_attachThis As pointed out in the documentation of thread_attachThis, you might also want to call rt_moduleTlsCtor, to run thread local static constructors. Depending on your usage, this might not be necessary. -- Biotronic
Thanks for very useful information!
Jul 11
parent reply Guillaume Piolat <contact spam.com> writes:
On Tuesday, 11 July 2017 at 22:59:42 UTC, Igor Shirkalin wrote:
 On Tuesday, 11 July 2017 at 06:18:44 UTC, Biotronic wrote:
 On Monday, 10 July 2017 at 20:03:32 UTC, Igor Shirkalin wrote:
 [...]
If DRuntime is not made aware of the thread's existence, the thread will not be stopped by the GC, and the GC might collect memory that the thread is referencing on the stack or in non-GC memory. Anything allocated by the GC would still be scanned. To inform DRuntime about your thread, you should call thread_attachThis: https://dlang.org/phobos/core_thread.html#.thread_attachThis As pointed out in the documentation of thread_attachThis, you might also want to call rt_moduleTlsCtor, to run thread local static constructors. Depending on your usage, this might not be necessary. -- Biotronic
Thanks for very useful information!
Just one small note. If you don't know the foreign thread lifetime, it's cleaner to detach it from the runtime upon exit. Else may fall in the following scenario. 1. you register thread A 2. thread A is destroyed later on, in the C++ code 3. another thread B come into your callback and allocate. The GC triggers and try to pause a non-existing thread A.
Jul 12
parent Igor Shirkalin <mathsoft inbox.ru> writes:
On Wednesday, 12 July 2017 at 09:49:32 UTC, Guillaume Piolat 
wrote:
 On Tuesday, 11 July 2017 at 22:59:42 UTC, Igor Shirkalin wrote:
 [...]
-- Biotronic
Thanks for very useful information!
Just one small note. If you don't know the foreign thread lifetime, it's cleaner to detach it from the runtime upon exit. Else may fall in the following scenario. 1. you register thread A 2. thread A is destroyed later on, in the C++ code 3. another thread B come into your callback and allocate. The GC triggers and try to pause a non-existing thread A.
This is important note. Yes, usually the lifetime of foreign thread is unknown. You, guys, helped me a lot.
Jul 12
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2017-07-11 08:18, Biotronic wrote:

 If DRuntime is not made aware of the thread's existence, the thread will 
 not be stopped by the GC, and the GC might collect memory that the 
 thread is referencing on the stack or in non-GC memory.
Are you sure? Wouldn't that make malloc or any other custom allocators completely useless and the D GC would completely break the C standard library because it could collect memory allocated by the C standard library? From "How Garbage Collection Works": "5. Freeing all **GC** allocated memory that has no active pointers to it and do not need destructors to run" [1]. I added the emphasize on "GC". From "Interfacing Garbage Collected Objects With Foreign Code" "If the only pointer to an object is held outside of these areas, then the collector will miss it and free the memory. To avoid this from happening, either * reallocate and copy the object using the foreign code's storage allocator or using the C runtime library's malloc/free." [2]. [1] http://dlang.org/spec/garbage.html#how_gc_works [2] http://dlang.org/spec/garbage.html#gc_foreign_obj -- /Jacob Carlborg
Jul 12
parent reply Biotronic <simen.kjaras gmail.com> writes:
On Wednesday, 12 July 2017 at 09:10:07 UTC, Jacob Carlborg wrote:
 On 2017-07-11 08:18, Biotronic wrote:

 If DRuntime is not made aware of the thread's existence, the 
 thread will not be stopped by the GC, and the GC might collect 
 memory that the thread is referencing on the stack or in 
 non-GC memory.
Are you sure? Wouldn't that make malloc or any other custom allocators completely useless and the D GC would completely break the C standard library because it could collect memory allocated by the C standard library? From "How Garbage Collection Works": "5. Freeing all **GC** allocated memory that has no active pointers to it and do not need destructors to run" [1]. I added the emphasize on "GC". From "Interfacing Garbage Collected Objects With Foreign Code" "If the only pointer to an object is held outside of these areas, then the collector will miss it and free the memory. To avoid this from happening, either * reallocate and copy the object using the foreign code's storage allocator or using the C runtime library's malloc/free." [2]. [1] http://dlang.org/spec/garbage.html#how_gc_works [2] http://dlang.org/spec/garbage.html#gc_foreign_obj
That's basically what I tried to say - the GC may collect memory *it has allocated* if the only pointers to it are in memory the GC doesn't scan (i.e. on the stack of an unregistered thread or in memory not allocated via the GC). It will not collect memory allocated by other means, but that Foo* you got from D and are using in C++ might point to a Bar soon after the GC runs. -- Biotronic
Jul 12
parent reply Jacob Carlborg <doob me.com> writes:
On 2017-07-12 11:28, Biotronic wrote:

 That's basically what I tried to say 
It wasn't very clear to me at least.
 - the GC may collect memory *it has 
 allocated* if the only pointers to it are in memory the GC doesn't scan 
 (i.e. on the stack of an unregistered thread or in memory not allocated 
 via the GC).
 
 It will not collect memory allocated by other means, but that Foo* you 
 got from D and are using in C++ might point to a Bar soon after the GC 
 runs.
Yes, that can happen. -- /Jacob Carlborg
Jul 12
parent Biotronic <simen.kjaras gmail.com> writes:
On Wednesday, 12 July 2017 at 12:08:35 UTC, Jacob Carlborg wrote:
 On 2017-07-12 11:28, Biotronic wrote:

 That's basically what I tried to say
It wasn't very clear to me at least.
Yeah, I see it in retrospect. "might collect memory that the thread is referencing on the stack or in non-GC memory" doesn't convey that the collected memory is only that which is allocated by the GC. -- Biotronic
Jul 12