www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Interfacing with C - callbacks

reply "Ilya Zaitseff" <sark7 mail333.com> writes:
For wrappers for C libs, that use callbacks, it is very handy to pass in  
callback data reference to some object.
But D garbage collector can suddenly move objects in memory, thus  
invalidating callback data.

What is a right way to handle this?
Jul 25 2004
next sibling parent reply Ilya Minkov <minkov cs.tum.edu> writes:
Ilya Zaitseff schrieb:
 For wrappers for C libs, that use callbacks, it is very handy to pass 
 in  callback data reference to some object.
 But D garbage collector can suddenly move objects in memory, thus  
 invalidating callback data.

No, it cannot move anything. But you should make sure you have this data also pointed-to somewhere in the program so that it doesn't get collected.
 What is a right way to handle this?

I'd say the best way is to allocate this memory out of the reach of garbage colector, for example using std.c.stdlib.malloc, then it also has to be freed using std.c.stdlib.free. -eye
Jul 26 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ce2qef$quf$2 digitaldaemon.com>, Ilya Minkov says...
No, it cannot move anything.

From the D manual, Garbage Collection section: "A copying garbage collector can arbitrarily move objects around in memory". Jill
Jul 26 2004
parent reply Ilya Minkov <minkov cs.tum.edu> writes:
Arcane Jill schrieb:

 In article <ce2qef$quf$2 digitaldaemon.com>, Ilya Minkov says...
 
No, it cannot move anything.

From the D manual, Garbage Collection section: "A copying garbage collector can arbitrarily move objects around in memory".

In principle you're right and i'm aware of it, but it's not copying now, and i hardly see the possibility that such a GC could possibly take place after a while. I can't remember exactly what i was thinking of, but most likely what we would have is a further optimized non-copying GC. There are some obstacles to copying, i just don't exactly remember what. And what i also thought was that a completely strict GC is not possible, it would always have a conservative component to it. If i recall correctly the current GC can even track the references of application's memory within the memory allocated by foreign DLLs, but i think it's even worse to rely on. :) -eye
Jul 26 2004
parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <ce33lt$12bh$1 digitaldaemon.com>, Ilya Minkov says...
Arcane Jill schrieb:

 From the D manual, Garbage Collection section: "A copying garbage collector can
 arbitrarily move objects around in memory".

In principle you're right and i'm aware of it, but it's not copying now, and i hardly see the possibility that such a GC could possibly take place after a while.

Well, my life would be S-O-O-O much easier if that sentence I quoted from the D manual could be removed, and replaced with "We guarantee that at no future time will the garbage collector ever move objects around in memory". But it doesn't. It says what it says, so I have to assume that the gc will move things, even if it doesn't. The key word here is "guarantee". If you can't /guarantee/ that things will stay put, then you have no choice but to assume the possibility that they might not. Unfortunately. Arcane Jill
Jul 26 2004
prev sibling next sibling parent reply Ant <Ant_member pathlink.com> writes:
In article <opsbpwigsaaaezs2 ilya.tec.amursk.ru>, Ilya Zaitseff says...
For wrappers for C libs, that use callbacks, it is very handy to pass in  
callback data reference to some object.
But D garbage collector can suddenly move objects in memory, thus  
invalidating callback data.

What is a right way to handle this?

I don't know the right way but I use: std.gc void addRoot(void* p) Add p to list of roots. Roots are references to memory allocated by the collector that are maintained in memory outside the collector pool. The garbage collector will by default look for roots in the stacks of each thread, the registers, and the default static data segment. If roots are held elsewhere, use addRoot() or addRange() to tell the collector not to free the memory it points to. void removeRoot(void* p) Remove p from list of roots http://www.digitalmars.com/d/phobos.html#gc seems to work. Is it the right way? Ant
Jul 26 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Ant" <Ant_member pathlink.com> wrote in message
news:ce2qjv$r5g$1 digitaldaemon.com...
 In article <opsbpwigsaaaezs2 ilya.tec.amursk.ru>, Ilya Zaitseff says...
For wrappers for C libs, that use callbacks, it is very handy to pass in
callback data reference to some object.
But D garbage collector can suddenly move objects in memory, thus
invalidating callback data.

What is a right way to handle this?

I don't know the right way but I use: std.gc void addRoot(void* p) Add p to list of roots. Roots are references to memory allocated by the collector that are maintained in memory outside the collector pool. The

 collector will by default look for roots in the stacks of each thread, the
 registers, and the default static data segment. If roots are held

 addRoot() or addRange() to tell the collector not to free the memory it

 to.

 void removeRoot(void* p)
 Remove p from list of roots

 http://www.digitalmars.com/d/phobos.html#gc

 seems to work.
 Is it the right way?

Almost. The gc scans the stacks for roots. A function call's parameters are on the stack, and so are active until the C function returns. So, just passing a pointer to a C function does not in itself mean you need to notify the gc about it. The problem comes in if the pointer is pointing to a pool of data, say allocated by malloc(), and within that pool are pointers to gc'd data. The gc doesn't know about those roots, and so needs to be informed about them.
Jul 26 2004
parent Ant <Ant_member pathlink.com> writes:
In article <ce3hrt$1bk9$1 digitaldaemon.com>, Walter says...
"Ant" <Ant_member pathlink.com> wrote in message
news:ce2qjv$r5g$1 digitaldaemon.com...
 In article <opsbpwigsaaaezs2 ilya.tec.amursk.ru>, Ilya Zaitseff says...
For wrappers for C libs, that use callbacks, it is very handy to pass in
callback data reference to some object.
But D garbage collector can suddenly move objects in memory, thus
invalidating callback data.

What is a right way to handle this?

I don't know the right way but I use: std.gc void addRoot(void* p) Add p to list of roots. Roots are references to memory allocated by the collector that are maintained in memory outside the collector pool. The

 collector will by default look for roots in the stacks of each thread, the
 registers, and the default static data segment. If roots are held

 addRoot() or addRange() to tell the collector not to free the memory it

 to.

 void removeRoot(void* p)
 Remove p from list of roots

 http://www.digitalmars.com/d/phobos.html#gc

 seems to work.
 Is it the right way?

Almost. The gc scans the stacks for roots. A function call's parameters are on the stack, and so are active until the C function returns. So, just passing a pointer to a C function does not in itself mean you need to notify the gc about it. The problem comes in if the pointer is pointing to a pool of data, say allocated by malloc(), and within that pool are pointers to gc'd data. The gc doesn't know about those roots, and so needs to be informed about them.

ok, I think I fully understand the root things. well, I think because as I understand it it confirms the usage of addRoot in this case. What's going on is that my C external function has a pointer to my root that the GC doesn't know about, exactly as you say. I don't seem to have problems with objects being moved or collect (I used gc.fullCollect() extensivly when testing, maybe I should test with gc.minimize() also). the root is removed when my external C object notifies it's own removal. The method Andy told us might fail if the external application objects would be moved by it's how GC ... (?) My external application (GTK+) collects it's own objects, I have no idea of it's internals (except that I have to use the addRoot equivalent on some operations). Ant
Jul 26 2004
prev sibling parent Andy Friesen <andy ikagames.com> writes:
Ilya Zaitseff wrote:

 For wrappers for C libs, that use callbacks, it is very handy to pass 
 in  callback data reference to some object.
 But D garbage collector can suddenly move objects in memory, thus  
 invalidating callback data.
 
 What is a right way to handle this?

dfbth (yet another windowing toolkit; not important except as an example) maps HWNDs/GtkWidget*s to object instances using an associative array. This way, D object references stay on the D side and the GC doesn't need any extra information to work correctly. (copying/compacting or not) -- andy
Jul 26 2004