www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Questions about destructors, threads, SDL, OpenGL and GC

reply Peter Mackay <a_pointy_stick.NoJunkMail yahoo.co.uk> writes:
Hello,

Sorry for the long subject line. Hopefully it will help people searching 
for these topics later on.

This is my first post after a month or so of lurking. I'm trying out D 
after years of using C++, C and Java for home projects and in work (I'm 
an ex full-time game developer).

I'm experimenting with creating a thin wrapper around SDL and OpenGL 
using DerelictSDL, and DerelictGL, and my own Surface, Texture, etc objects.

I'd like not to have to manually call SDL_FreeSurface, SDL_CloseFont, 
etc before my Surface and Font objects get GC'd, so I put my cleanup 
calls in my destructors.

 From what I've read here (I searched for 'thread', 'destructor' and 
'~this' using a subject search (the only useful kind of search 
available) in my news client), the GC is non-deterministic. Walter said 
at some point that destructors may not even be called for non auto 
objects (which makes things hard).

I have a few questions to help me handle this. I'd be really grateful if 
anyone has some insights they could offer me.

(a) What thread will destructors be called from? Java calls finalizers 
from a separate thread. Is this behaviour specified in D? Wrapping 
OpenGL will be awkward if I call glDeleteTextures in a different thread 
to my GL context.

(b) Can I rely on destructors being called at all? If not, what is their 
purpose for non auto objects? I can't use auto classes because then I 
can't embed references inside other classes.

(c) Just calling SDL_FreeSurface in my destructor causes a crash. I'm 
guessing that the memory I'm referencing is garbage since SDL has been 
shut down elsewhere. I currently maintain a static associative array per 
class to track them, in the hope that they will then get collected in 
the module cleanup. Is this wise? It currently seems not to crash, but 
it could be leaking memory and I just can't tell.

(d) Would I be better off just wrapping everything in try...finally 
blocks and manually calling my own finalize() and close() methods? Do 
people have to do this for files anyway?

Thanks in advance for any hints anyone may be able to give me.

Peter
May 08 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Peter Mackay" <a_pointy_stick.NoJunkMail yahoo.co.uk> wrote in message 
news:d5ksit$15bc$1 digitaldaemon.com...
 Hello,

 Sorry for the long subject line. Hopefully it will help people searching 
 for these topics later on.

 This is my first post after a month or so of lurking. I'm trying out D 
 after years of using C++, C and Java for home projects and in work (I'm an 
 ex full-time game developer).

 I'm experimenting with creating a thin wrapper around SDL and OpenGL using 
 DerelictSDL, and DerelictGL, and my own Surface, Texture, etc objects.

 I'd like not to have to manually call SDL_FreeSurface, SDL_CloseFont, etc 
 before my Surface and Font objects get GC'd, so I put my cleanup calls in 
 my destructors.

 From what I've read here (I searched for 'thread', 'destructor' and 
 '~this' using a subject search (the only useful kind of search available) 
 in my news client), the GC is non-deterministic. Walter said at some point 
 that destructors may not even be called for non auto objects (which makes 
 things hard).

correct. In fact a dtor might not even get called today if the GC finds an "ambiguous pointer" to the object. An ambiguous pointer is a number scanned by the GC that looks like a pointer (ie points to something legit) but without the type information the GC can't actually tell if it is a pointer or not. In those cases the GC must assume it is a pointer. It is also unclear if the GC is run at program exit or not.
 I have a few questions to help me handle this. I'd be really grateful if 
 anyone has some insights they could offer me.

 (a) What thread will destructors be called from? Java calls finalizers 
 from a separate thread. Is this behaviour specified in D? Wrapping OpenGL 
 will be awkward if I call glDeleteTextures in a different thread to my GL 
 context.

The thread is unspecified. Right now it is on the same thread as the allocation request that triggered the garbage collect. Different collectors might run in another thread, though.
 (b) Can I rely on destructors being called at all? If not, what is their 
 purpose for non auto objects? I can't use auto classes because then I 
 can't embed references inside other classes.

You can't rely on dtors ever being run. They are run with high probability, though. Are your resources cleaned up automatically by the OS on program exit? If so you might not need to worry about those rare instances. One can also have auto instances without declaring the entire class as auto. The downside of that, though, is auto variables can't be passed to functions or used outside of the current function. Here's a simple guard that will always free an object at scope exit: auto class Guard(T) { T obj; // obj will never gets GC'ed while we exist this(T obj) { this.obj = obj; } ~this() { delete obj; } } ... foo() { Surface s = ...; auto Guard!(Surface) sg = new Guard!(Surface)(s); // ensure s gets cleaned up. ... } Note you might want to use a "finalize" or "close" method instead of a dtor since you can control which threads call finalize but you can't control which threads call the dtor.
 (c) Just calling SDL_FreeSurface in my destructor causes a crash. I'm 
 guessing that the memory I'm referencing is garbage since SDL has been 
 shut down elsewhere. I currently maintain a static associative array per 
 class to track them, in the hope that they will then get collected in the 
 module cleanup. Is this wise? It currently seems not to crash, but it 
 could be leaking memory and I just can't tell.

Are you shutting down SDL right before returning from main()? If so then probably the GC run that happens after main() is done is the culprit. Be aware the static assoc array won't get collected. Also note that the references from the static array will prevent the surfaces from every getting GC'ed. If there is only one thread that can release the resources you might want to use a global free list (malloc'ed so that it doesn't get scanned by the GC) that objects append themselves to in their dtor and then the thread that can free resources can free the list when it can. The tricky part is that a dtor can't reference any other GC-managed resource since order in which dtors are run is not deterministic.
 (d) Would I be better off just wrapping everything in try...finally blocks 
 and manually calling my own finalize() and close() methods? Do people have 
 to do this for files anyway?

Files get closed automatically at program exit so the rare ambiguous pointer doesn't hurt in practice. Also files can be closed from any thread so it is much easier.
 Thanks in advance for any hints anyone may be able to give me.

 Peter 

May 08 2005
parent Peter Mackay <a_pointy_stick.NoJunkMail yahoo.co.uk> writes:
Hello,

Thank you very much for your reply Ben, it helped me out a lot.

the GC is non-deterministic. Walter said at some point 
that destructors may not even be called for non auto objects (which makes 
things hard).

correct. In fact a dtor might not even get called today if the GC finds an "ambiguous pointer" to the object. An ambiguous pointer is a number scanned by the GC that looks like a pointer (ie points to something legit) but without the type information the GC can't actually tell if it is a pointer or not. In those cases the GC must assume it is a pointer. It is also unclear if the GC is run at program exit or not.

Once when trying to call SDL_FreeSurface in my destructor, my app crashed in ~this which was called from inside a compiler-generated function called "fullcollect", which was called from mainCRTStartup, or similar. So, the GC is called, but whether it's in the spec and could be relied upon, I don't know. I'm not going to bank on it.
(a) What thread will destructors be called from?

The thread is unspecified. Right now it is on the same thread as the allocation request that triggered the garbage collect. Different collectors might run in another thread, though.

That makes sense, thanks. :-)
 You can't rely on dtors ever being run. They are run with high probability, 
 though. Are your resources cleaned up automatically by the OS on program 
 exit? If so you might not need to worry about those rare instances. One can 
 also have auto instances without declaring the entire class as auto. The 
 downside of that, though, is auto variables can't be passed to functions or 
 used outside of the current function. Here's a simple guard that will always 
 free an object at scope exit:
   auto class Guard(T) {
     T obj; // obj will never gets GC'ed while we exist
     this(T obj) { this.obj = obj; }
     ~this() { delete obj; }
   }
   ...
   foo() {
     Surface s = ...;
     auto Guard!(Surface) sg = new Guard!(Surface)(s); // ensure s gets 
 cleaned up.
     ...
   }
 Note you might want to use a "finalize" or "close" method instead of a dtor 
 since you can control which threads call finalize but you can't control 
 which threads call the dtor.

I think this is what I'm going to do.
(c) Just calling SDL_FreeSurface in my destructor causes a crash. I'm 
guessing that the memory I'm referencing is garbage since SDL has been 
shut down elsewhere. I currently maintain a static associative array per 
class to track them, in the hope that they will then get collected in the 
module cleanup. Is this wise? It currently seems not to crash, but it 
could be leaking memory and I just can't tell.

Are you shutting down SDL right before returning from main()?

I'm shutting it down in my sdl wrapper module static destructor. Here's the solution I came up with, hopefully I'm doing things the 'right way'. Sorry about the lack of comments. I don't yet know the proper way to add document comments in D. The code should be easily readable though. Here's the rundown, for people who would prefer not to read the code: 1. Every SDL object inherits from SDLObject, which has an abstract finalize method. 2. SDLObject has a static AA of refs to it's instances. 3. In my module dtor, I ask SDLObject to go through all the instances and finalize them, if they weren't finalized elsewhere. Problems: 1. If the app creates lots of SDL objects, the AA will keep growing and growing... 2. I'm still not sure what will get GC'd and what won't. Maybe that's normal. 3. Threading is completely ignored. I usually don't use extra threads for external API calls, only for loading files into buffers, streaming sounds, etc. --- code follows --- module saurus.sdl; import derelict.sdl.sdl; private import derelict.sdl.image; private import derelict.sdl.ttf; private import saurus.drawing; private import std.stdio; private abstract class SDLObject { private static SDLObject[SDLObject] objects; protected bool finalized; this() { objects[this] = this; finalized = false; } static void finalizeAll() { foreach (SDLObject object; objects) { if (!object.finalized) { object.finalize(); assert(object.finalized == true); } delete objects[object]; } } abstract void finalize(); } static this() { DerelictSDL_Load(); if (SDL_Init(SDL_INIT_EVERYTHING) < 0) { throw new Error("SDL_Init failed."); } DerelictSDLImage_Load(); DerelictSDLttf_Load(); if (TTF_Init() < 0) { throw new Error("TTF_Init failed."); } } static ~this() { SDLObject.finalizeAll(); TTF_Quit(); SDL_Quit(); } class Surface : SDLObject { private SDL_Surface* surface; this(SDL_Surface* surface) { this.surface = surface; } this(char[] name) { this(IMG_Load(name ~ "\0")); } ... protected void finalize() { if (!finalized) { SDL_FreeSurface(surface); surface = null; finalized = true; } } invariant { if (!finalized) { assert(surface != null); assert(surface.refcount == 1); } } } --- end of code --- I hope this thread helps anyone who is trying to solve a similar problem. Peter
May 08 2005