www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Need some help with this...

reply Bane <branimir.milosavljevic gmail.com> writes:
Following code will freeze app on std.gc.fullCollect(), when sqlite3_close() in
destructor is called. If destructor is called manualy, everything goes ok.

Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2
using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are
tested so I do not suspect problem lies in them (they are compiled with dmc/gcc
using full threading support).

Is this some problem with GC or, more likely, my knowledge? I would appreciate
some clarification, this thing took me a lot of hours to track.

Thanks, 
Bane

==========================================


import std.stdio;
import std.gc;
import std.string;
import std.thread;

pragma(lib, "sqlite3.lib");
const int SQLITE_OK = 0;	// Successful result.
struct sqlite3 {}
extern(C) int sqlite3_open (char* filename, sqlite3** database);
extern(C) int sqlite3_close(sqlite3* database);

class SQLite {
  sqlite3* h;
  this(){
    assert(sqlite3_open(toStringz(":memory:"), &h) == SQLITE_OK);
  }
  ~this(){
    writefln("~this start"); // to help debug
    assert(sqlite3_close(h) == SQLITE_OK);
    writefln("~this stop"); // to help debug
  }
}

class T : Thread {
  int run(){
    SQLite s = new SQLite;
    // if next line is uncommented then app wont freeze
    // delete s;
    return 0;
  }
}

void main(){
  while(true){
    T t = new T;
    t.start;
    writefln(Thread.nthreads);
    if(Thread.nthreads > 10)
      fullCollect; // this will freeze app
  }
}
Oct 28 2009
next sibling parent Jason House <jason.james.house gmail.com> writes:
Object destructors can be tricky in a GC'd language. It looks like you're
accessing a deallocated pointer in your destructor. Order of
collection/destruction is not guaranteed.

Bane Wrote:

 Following code will freeze app on std.gc.fullCollect(), when sqlite3_close()
in destructor is called. If destructor is called manualy, everything goes ok.
 
 Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2
using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are
tested so I do not suspect problem lies in them (they are compiled with dmc/gcc
using full threading support).
 
 Is this some problem with GC or, more likely, my knowledge? I would appreciate
some clarification, this thing took me a lot of hours to track.
 
 Thanks, 
 Bane
 
 ==========================================
 
 
 import std.stdio;
 import std.gc;
 import std.string;
 import std.thread;
 
 pragma(lib, "sqlite3.lib");
 const int SQLITE_OK = 0;	// Successful result.
 struct sqlite3 {}
 extern(C) int sqlite3_open (char* filename, sqlite3** database);
 extern(C) int sqlite3_close(sqlite3* database);
 
 class SQLite {
   sqlite3* h;
   this(){
     assert(sqlite3_open(toStringz(":memory:"), &h) == SQLITE_OK);
   }
   ~this(){
     writefln("~this start"); // to help debug
     assert(sqlite3_close(h) == SQLITE_OK);
     writefln("~this stop"); // to help debug
   }
 }
 
 class T : Thread {
   int run(){
     SQLite s = new SQLite;
     // if next line is uncommented then app wont freeze
     // delete s;
     return 0;
   }
 }
 
 void main(){
   while(true){
     T t = new T;
     t.start;
     writefln(Thread.nthreads);
     if(Thread.nthreads > 10)
       fullCollect; // this will freeze app
   }
 }

Oct 28 2009
prev sibling parent reply grauzone <none example.net> writes:
Bane wrote:
 Following code will freeze app on std.gc.fullCollect(), when sqlite3_close()
in destructor is called. If destructor is called manualy, everything goes ok.
 
 Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2
using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are
tested so I do not suspect problem lies in them (they are compiled with dmc/gcc
using full threading support).

It's not your fault, it's a well known bug. The following is what happens: - in thread 1, a C function (e.g. malloc()) enters an internal lock - while thread 1 holds the lock, thread 2 triggers a D garbage collection cycle - thread 2 pauses all threads forcibly, including thread 1 - thread 2 collects some objects and calls finalizers on it - your finalizer calls a C function, which tries to enter the same lock that is held by thread 1 - but thread 1 has been paused - the GC won't resume the other threads until your function returns, and you have a deadlock As a solution, switch to D2 or Tango. These resume all suspended threads before running the finalizers.
Oct 28 2009
parent Bane <branimir.milosavljevic gmail.com> writes:
grauzone Wrote:

 Bane wrote:
 Following code will freeze app on std.gc.fullCollect(), when sqlite3_close()
in destructor is called. If destructor is called manualy, everything goes ok.
 
 Is it a bug, and if is, with what? It behaves same on winxp64 and centos5.2
using dmd 1.30 and sqlite 3.6.5 or 3.6.19 statically import lib. Libraries are
tested so I do not suspect problem lies in them (they are compiled with dmc/gcc
using full threading support).

It's not your fault, it's a well known bug. The following is what happens: - in thread 1, a C function (e.g. malloc()) enters an internal lock - while thread 1 holds the lock, thread 2 triggers a D garbage collection cycle - thread 2 pauses all threads forcibly, including thread 1 - thread 2 collects some objects and calls finalizers on it - your finalizer calls a C function, which tries to enter the same lock that is held by thread 1 - but thread 1 has been paused - the GC won't resume the other threads until your function returns, and you have a deadlock As a solution, switch to D2 or Tango. These resume all suspended threads before running the finalizers.

Thank you, grauzone. That clears it. Switching to D2 or Tango is a bit overkill as existing codebase is big and fairly tested (not including this issue :). I assume this happens pretty rarely, because I haven't noticed this bug so far. Manual delete seems to be workaround, so it is not a critical issue. Thanks again :) Love D.
Oct 28 2009