www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to be more careful about null pointers?

reply cy <dlang verge.info.tm> writes:
I finally found the null pointer. It took a week. I was assigning 
"db = db" when I should have been assigning "this.db = db". 
Terrible, I know. But...

I invoked db.find_chapter.bindAll(8,4), when db was a null 
pointer. There was no null pointer error. No exception raised for 
dereferencing a null. I'm not in release mode. Assertions are 
enabled. Shouldn't that have raised a null pointer exception?

Instead, it accesses db as if db were not null, producing a 
garbage structure in find_chapter, which bindAll chokes on, then 
causes the whole program to segfault.

I realize enforce(db).find_chapter would work, but... I thought D 
was more careful about null pointers? At least enough to die on 
dereferencing them?
Mar 28 2016
next sibling parent cy <dlang verge.info.tm> writes:
On Monday, 28 March 2016 at 21:01:19 UTC, cy wrote:

 I invoked db.find_chapter.bindAll(8,4), when db was a null 
 pointer.


No, no, no it's worse than that. What I did was (db.)find_chapter 
= (db.)backend.prepare("...") when backend was null, and got no 
error. find_chapter was garbage of course, but there was no 
error. And only much later, when I called 
db.find_chapter.bindAll(...) did it react to the garbage data, by 
messily segfaulting. db was a good pointer, but db.backend was 
bad, and db.backend.prepare() didn't even flinch getting passed a 
null this, and dereferencing that null internally to construct 
Statement(Database,string).
Mar 28 2016
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 28 March 2016 at 21:01:19 UTC, cy wrote:

 No exception raised for dereferencing a null.


If it didn't give the error, either you swallowed it or you 
didn't actually dereference null.

The latter is a kinda strange happenstance, but if you are 
calling a static or final method on an object and it doesn't 
actually use any member variables, it is possible for the call to 
succeed even if null.

void doSomething(void* this) {
   // if you never actually use this....
}

// then this is no error:

doSomething(null);


What is the db library you are using? Did you compile it along 
with your program or use a .lib with it?
Mar 28 2016
parent reply cy <dlang verge.info.tm> writes:
On Monday, 28 March 2016 at 21:24:48 UTC, Adam D. Ruppe wrote:

 If it didn't give the error, either you swallowed it or you 
 didn't actually dereference null.


Okay, so it's not actually supposed to happen. Hopefully it's 
something I did wrong...


 What is the db library you are using? Did you compile it along 
 with your program or use a .lib with it?


d2sqlite3: https://github.com/biozic/d2sqlite3

Compiled it along with my program. And "Database.prepare" is 
neither static, nor final. I keep describing it wrong though, 
what I ended up doing. It's easier to just write a program to 
describe it. What I ended up doing was like this:

struct Database {
   string derp;
   Statement prepare(string s) {
	return Statement(1234);
   }
}

struct Statement {
   int member;
   void bind(int column, int value) {
	import std.stdio;
	writeln("derp",member);
   }

}


class Wrapper {
   Database something;
   Statement prep;
   this() {
     something = Database("...");
     prep = something.prepare("...");
   }
}

Wrapper oops;
void initialize() {
   oops = new Wrapper();
}

class Entry {
   Wrapper parent;
   this(Wrapper parent) {
     //this.parent = parent;
     //oops
     parent = parent;
   }
   void usefulmethod() {
     parent.prep.bind(1,42);
     //parent.prep.execute();
     //parent.prep.reset();
   }
}

void main() {
   initialize();
   auto entry = new Entry(oops);
   entry.usefulmethod();
}

That program causes a segmentation fault on my machine. Somehow 
despite never initializing Entry.parent, a class object (whose 
default init is a null pointer), I can still call methods on it, 
access members on it, and call methods on those members. No 
warnings or errors. The segfault doesn't happen until the bind() 
method.
Mar 28 2016
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 03/28/2016 11:00 PM, cy wrote:


 struct Database {
    string derp;
    Statement prepare(string s) {
      return Statement(1234);
    }
 }

 struct Statement {
    int member;
    void bind(int column, int value) {
      import std.stdio;
      writeln("derp",member);


Change that to &member and it will not segfault. It will print a value 
as low as 20. Your program is thinking that the Statement object is at 
address 20. :(


    }

 }


 class Wrapper {
    Database something;
    Statement prep;
    this() {
      something = Database("...");
      prep = something.prepare("...");
    }
 }

 Wrapper oops;
 void initialize() {
    oops = new Wrapper();
 }

 class Entry {
    Wrapper parent;
    this(Wrapper parent) {
      //this.parent = parent;
      //oops
      parent = parent;
    }
    void usefulmethod() {
      parent.prep.bind(1,42);


parent.prep.bind is translated to the following by the compiler:

"Call bind() for the object at address... let's calculate... Wherever 
parent is, we should add the offset of prep inside that object."

Since 'parent' is a class reference, implemented in terms of a CPU 
pointer, which happens to be uninitialized, meaning that its value is 0, 
the overall address is the following on my machine:

   0 + 20

As you've observed, the program never accesses location 0 for 'parent', 
rather, calculates the member starting with 'parent' and tries to access 
a location inside the first page, which is reserved by the operating 
system to catch exactly situations like this.

Ali
Mar 28 2016
parent reply cy <dlang verge.info.tm> writes:
On Tuesday, 29 March 2016 at 06:21:49 UTC, Ali Çehreli wrote:

 parent.prep.bind is translated to the following by the compiler:

 "Call bind() for the object at address... let's calculate... 
 Wherever parent is, we should add the offset of prep inside 
 that object."


Okay, that's helpful actually. So, when accessing members of an 
object, it won't trigger a NPE, because the object is not 
dereferenced immediately, but the expected offset to the member 
is dereferenced. D puts off resolving pointer addresses as much 
as possible, so instead of bind((*parent).(*prep)) it's more like 
bind(parentlocation + prepoffset) and only (this.)member resolves 
to (*(parentlocation + prepoffset).member).

I thought it had an extra layer of pointer indirection, like if 
class object A has address 1234, and member A.b has offset 0x20, 
then you first dereference address 1234 (to get 4321 say) and 
then add 0x20 to that. But actually a class object is... already 
dereferenced, and is an integer representing the address to the 
object, not a pointer to that address.

Variables representing fixed, definite memory locations really 
throws me sometimes. You can't assign to a const, because that 
assigns to const marked memory, and doesn't just move the 
variable somewhere else. And NPE doesn't occur with a class, 
because setting it to NULL, and accessing a member is more like 
this in C:

intptr_t a = 0;
(struct B*)(a+0x20).member

and less like this in C:

intptr_t* a = 0;
(struct B*)(*a+0x20).member
Mar 29 2016
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 03/29/2016 11:57 AM, cy wrote:

 On Tuesday, 29 March 2016 at 06:21:49 UTC, Ali Çehreli wrote:

 parent.prep.bind is translated to the following by the compiler:

 "Call bind() for the object at address... let's calculate... Wherever
 parent is, we should add the offset of prep inside that object."


 Okay, that's helpful actually. So, when accessing members of an object,
 it won't trigger a NPE, because the object is not dereferenced
 immediately, but the expected offset to the member is dereferenced. D
 puts off resolving pointer addresses as much as possible, so instead of
 bind((*parent).(*prep)) it's more like bind(parentlocation + prepoffset)
 and only (this.)member resolves to (*(parentlocation + 


prepoffset).member).

 I thought it had an extra layer of pointer indirection, like if class
 object A has address 1234,


The class object would be sitting at that address. Our access to it is 
through a class variable:

     Foo var = new Foo();

Although you don't see a pointer in sight, behind scenes 'var' is 
nothing but a pointer.


 and member A.b has offset 0x20, then you
 first dereference address 1234 (to get 4321 say) and then add 0x20 to
 that.


No, there is no dereferencing of 'var'. However, you could have 'Foo*', 
which would be the equivalent of a "pointer to  pointer" in C.


 But actually a class object is... already dereferenced, and is an
 integer representing the address to the object, not a pointer to that
 address.


Correct.


 Variables representing fixed, definite memory locations really throws me
 sometimes. You can't assign to a const, because that assigns to const
 marked memory


The object *may* really be on const-marked memory but not necessarily. 
You can have const variables anywhere; the const access is checked at 
compile time.


 and doesn't just move the variable somewhere else.


That does not happen for various reasons. Firstly, if we could mutate, 
what would 'const' mean? Then, the compiler would have to have a 
complete view of the program to update all other references, which would 
be slow and the semantics would be difficult.


 And
 NPE doesn't occur with a class, because setting it to NULL, and
 accessing a member is more like this in C:

 intptr_t a = 0;
 (struct B*)(a+0x20).member

 and less like this in C:

 intptr_t* a = 0;
 (struct B*)(*a+0x20).member


Yes.

Ali
Mar 29 2016
prev sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 29 Mar 2016 06:00:32 +0000
schrieb cy <dlang verge.info.tm>:


 struct Database {
    string derp;
    Statement prepare(string s) {
 	return Statement(1234);
    }
 }
 
 struct Statement {
    int member;
    void bind(int column, int value) {
 	import std.stdio;
 	writeln("derp",member);
    }
 
 }
 
 
 class Wrapper {
    Database something;
    Statement prep;
    this() {
      something = Database("...");
      prep = something.prepare("...");
    }
 }
 
 Wrapper oops;
 void initialize() {
    oops = new Wrapper();
 }
 
 class Entry {
    Wrapper parent;
    this(Wrapper parent) {
      //this.parent = parent;
      //oops
      parent = parent;
    }
    void usefulmethod() {
      parent.prep.bind(1,42);
      //parent.prep.execute();
      //parent.prep.reset();
    }
 }
 
 void main() {
    initialize();
    auto entry = new Entry(oops);
    entry.usefulmethod();
 }
 
 That program causes a segmentation fault on my machine. Somehow 
 despite never initializing Entry.parent, a class object (whose 
 default init is a null pointer), I can still call methods on it, 
 access members on it, and call methods on those members. No 
 warnings or errors. The segfault doesn't happen until the bind() 
 method.


Dlang aborts programs that run into invalid states. Common
invalid states are failing contract assertions, out of memory
conditions or in this case a null-pointer dereference.

When designing Dlang, Walter decided that null pointers don't
require checks as the operating system will eventually abort
the program and debuggers are designed to assist you in this
situation.

Some people with a different programming background feel that
a modern language should wrap every pointer access in a check
and possibly throw a recoverable Exception here or make
null-pointer opt-in to begin with. The topic is still open for
discussion:
http://wiki.dlang.org/Language_issues#Null_References

-- 
Marco
Mar 29 2016
prev sibling parent QAston <qaston gmail.com> writes:
On Monday, 28 March 2016 at 21:01:19 UTC, cy wrote:

 I finally found the null pointer. It took a week. I was 
 assigning "db = db" when I should have been assigning "this.db 
 = db". Terrible, I know. But...

 I invoked db.find_chapter.bindAll(8,4), when db was a null 
 pointer. There was no null pointer error. No exception raised 
 for dereferencing a null. I'm not in release mode. Assertions 
 are enabled. Shouldn't that have raised a null pointer 
 exception?

 Instead, it accesses db as if db were not null, producing a 
 garbage structure in find_chapter, which bindAll chokes on, 
 then causes the whole program to segfault.

 I realize enforce(db).find_chapter would work, but... I thought 
 D was more careful about null pointers? At least enough to die 
 on dereferencing them?


You can use a lint to warn you about silly mistakes, check out 
dscanner and possibly others.
Mar 29 2016