www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Safe method wont check dangling pointer?

reply "lzzll" <ownrepos gmail.com> writes:
Looks like dangling point is not checked even in method mark as 
safe.
Example:
---
import std.stdio;

class A {
	int value;
	void set_value(int value)  safe {
		this.value = value;
	}
}

void test_safe(A a)  safe {
	a.set_value(1);
}

int main(string[] args) {
	A a = new A();
	test_safe(a);
	test_safe(null);
	test_safe(*(&a+100));
	
	writeln("done.");
	return 0;
}
---
test_safe(null);
and
test_safe(*(&a+100));
will cause segmentation fault.

I guess reason is check dangling pointer is very inefficient.
I found another post about this
http://forum.dlang.org/thread/llezieyytpcbcaoqeajz forum.dlang.org#post-miyvktgkczatvoguawda:40forum.dlang.org
null pointer is not a safety problem, but pointer like *(&a+100) 
maybe.

Regard.
Apr 14 2014
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 19:28:06 -0400, lzzll <ownrepos gmail.com> wrote:

 Looks like dangling point is not checked even in method mark as safe.
 Example:
 ---
 import std.stdio;

 class A {
 	int value;
 	void set_value(int value)  safe {
 		this.value = value;
 	}
 }

 void test_safe(A a)  safe {
 	a.set_value(1);
 }

 int main(string[] args) {
 	A a = new A();
 	test_safe(a);
 	test_safe(null);
 	test_safe(*(&a+100));
 	
 	writeln("done.");
 	return 0;
 }
 ---
 test_safe(null);
 and
 test_safe(*(&a+100));
 will cause segmentation fault.

Safe cannot verify its inputs. main() is not marked as safe, therefore it will not help. But even so, dereferencing null is safe, since it does not corrupt memory. Your *(&a + 100) will definitely not compile if main is marked safe. -Steve
Apr 14 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/14/2014 8:14 PM, lzzll wrote:
 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error and trun it off
when
 release.
 I'll be glad if I can see it on D after some years.

Valgrind is an incredibly useful tool, but programs run terribly slowly under it.
Apr 14 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/14/14, 10:47 PM, Walter Bright wrote:
 On 4/14/2014 8:14 PM, lzzll wrote:
 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error and trun
 it off when
 release.
 I'll be glad if I can see it on D after some years.

Valgrind is an incredibly useful tool, but programs run terribly slowly under it.

ASAN = like valgrind but a crapton faster. -- Andrei
Apr 15 2014
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 2:33 AM, bearophile wrote:
 On the other hand the C/C++ world in the last years has seen
 numerous advancements that D should keep an eye on. If you look
 at the latest versions of LLVM-Clang and GCC you see various
 "sanitizers" (available as built-in tools of the compiler) that
 don't use too much memory, don't slow down your code too much,
 and catch dangling or wrong pointers, integer overflows,
 past-by-one errors, and more. One of those tools is less needed
 by D (thanks to the good management of the array bounds), but the
 others are nice.

Or you can use safe, which doesn't slow your code down at all, at least for the memory corruption problems.
Apr 15 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 4:05 AM, bearophile wrote:
 Perhaps it's a good idea to create something similar to the sanitizers (the
 memory one, etc) that work on all D compilers.

That's what safe is for.
Apr 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 10:26 AM, bearophile wrote:
 Walter Bright:

 That's what  safe is for.

I think those sanitizers (but the integer-related one) are meant to help D programmers catch bugs in system code.

I understand that. I've written my own sanitizers in the past and used them heavily. The big advantage of safe is that it offers a guarantee, sanitizers do not. Very little, however, of even a hardcore app needs to be system. What little remains is often system for performance reasons, where you'd turn off a sanitizer anyway. To sum up, a sanitizer for D offers little incremental benefit, and has a substantial implementation cost. Such cost would take away from other improvements to D that would be far more valuable.
Apr 15 2014
prev sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 15.04.2014 13:05, schrieb bearophile:
 Paulo Pinto:

 Except, as far as I am aware, they only work on GNU/Linux and Mac OS
 X, leaving out all other operating systems out there.

Is "-fsanitize=integer" not available in the Windows version of Clang? Perhaps it's a good idea to create something similar to the sanitizers (the memory one, etc) that work on all D compilers. Bye, bearophile

I had the idea that they require some kind of OS MMU API support, which neither Windows, nor commercial UNIX or embedded OS offer. Maybe I am wrong as I just saw some LLVM presentations about it and currently only code C++ occasionally so I don't follow everything in C++ world. -- Paulo
Apr 15 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 6:55 AM, "Marc Sch├╝tz" <schuetzm gmx.net>" wrote:
 I think the reason is that it returns `void*` (needs a cast and has no length)
 and points to uninitialized memory, not because you need to free it manually.

Yes.
 A memory leak is still memory safe.

True, which is why we can guarantee memory safety with a GC.
Apr 15 2014
prev sibling next sibling parent "lzzll" <ownrepos gmail.com> writes:
On Monday, 14 April 2014 at 23:44:47 UTC, Steven Schveighoffer 
wrote:
 On Mon, 14 Apr 2014 19:28:06 -0400, lzzll <ownrepos gmail.com> 
 wrote:

 Looks like dangling point is not checked even in method mark 
 as safe.
 Example:
 ---
 import std.stdio;

 class A {
 	int value;
 	void set_value(int value)  safe {
 		this.value = value;
 	}
 }

 void test_safe(A a)  safe {
 	a.set_value(1);
 }

 int main(string[] args) {
 	A a = new A();
 	test_safe(a);
 	test_safe(null);
 	test_safe(*(&a+100));
 	
 	writeln("done.");
 	return 0;
 }
 ---
 test_safe(null);
 and
 test_safe(*(&a+100));
 will cause segmentation fault.

Safe cannot verify its inputs. main() is not marked as safe, therefore it will not help. But even so, dereferencing null is safe, since it does not corrupt memory. Your *(&a + 100) will definitely not compile if main is marked safe. -Steve

That's correct. Let me ask: 1. That's mean if I write a safe library and another guy use it in the wrong way, it still not really safe, right? 2. In the real world use, if I received a segmentation fault that mean I had to get the core dump and trace where is the problem, that's all right. But if I not received anything but actually the bad memory has been write or leak, that's the security issue. 3. I hope it will be truly safe in the future, prevent the access to dangling pointer, is there any plan or idea for this?
Apr 14 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
lzzll:

 1. That's mean if I write a safe library and another guy use it 
 in the wrong way, it still not really safe, right?

Right.
 3. I hope it will be truly safe in the future, prevent the 
 access to dangling pointer, is there any plan or idea for this?

How do you suggest to design & implement it? Bye, bearophile
Apr 14 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 21:10:57 -0400, lzzll <ownrepos gmail.com> wrote:

 Let me ask:
 1. That's mean if I write a safe library and another guy use it in the  
 wrong way, it still not really safe, right?

Garbage in, garbage out. The safe function must have reasonable expectations, and it's up to you to meet them. Is it "mean"? I don't think so. I think you have to adjust what you think safe means.
 2. In the real world use, if I received a segmentation fault that mean I  
 had to get the core dump and trace where is the problem, that's all  
 right. But if I not received anything but actually the bad memory has  
 been write or leak, that's the security issue.

If it's for a null pointer, you will not have a memory corruption.
 3. I hope it will be truly safe in the future, prevent the access to  
 dangling pointer, is there any plan or idea for this?

This is actually impossible to implement. -Steve
Apr 14 2014
prev sibling next sibling parent "lzzll" <ownrepos gmail.com> writes:
Let me show some exmaple on c, and two common memory error detect 
tool.

example 1 (stack overflow):
---
int a = 100;
printf("%p\n", &a);

int *b = &a+1;
printf("%p\n", &b);

*b = 100;
---
valgrind: nothing detected
address sanitizer: ==1996== ERROR: AddressSanitizer: 
stack-buffer-overflow on address 0x7fffc976dbc4

example 2 (cross address)
---
int a = 100;
int b = 200;
printf("%p\n", &a);
printf("%p\n", &b);

int *c = &a+(&b-&a);
printf("%p\n", c);

*c = 100;
---
Of course it can't be detected.

example 3 (heap overflow)
---
int *a = (int*) malloc(sizeof(int));
printf("%p\n", a);

int *b = a + 1;
printf("%p\n", b);

*b = 100;
---
valgrind: Address 0x51f0044 is 0 bytes after a block of size 4 
alloc'd
address sanitizer: AddressSanitizer: heap-buffer-overflow on 
address 0x60040000dff4

It's possible to a certain extent.
Reference:
http://valgrind.org/docs/manual/mc-manual.html#mc-manual.vaddress
http://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm

I understand implemented this is hard and it need huge cost.
It still be useful if we only use it to detect memory error and 
trun it off when release.
I'll be glad if I can see it on D after some years.
Apr 14 2014
prev sibling next sibling parent "lzzll" <ownrepos gmail.com> writes:
On Tuesday, 15 April 2014 at 07:09:34 UTC, Andrei Alexandrescu 
wrote:
 On 4/14/14, 10:47 PM, Walter Bright wrote:
 On 4/14/2014 8:14 PM, lzzll wrote:
 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error 
 and trun
 it off when
 release.
 I'll be glad if I can see it on D after some years.

Valgrind is an incredibly useful tool, but programs run terribly slowly under it.

ASAN = like valgrind but a crapton faster. -- Andrei

I think if stuff like this implemented on D will be more faster, because D have "normal pointer" and "raw pointer", that mean we only need check them on convert. example: A a = new A(); //a is safe A *b = *a+1; //from normal pointer to raw pointer, no need check A c = *b; //from raw point to normal point, so check here But it require the code of D itself is safe, and asan is much low-level.
Apr 15 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 Valgrind is an incredibly useful tool, but programs run 
 terribly slowly under it.

On the other hand the C/C++ world in the last years has seen numerous advancements that D should keep an eye on. If you look at the latest versions of LLVM-Clang and GCC you see various "sanitizers" (available as built-in tools of the compiler) that don't use too much memory, don't slow down your code too much, and catch dangling or wrong pointers, integer overflows, past-by-one errors, and more. One of those tools is less needed by D (thanks to the good management of the array bounds), but the others are nice. Bye, bearophile
Apr 15 2014
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 15 April 2014 at 09:33:19 UTC, bearophile wrote:
 Walter Bright:

 Valgrind is an incredibly useful tool, but programs run 
 terribly slowly under it.

On the other hand the C/C++ world in the last years has seen numerous advancements that D should keep an eye on. If you look at the latest versions of LLVM-Clang and GCC you see various "sanitizers" (available as built-in tools of the compiler) that don't use too much memory, don't slow down your code too much, and catch dangling or wrong pointers, integer overflows, past-by-one errors, and more. One of those tools is less needed by D (thanks to the good management of the array bounds), but the others are nice. Bye, bearophile

Except, as far as I am aware, they only work on GNU/Linux and Mac OS X, leaving out all other operating systems out there. -- Paulo
Apr 15 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
I think  safe guarantees are much stronger if you _start_ with 
 safe main and only interact with system code via small  trusted 
gateways. If it is  system code calling  safe, low-level trickery 
it allows is just too powerful to avoid with reasonably fast 
checks.
Apr 15 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Paulo Pinto:

 Except, as far as I am aware, they only work on GNU/Linux and 
 Mac OS X, leaving out all other operating systems out there.

Is "-fsanitize=integer" not available in the Windows version of Clang? Perhaps it's a good idea to create something similar to the sanitizers (the memory one, etc) that work on all D compilers. Bye, bearophile
Apr 15 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 23:14:06 -0400, lzzll <ownrepos gmail.com> wrote:

 Let me show some exmaple on c, and two common memory error detect tool.

 example 1 (stack overflow):
 ---
 int a = 100;
 printf("%p\n", &a);

 int *b = &a+1;
 printf("%p\n", &b);

 *b = 100;
 ---

import std.stdio; void main() safe { int a = 100; writeln(&a); // line 6 int *b = &a+1; // line 8 *b = 100; } testsafe.d(6): Error: cannot take address of local a in safe function main testsafe.d(6): Error: safe function 'D main' cannot call system function 'std.stdio.writeln!(int*).writeln' (wow, really?) testsafe.d(8): Error: cannot take address of local a in safe function main testsafe.d(8): Error: pointer arithmetic not allowed in safe functions safe prevents memory errors by preventing pointer arithmetic, and taking addresses of stack locals. I was not aware that writeln is also unsafe! That must be a bug. Even writeln!int is unsafe.
 valgrind: nothing detected
 address sanitizer: ==1996== ERROR: AddressSanitizer:  
 stack-buffer-overflow on address 0x7fffc976dbc4

 example 2 (cross address)
 ---
 int a = 100;
 int b = 200;
 printf("%p\n", &a);
 printf("%p\n", &b);

 int *c = &a+(&b-&a);
 printf("%p\n", c);

 *c = 100;
 ---

illegal to use c's initializer in safe code.
 Of course it can't be detected.

It's also not a memory corruption :) optimizer will turn int *c = &a+(&b-&a) to int *c = &b;
 example 3 (heap overflow)
 ---
 int *a = (int*) malloc(sizeof(int));
 printf("%p\n", a);

 int *b = a + 1;
 printf("%p\n", b);

 *b = 100;
 ---

illegal to use b's initializer in safe code. The malloc is allowed, but only GC.malloc. C's malloc is not safe as it requires free.
 valgrind: Address 0x51f0044 is 0 bytes after a block of size 4 alloc'd
 address sanitizer: AddressSanitizer: heap-buffer-overflow on address  
 0x60040000dff4

 It's possible to a certain extent.
 Reference:
 http://valgrind.org/docs/manual/mc-manual.html#mc-manual.vaddress
 http://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm

 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error and trun it  
 off when release.
 I'll be glad if I can see it on D after some years.

It's impossible to have 100% coverage. Using sentinels and instrumenting can help find memory errors, but just not allowing memory-unsafe code is more efficient and guaranteed. Note that valgrind should be able to instrument D code as well. -Steve
Apr 15 2014
prev sibling next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 15 April 2014 at 12:12:00 UTC, Steven Schveighoffer 
wrote:
 The malloc is allowed, but only GC.malloc. C's malloc is not 
  safe as it requires free.

I think the reason is that it returns `void*` (needs a cast and has no length) and points to uninitialized memory, not because you need to free it manually. A memory leak is still memory safe.
Apr 15 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 That's what  safe is for.

I think those sanitizers (but the integer-related one) are meant to help D programmers catch bugs in system code. Bye, bearophile
Apr 15 2014
prev sibling next sibling parent "lzzll" <ownrepos gmail.com> writes:
On Tuesday, 15 April 2014 at 17:40:13 UTC, Walter Bright wrote:
 On 4/15/2014 10:26 AM, bearophile wrote:
 Walter Bright:

 That's what  safe is for.

I think those sanitizers (but the integer-related one) are meant to help D programmers catch bugs in system code.

I understand that. I've written my own sanitizers in the past and used them heavily. The big advantage of safe is that it offers a guarantee, sanitizers do not. Very little, however, of even a hardcore app needs to be system. What little remains is often system for performance reasons, where you'd turn off a sanitizer anyway. To sum up, a sanitizer for D offers little incremental benefit, and has a substantial implementation cost. Such cost would take away from other improvements to D that would be far more valuable.

I agree about implementation cost, and I didn't have enough skills to do it. I think safe just a guide or helper, it doesn't offer a guarantee, because phobos and druntime use a lot of pointer, but less test with them (or didn't release?). Also we need pointer because many library, kernel of windows/linux/bsd, communication api from outer device is written in c/c++. Free from pointer is a good dream although it is unrealistic. Thanks that valgrind can use with d, so it's the only choise now.
Apr 15 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 What little remains is often  system for performance reasons, 
 where you'd turn off a sanitizer anyway.

You are wrong. Both integer overflow sanitizers and those various modern memory/address sanitizers are not meant to be used in the final release of the code. They are meant to be used one at a time during testing or debugging (and the memory sanitizers slow down the code 2-3 times, so they can be used in many cases of debugging), and then later disabled. Bye, bearophile
Apr 15 2014
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:

 Such cost would take away from other improvements to D that
 would be far more valuable.

But I agree with this. In this moment there are more important things to do (like finishing synchronized, scope, SIMD, vector operations, operator overloading, dynamic libraries, GC, and more). Bye, bearophile
Apr 15 2014