www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Looks like dereferencing a null pointer, but is ok???

reply Georg Wrede <georg.wrede nospam.org> writes:
In an old message,


 I don't think that any extra overhead is involved in slicing. 
 D doesn't even check if a pointer is valid.
 (That might be a gotcha come to think of it)

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


 
 Great example! I am now convinced there is nothing

 to worry about. Thanks everyone for the help! 




I'm baffled.

I would have expected this to cause an error!

And if really not, what in the world is

    (cast(char*)null)[0..10]

supposed to mean, or return???
Sep 11 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Georg Wrede wrote:

 In an old message,
 

 I don't think that any extra overhead is involved in slicing. D 
 doesn't even check if a pointer is valid.
 (That might be a gotcha come to think of it)

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


 Great example! I am now convinced there is nothing

 to worry about. Thanks everyone for the help! 




 
 I'm baffled.
 
 I would have expected this to cause an error!
 
 And if really not, what in the world is
 
    (cast(char*)null)[0..10]
 
 supposed to mean, or return???


Who knows what it means--it should cause an access violation at run time.


Sean
Sep 11 2006
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 11 Sep 2006 14:12:20 -0700, Sean Kelly <sean f4.ca> wrote:

 Georg Wrede wrote:

 In an old message,


 I don't think that any extra overhead is involved in slicing. D  
 doesn't even check if a pointer is valid.
 (That might be a gotcha come to think of it)

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


 Great example! I am now convinced there is nothing

 to worry about. Thanks everyone for the help!




  I'm baffled.
  I would have expected this to cause an error!
  And if really not, what in the world is
     (cast(char*)null)[0..10]
  supposed to mean, or return???


 Who knows what it means--it should cause an access violation at run time.


Why? I mean null is the memory address 0x0 and slicing simply assigns a  
string to the values given, eg.

//assuming
struct string {
   void* data;
   int length;
}

//we get
string a;
a.data = null;
a.length = 10;

which, while the result is a totally invalid string it doesn't actually  
read/write anything to the memory address 0x0 (null), so, no access  
violation or similar.

If anything it's an array bounds error.. but array bounds checking doesn't  
occur when creating slices from pointers (for obvious reasons) so.. unless  
a special case check was added for null, I can't see what D/DMD could do  
about it.

Regan
Sep 11 2006
parent reply Sean Kelly <sean f4.ca> writes:
Regan Heath wrote:

 On Mon, 11 Sep 2006 14:12:20 -0700, Sean Kelly <sean f4.ca> wrote:

 Georg Wrede wrote:

 In an old message,


 I don't think that any extra overhead is involved in slicing. D 
 doesn't even check if a pointer is valid.
 (That might be a gotcha come to think of it)

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


 Great example! I am now convinced there is nothing

 to worry about. Thanks everyone for the help!




  I'm baffled.
  I would have expected this to cause an error!
  And if really not, what in the world is
     (cast(char*)null)[0..10]
  supposed to mean, or return???


 Who knows what it means--it should cause an access violation at run time.


 
 Why? I mean null is the memory address 0x0 and slicing simply assigns a 
 string to the values given, eg.
 
 //assuming
 struct string {
   void* data;
   int length;
 }
 
 //we get
 string a;
 a.data = null;
 a.length = 10;


Oh true.  I forgot that the slice syntax doesn't actually dereference 
the memory if the slice occurs on an rvalue.  So I suppose it would be 
okay.  But you obviously couldn't do anything with the slice.


Sean
Sep 11 2006
parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 11 Sep 2006 16:41:52 -0700, Sean Kelly <sean f4.ca> wrote:

 Regan Heath wrote:

 On Mon, 11 Sep 2006 14:12:20 -0700, Sean Kelly <sean f4.ca> wrote:

 Georg Wrede wrote:

 In an old message,


 I don't think that any extra overhead is involved in slicing. D  
 doesn't even check if a pointer is valid.
 (That might be a gotcha come to think of it)

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


 Great example! I am now convinced there is nothing

 to worry about. Thanks everyone for the help!




  I'm baffled.
  I would have expected this to cause an error!
  And if really not, what in the world is
     (cast(char*)null)[0..10]
  supposed to mean, or return???


 Who knows what it means--it should cause an access violation at run  
 time.


  Why? I mean null is the memory address 0x0 and slicing simply assigns  
 a string to the values given, eg.
  //assuming
 struct string {
   void* data;
   int length;
 }
  //we get
 string a;
 a.data = null;
 a.length = 10;


 Oh true.  I forgot that the slice syntax doesn't actually dereference  
 the memory if the slice occurs on an rvalue.  So I suppose it would be  
 okay.  But you obviously couldn't do anything with the slice.


True. In fact the slice is toxic, doing anything with it is likely to  
cause a crash. It's possible that creating an array by slicing a null  
pointer will occur often enough that adding a special check for that case  
will prevent a number of bugs.

Regan
Sep 11 2006
prev sibling next sibling parent reply nobody <nobody mailinator.com> writes:
Georg Wrede wrote:

 In an old message,
 

 I don't think that any extra overhead is involved in slicing. D 
 doesn't even check if a pointer is valid.
 (That might be a gotcha come to think of it)

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }


 Great example! I am now convinced there is nothing

 to worry about. Thanks everyone for the help! 




 
 I'm baffled.
 
 I would have expected this to cause an error!
 
 And if really not, what in the world is
 
    (cast(char*)null)[0..10]
 
 supposed to mean, or return???


It is an array of the first 10 char elements starting from memory address 0.
Sep 11 2006
parent reply Georg Wrede <georg.wrede nospam.org> writes:
nobody wrote:

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }




 I would have expected this to cause an error!

 And if really not, what in the world is

    (cast(char*)null)[0..10]

 supposed to mean, or return???


 
 It is an array of the first 10 char elements starting from memory 
 address 0.


In the ways not otherwise specified, D is alike C and C++, at least in 
spirit.

Bjarne says (C++, 3rd ed. p 5.1.1) "No object is allocated with the 
address 0. Consequently, 0 acts as a pointer literal, indicating that a 
pointer doesn't refer to an object." ((I take it this means not only 
class object instances but anything pointable at all.))

For some reason I got no compile time nor runtime error from this.

We're using the slice as an rvalue, so this constitutes a dereference of 
a null pointer. And therefore an error.

(I seem to remember having read somewhere that modern systems don't give 
user programs address spaces containing the zero address (even 
virtually), precisely to avoid problems and pitfalls with null. Can 
anybody confirm this?)

Further, since NULL is explicitly defined to not point to anything 
(therefore not even the address zero), at best this code attempts to 
return a slice out of thin air. In other words, garbage.

And specifically, even if there /was/ a zero address to point to, it 
would still be illegal to return the slice.

---

The following program does die (in windows) with an Access Violation:

import std.stdio;

void main()
{
         char[] foo = (cast(char*)null)[0..10];

	char c = foo[1];
}

And we weren't even accessing foo[0]. :-)

---

I consider it a bug when the compiler creates such an unusable array.
I consider it an bug to not error on (cast(char*)null)[0..10].
Sep 12 2006
parent reply Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
Georg Wrede wrote:

 nobody wrote:

 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }




 I would have expected this to cause an error!

 And if really not, what in the world is

    (cast(char*)null)[0..10]

 supposed to mean, or return???


 It is an array of the first 10 char elements starting from memory 
 address 0.


 
 In the ways not otherwise specified, D is alike C and C++, at least in 
 spirit.
 
 Bjarne says (C++, 3rd ed. p 5.1.1) "No object is allocated with the 
 address 0. Consequently, 0 acts as a pointer literal, indicating that a 
 pointer doesn't refer to an object." ((I take it this means not only 
 class object instances but anything pointable at all.))
 
 For some reason I got no compile time nor runtime error from this.


I don't see any reason you should. Merely handling null pointers is not 
an error.


 We're using the slice as an rvalue, so this constitutes a dereference of 
 a null pointer. And therefore an error.


It doesn't constitute a dereferencing as far as I can tell. You merely 
define a slice starting at the address null, being 10 chars long.

Should:

char *start = null;
int len = 10;

be illegal?


 (I seem to remember having read somewhere that modern systems don't give 
 user programs address spaces containing the zero address (even 
 virtually), precisely to avoid problems and pitfalls with null. Can 
 anybody confirm this?)
 
 Further, since NULL is explicitly defined to not point to anything 
 (therefore not even the address zero), at best this code attempts to 
 return a slice out of thin air. In other words, garbage.


Yes. But it never touches the garbage.


 And specifically, even if there /was/ a zero address to point to, it 
 would still be illegal to return the slice.


Why? Should it be illegal to return an invalid pointer too?


 The following program does die (in windows) with an Access Violation:
 
 import std.stdio;
 
 void main()
 {
         char[] foo = (cast(char*)null)[0..10];
 
     char c = foo[1];
 }
 
 And we weren't even accessing foo[0]. :-)


AFAIK, most OS never map the virtual memory page 0 to anything, making 
accesses to this page illegal. With a page size of 4k, accessing 
anything within [0..4096] will give a guaranteed page fault.


 I consider it a bug when the compiler creates such an unusable array.
 I consider it an bug to not error on (cast(char*)null)[0..10].


I humbly disagree on both points.

/Oskar
Sep 12 2006
parent reply Georg Wrede <georg.wrede nospam.org> writes:
Oskar Linde wrote:

 Georg Wrede wrote:
 

 nobody wrote:


 void main()
 {
     char[] foo = (cast(char*)null)[0..10];
 }





 I would have expected this to cause an error!

 And if really not, what in the world is

    (cast(char*)null)[0..10]

 supposed to mean, or return???



 It is an array of the first 10 char elements starting from memory 
 address 0.



 In the ways not otherwise specified, D is alike C and C++, at least in 
 spirit.

 Bjarne says (C++, 3rd ed. p 5.1.1) "No object is allocated with the 
 address 0. Consequently, 0 acts as a pointer literal, indicating that 
 a pointer doesn't refer to an object." ((I take it this means not only 
 class object instances but anything pointable at all.))

 For some reason I got no compile time nor runtime error from this.


 
 
 I don't see any reason you should. Merely handling null pointers is not 
 an error.
 

 We're using the slice as an rvalue, so this constitutes a dereference 
 of a null pointer. And therefore an error.


 
 
 It doesn't constitute a dereferencing as far as I can tell. You merely 
 define a slice starting at the address null, being 10 chars long.
 
 Should:
 
 char *start = null;
 int len = 10;
 
 be illegal?
 

 (I seem to remember having read somewhere that modern systems don't 
 give user programs address spaces containing the zero address (even 
 virtually), precisely to avoid problems and pitfalls with null. Can 
 anybody confirm this?)

 Further, since NULL is explicitly defined to not point to anything 
 (therefore not even the address zero), at best this code attempts to 
 return a slice out of thin air. In other words, garbage.


 
 
 Yes. But it never touches the garbage.
 

 And specifically, even if there /was/ a zero address to point to, it 
 would still be illegal to return the slice.


 
 
 Why? Should it be illegal to return an invalid pointer too?
 

 The following program does die (in windows) with an Access Violation:

 import std.stdio;

 void main()
 {
         char[] foo = (cast(char*)null)[0..10];

     char c = foo[1];
 }

 And we weren't even accessing foo[0]. :-)


 
 
 AFAIK, most OS never map the virtual memory page 0 to anything, making 
 accesses to this page illegal. With a page size of 4k, accessing 
 anything within [0..4096] will give a guaranteed page fault.
 

 I consider it a bug when the compiler creates such an unusable array.
 I consider it an bug to not error on (cast(char*)null)[0..10].


 
 
 I humbly disagree on both points.


I humbly agree with you, on second thought. :-)

And with Sean and Regan in the other posts.

I admit I forgot that the slice is nothing more than a real estate 
property claim, and not the thing itself. (But it did look scary at 
first sight!)

I seldom need slices... which reminds me:

import std.stdio;
import std.gc;

void main()
{
	int[] ia = new int[100];
	int[] ib = ia[2..4];

	ib[0] = 666;
	ib[1] = 777;

	delete ia;

	std.gc.fullCollect();

	char[] ca = new char[400];

	writefln("ib[0]=%d and ib[1]=%d", ib[0], ib[1]);
}

which of course gives

ib[0]=-1 and ib[1]=-1

as totally expected -- by everyone else except the poor programmer, 
whose code is larger than this example. :-)

Time for Dlint, at least for the programmer who forgot to adhere to the 
Bovine rule.
Sep 12 2006
parent Max Samuha <maxter i.com.ua> writes:
On Wed, 13 Sep 2006 02:26:56 +0300, Georg Wrede
<georg.wrede nospam.org> wrote:


I seldom need slices... which reminds me:

import std.stdio;
import std.gc;

void main()
{
	int[] ia = new int[100];
	int[] ib = ia[2..4];

	ib[0] = 666;
	ib[1] = 777;

	delete ia;

	std.gc.fullCollect();

	char[] ca = new char[400];

	writefln("ib[0]=%d and ib[1]=%d", ib[0], ib[1]);
}

which of course gives

ib[0]=-1 and ib[1]=-1

as totally expected -- by everyone else except the poor programmer, 
whose code is larger than this example. :-)

Time for Dlint, at least for the programmer who forgot to adhere to the 
Bovine rule.


And you even don't need to trigger the garbage collection. delete will
free the whole memory block occupied by the array.

If you want the slice to the array to remain valid, replace 'delete
ia' with 'ia = null'.this will set the array data pointer to null
without freeing the memory and ib will still point to the array's
memory block preventing the garbage collector from collecting it. If
you want to delete the array explicitly, duplicate the slice
beforehand. 

Please correct me if i'm wrong.
Sep 14 2006
prev sibling parent Steve Horne <stephenwantshornenospam100 aol.com> writes:
On Tue, 12 Sep 2006 00:03:55 +0300, Georg Wrede
<georg.wrede nospam.org> wrote:


And if really not, what in the world is

    (cast(char*)null)[0..10]

supposed to mean, or return???


Reminds me of the definition of the offsetof macro in C.

Cast a null pointer to the type of the struct, then take the address
of the needed field, then cast to integer.

Sure, you're dealing with silly addresses - so long as you don't read
or write to them, there's no error. And all you're doing is getting an
offset, presumably to use in pointer arithmetic later on.

How you'd get a practical advantage from your example, I don't know,
but so long as the language doesn't outlaw things on principle, you
can often find useful tricks like that.

-- 
Remove 'wants' and 'nospam' from e-mail.
Sep 20 2006