www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Spec#, nullables and more

reply bearophile <bearophileHUGS lycos.com> writes:
Spec# is a Microsoft language, its development started in 2003, so it's not a
new language. But both its specs and its implementation are unfinished still.
Designing it is a long & hard task.

I have recently found a quite nice introduction to Spec#, "Using the Spec#
Language, Methodology, and Tools to Write Bug-Free Programs" di K. Rustan M.
Leino and Peter Muller (2009):
http://research.microsoft.com/en-us/um/people/leino/papers/krml189.pdf

Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
http://www.rise4fun.com/SpecSharp

Spec# is a "spinoff" of C# 2.0, it is designed to write more correct programs,
using static verification too. This means that writing Spec# code is heavy,
slow, and it may be a bit painful too. So it's meant only for code that needs
to be reliable, not for quick scripts.

Spec# adds only few things to C# 2.0:
- Non-nullable types;
- Statically enforced Contract Programming syntax and semantics;
- A way to list what attributes are modified in a method (similar to my  outer).
- Checked exceptions (like Java ones).

It looks like a short list, but that list is both quite hard to implement, and
changes a lot the idiomatic way you code, so it's not C# any more.

(The Spec# has produced two things: the Code Contracts library for C#4 and a C
compiler that verifies the code).

Spec# looks very well designed and thought out, yet I don't see a lot of people
interested in it. I presume the "Microsoft" tag drags it down a little. And
probably not being really free doesn't help (maybe there is a partially open
source version now). (And maybe people are not so interested in a language for
low-bug-count programs).

Spec# isn't a system language, but a spinoff of Spec# named Sing# is a system
language with low level features too (it has not GC-managed structs and
pointers and some other things).

In Spec# contracts and assertions are verified statically. This changes a lot
the way you code, and requires more brain from the programmer too. Reading the
specs of Spec# it seems the designers have overdo it a bit here and there
(better is not always better), you need to keep in mind some complex rules, and
so on. I think Spec# is borderline usable, I have seen academic languages far
more complex and hard to use than Spec#. For purposes where high integrity is
important (where you may want to use Ada or even Spark) Spec# looks acceptable
to me.

Several of the ideas I have suggested here or I have put in the D Bugzilla are
already implemented (and well designed) in Spec#. I am happy.

The two main features of Spec# are its non-nullable types and its statically
enforced Contract Programming. Its DbC uses a static verifier that I don't
think will ever be added to DMD (despite eventually someone may try to write
some static verifier for D). But DbC Spec# is well designed, and some of those
ideas are useful to fix D design too.

Regarding Spec# non-nullable types I think they don't need a static verifier
(beside the normal simple tests done by a static type system), and I think they
are an additive change for D2, so they have a chance to be implementable for D3
too.

-----------------------------

Three quotations from the little tutorial (krml189.pdf):


Also, regardless of the compiler mode used, both inflections ? and ! are useful
in the implementations of generic classes: if T is a type parameter constrained
to be a reference type, then the naked name T stands for the actual type
parameter (which might be a possibly-null type or a non-null type), T? stands
for the possibly-null version of the type, and T! stands for the non-null
version of the type.

-----------

Regardless of old, an in-parameter mentioned in a method contract always refers
to the value of the parameter on entry (in other words, the fact that the
language allows in-parameters to be used as local variables inside the method
body has no effect on the meaning of the contract), and an out parameter always
refers to the value of the parameter on exit; ref parameters, which are treated
as copy-in copy-out, are sensitive to the use of old.

-----------

Non-null types express a special kind of invariant that needs to be established
by each
constructor. The virtual machine initially sets all fields of a new object to
zero-equivalent
values, in particular, fields of reference types to null. So before the new
object has been
fully initialized, it would be unjustified to assume that non-null fields
actually contain
non-null values.

Until the initialization of an object is completed, we say that the object is
delayed, meaning
that it is in a raw state where we can rely neither on the non-nullness of its
fields nor
on its object invariants. Moreover, field values of delayed objects are
themselves allowed
to be delayed. By default, the this object is delayed inside constructors.

A delayed object is in general not allowed to escape from its constructor.
However, sometimes
it is useful to call methods on delayed objects or to pass a delayed object as
an
argument to a method call. This is permitted if the callee method or its
parameter is
marked with the attribute [Delayed]. The consequence of this choice is that the
method
is then not allowed to assume fields of non-null types to have non-null values,
let alone
assume the object invariant to hold.(a)

An alternative is to mark the constructor with the attribute [NotDelayed]. This
requires
that all non-null fields of the class be initialized before an explicit call of
the superclass
(aka base class) constructor, base. A constructor can make a base call to a
[NotDelayed]
superclass constructor only if it itself is marked as [NotDelayed]. A
consequence of this
design is that after the base call, the this object is fully initialized and no
longer delayed.
Therefore, it can be passed to methods without the [Delayed] attribute.

Fähndrich and Xia [7] describe the details of delayed objects. Examples for
delayed objects
and explicit base calls can be found on the tutorial web page.

a) Any reference-valued parameter of a method, not just the receiver, can be
marked with
[Delayed]. However, there is a bug in the current version of the program
verifier that
makes the verification of methods with more than one [Delayed] parameter
unsound.


The article also suggests the usage of a shorter form to cast to nullable or
not nullable:
cast( )someRef
cast(?)someRef


My reference issue:
http://d.puremagic.com/issues/show_bug.cgi?id=4571

Bye,
bearophile
Nov 04 2010
next sibling parent reply Gary Whatmore <no spam.sp> writes:
bearophile Wrote:

 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp

D can do that too. We had those interactive versions in the newsrgoup. We saw no value in them.
 Spec# adds only few things to C# 2.0:
 - Non-nullable types;

The day D implements non-nullable types is the day I burn my copy of TDPL and stop using D. Why keep you pushin this crap. I don't want to hear about it. It doesn't improve my productivity at all.
 - Statically enforced Contract Programming syntax and semantics;

Too difficult to implement. Not worth it.
 - A way to list what attributes are modified in a method (similar to my
 outer).

The compiler should do this itself.
 - Checked exceptions (like Java ones).

Oh god.. what were they smoking?
 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571

Walter, please close this as wontfix. We don't need those. These extra runtime checks will slow down my code. I know myself when my pointer is null. - G.W.
Nov 05 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:

 I know myself when my pointer is null.

You are unusual then: http://lambda-the-ultimate.org/node/3186 Bye, bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:
 Gary Whatmore:
 
 I know myself when my pointer is null.

You are unusual then: http://lambda-the-ultimate.org/node/3186

The $10 billion mistake was C's conversion of arrays to pointers when passing to a function. http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html Sadly, there's an ongoing failure to recognize this, as it is never addressed in any of the revisions to the C or C++ standards, and is missed by the supposedly "safe C" alternatives.
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:

 The $10 billion mistake was C's conversion of arrays to pointers when passing
to 
 a function.
 
 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html
 
 Sadly, there's an ongoing failure to recognize this, as it is never addressed
in 
 any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs. But Java, C#, and D have solved this problem already, so we can now go on and try to solve other problems. This thread was about a different problem (null-related bugs), currently present in Java, C#, and D too.
 and is missed by the supposedly "safe C" alternatives.

This is probably wrong. I don't know many C alternatives, but the well known Cyclone language uses fat pointers (and other things) to solve that C problem. Bye, bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:
 Walter Bright:
 
 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.
 
 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html
 
 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs.

It's infinitely worse. Null pointers do not result in memory corruption, buffer overflows, and security breaches.
 and is missed by the supposedly "safe C" alternatives.

This is probably wrong. I don't know many C alternatives, but the well known Cyclone language uses fat pointers (and other things) to solve that C problem.

The Cyclone user manual says you have to rewrite a parameter as: void foo(int * numelts(4) arr); to avoid the bugs with: void foo(int arr[]); I think that latter broken syntax is still supported by Cyclone, but with the inadequate manual http://cyclone.thelanguage.org/wiki/User%20Manual it's hard to tell. Oh, and you have to redeclare the C: int sum(int num, int *p); as: int sum(tag_t<`n> num, int * notnull numelts(valueof(`n)) p); No wonder Cyclone failed.
Nov 05 2010
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Walter Bright schrieb:
 bearophile wrote:
 Walter Bright:

 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs.

It's infinitely worse. Null pointers do not result in memory corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it. Cheers, - Daniel
Nov 05 2010
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Daniel Gibson wrote:
 Walter Bright schrieb:
 It's infinitely worse. Null pointers do not result in memory 
 corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.

I'm surprised. 20 years ago, OS design articles I've seen all said that the first thing to do was render the bottom 64Kb of address space inaccessible in order to catch null pointer dereferences. People were pretty fed up with Intel's decision to put the interrupt table in the first page of addresses, so any null pointers promptly trashed the operating system. (Intel should have put the BIOS boot rom at location 0.)
Nov 05 2010
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 4:12 PM, Leandro Lucarella wrote:
 Daniel Gibson, el  5 de noviembre a las 19:52 me escribiste:
 Walter Bright schrieb:
 bearophile wrote:
 Walter Bright:

 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs.

It's infinitely worse. Null pointers do not result in memory corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.

And then, you can corrupt memory with something like: struct S { int[1_000_000_000] data; int far_data; } S* s = null; s.far_data = 5; If you are unlucky enough to end up in a valid address. That might not be a practical example, of course, but theoretically null pointer could lead to memory corruption.

The language may limit the static size of object. That's what Java does - it limits the size of any class to 64KB, and then every VM implementation guarantees that the first 64KB are made verboten one way or another. Andrei
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Andrei Alexandrescu wrote:
 The language may limit the static size of object. That's what Java does 
 - it limits the size of any class to 64KB, and then every VM 
 implementation guarantees that the first 64KB are made verboten one way 
 or another.

I've meant to do that in D, but haven't gotten around to it.
Nov 05 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 19:27:03 -0400, Walter Bright <newshound2 digitalmars.com> said:

 Andrei Alexandrescu wrote:
 The language may limit the static size of object. That's what Java does 
 - it limits the size of any class to 64KB, and then every VM 
 implementation guarantees that the first 64KB are made verboten one way 
 or another.

I've meant to do that in D, but haven't gotten around to it.

On 32-bit OS X, that limit is 4 KB. And what happens if I dereference a null pointer to a static array of 65k elements and I try to read the last one? Disallowing objects longer than 64 KB can help, but it's not a complete solution. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 05 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Michel Fortin wrote:
 On 2010-11-05 19:27:03 -0400, Walter Bright <newshound2 digitalmars.com> 
 said:
 
 Andrei Alexandrescu wrote:
 The language may limit the static size of object. That's what Java 
 does - it limits the size of any class to 64KB, and then every VM 
 implementation guarantees that the first 64KB are made verboten one 
 way or another.

I've meant to do that in D, but haven't gotten around to it.

On 32-bit OS X, that limit is 4 KB.

That's good to know.
 And what happens if I dereference a null pointer to a static array of 
 65k elements and I try to read the last one?

Array index out of bounds.
 Disallowing objects longer than 64 KB can help, but it's not a complete 
 solution.

It's pretty darn close.
Nov 05 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 20:04:11 -0400, Walter Bright <newshound2 digitalmars.com> said:

 Michel Fortin wrote:
 On 32-bit OS X, that limit is 4 KB.

That's good to know.

Well, you should already know. I posted this on the Phobos mailing list in August and you posted a reply. :-)
 And what happens if I dereference a null pointer to a static array of 
 65k elements and I try to read the last one?

Array index out of bounds.

There's nothing out of the array's bounds in this case. Here's what I meant: byte[66000]* arrayPtr = null; byte b = (*arrayPtr)[66000-1]; I'm in the array's bounds here, the problem is that I'm dereferencing a null pointer but the program will actually only read 65999 bytes further, outside of the 64 KB "safe" zone. Should we limit static arrays to 64 KB too? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Michel Fortin wrote:
 On 2010-11-05 20:04:11 -0400, Walter Bright <newshound2 digitalmars.com> 
 said:
 
 Michel Fortin wrote:
 On 32-bit OS X, that limit is 4 KB.

That's good to know.

Well, you should already know. I posted this on the Phobos mailing list in August and you posted a reply. :-)

Yeah, well, my brain is full. In order to learn new facts, I must discard an equivalent number of existing ones. I've had to discard everything I ever learned about chemistry, for example.
 And what happens if I dereference a null pointer to a static array of 
 65k elements and I try to read the last one?

Array index out of bounds.

There's nothing out of the array's bounds in this case. Here's what I meant: byte[66000]* arrayPtr = null; byte b = (*arrayPtr)[66000-1]; I'm in the array's bounds here, the problem is that I'm dereferencing a null pointer but the program will actually only read 65999 bytes further, outside of the 64 KB "safe" zone. Should we limit static arrays to 64 KB too?

That's why pointer arithmetic (which is what this is) is disallowed in safe mode.
Nov 05 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:

 Yeah, well, my brain is full. In order to learn new facts, I must discard an 
 equivalent number of existing ones. I've had to discard everything I ever 
 learned about chemistry, for example.

As a human brain gets older, its ability to quickly retain new information decreases. When you are 18 years old you are able to learn lot of stuff the first time you hear it, while when you are 50 years old you need to listen to the same information some times to learn (unless you are very trained to learn a specific kind of information: a chess master is able to quickly memorize all the moves of a game even at old age). But what you have said is partially wrong. The mammal brain doesn't have a fixed space for information, the more you learn the more space you have to learn, because while neurons are in finite number and their number decreases with age, new and synapses can be build every day, and higher level ways to store information in the semantic network may be invented by a mind. Book titles about what I have said on request :-) Keep learning, bye, bearophile
Nov 05 2010
parent reply Jacob Carlborg <doob me.com> writes:
On 2010-11-06 05:46, bearophile wrote:
 Walter Bright:

 Yeah, well, my brain is full. In order to learn new facts, I must discard an
 equivalent number of existing ones. I've had to discard everything I ever
 learned about chemistry, for example.

As a human brain gets older, its ability to quickly retain new information decreases. When you are 18 years old you are able to learn lot of stuff the first time you hear it, while when you are 50 years old you need to listen to the same information some times to learn (unless you are very trained to learn a specific kind of information: a chess master is able to quickly memorize all the moves of a game even at old age). But what you have said is partially wrong. The mammal brain doesn't have a fixed space for information, the more you learn the more space you have to learn, because while neurons are in finite number and their number decreases with age, new and synapses can be build every day, and higher level ways to store information in the semantic network may be invented by a mind. Book titles about what I have said on request :-) Keep learning, bye, bearophile

Even though he may actually not forget a particular fact it will be harder and harder to find the fact because of other things "blocking the way". Eventaully it will seem you have forgot the fact. -- /Jacob Carlborg
Nov 06 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Jacob Carlborg:

 Even though he may actually not forget a particular fact it will be 
 harder and harder to find the fact because of other things "blocking the 
 way". Eventaully it will seem you have forgot the fact.

It's not a block of the way, it's the networks that accumulate noise or just slowly get used for other purposes. So it's not a matter of space, as Walter has said, but mostly a matter of time: if you find some time to refresh old memories you are able to keep them on the long term. Bye, bearophile
Nov 06 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 23:27:28 -0400, Walter Bright <newshound2 digitalmars.com> said:

 Michel Fortin wrote:
 There's nothing out of the array's bounds in this case. Here's what I meant:
 
     byte[66000]* arrayPtr = null;
     byte b = (*arrayPtr)[66000-1];
 
 I'm in the array's bounds here, the problem is that I'm dereferencing a 
 null pointer but the program will actually only read 65999 bytes 
 further, outside of the 64 KB "safe" zone.
 
 Should we limit static arrays to 64 KB too?

That's why pointer arithmetic (which is what this is) is disallowed in safe mode.

Really? I'm doing two things in "(*arrayPtr)[66000-1]" in the above code: 1. Dereferencing a pointer to a the static array; 2. Accessing the last element within the bounds of the static array. Which of these is disallowed in safe mode? I think you should reread this example more attentively. It's no different than if you had put the static array as a member in a struct. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 05 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Michel Fortin wrote:
 On 2010-11-05 23:27:28 -0400, Walter Bright <newshound2 digitalmars.com> 
 said:
 
 Michel Fortin wrote:
 There's nothing out of the array's bounds in this case. Here's what I 
 meant:

     byte[66000]* arrayPtr = null;
     byte b = (*arrayPtr)[66000-1];

 I'm in the array's bounds here, the problem is that I'm dereferencing 
 a null pointer but the program will actually only read 65999 bytes 
 further, outside of the 64 KB "safe" zone.

 Should we limit static arrays to 64 KB too?

That's why pointer arithmetic (which is what this is) is disallowed in safe mode.

Really? I'm doing two things in "(*arrayPtr)[66000-1]" in the above code: 1. Dereferencing a pointer to a the static array; 2. Accessing the last element within the bounds of the static array. Which of these is disallowed in safe mode? I think you should reread this example more attentively. It's no different than if you had put the static array as a member in a struct.

Looks like you're right.
Nov 06 2010
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 6:54 PM, Michel Fortin wrote:
 On 2010-11-05 19:27:03 -0400, Walter Bright <newshound2 digitalmars.com>
 said:

 Andrei Alexandrescu wrote:
 The language may limit the static size of object. That's what Java
 does - it limits the size of any class to 64KB, and then every VM
 implementation guarantees that the first 64KB are made verboten one
 way or another.

I've meant to do that in D, but haven't gotten around to it.

On 32-bit OS X, that limit is 4 KB. And what happens if I dereference a null pointer to a static array of 65k elements and I try to read the last one? Disallowing objects longer than 64 KB can help, but it's not a complete solution.

It is - it all depends on choosing the right limit for all OSs. Andrei
Nov 05 2010
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 6:27 PM, Walter Bright wrote:
 Andrei Alexandrescu wrote:
 The language may limit the static size of object. That's what Java
 does - it limits the size of any class to 64KB, and then every VM
 implementation guarantees that the first 64KB are made verboten one
 way or another.

I've meant to do that in D, but haven't gotten around to it.

Probably we should bugzillize it. http://d.puremagic.com/issues/show_bug.cgi?id=5176 Andrei
Nov 05 2010
prev sibling next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Leandro Lucarella Wrote:

 And then, you can corrupt memory with something like:
 
 struct S {
 	int[1_000_000_000] data;
 	int far_data;
 }
 
 S* s = null;
 s.far_data = 5;
 
 If you are unlucky enough to end up in a valid address. That might not
 be a practical example, of course, but theoretically null pointer could
 lead to memory corruption.

Interesting, I get this: test.d(13): Error: index 1000000000 overflow for static array I'm not exactly sure what you are trying to demonstrate with it. Are you filling up the stack such that the OS tries to assign to a proper data field? How is it still not trying to access location 0 causing an Access Violation?
Nov 05 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 05/11/2010 18:52, Daniel Gibson wrote:
 Walter Bright schrieb:
 bearophile wrote:
 Walter Bright:

 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs.

It's infinitely worse. Null pointers do not result in memory corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it. Cheers, - Daniel

I think Walter's point remains true: null pointers bugs are an order of magnitude less important, if not downright insignificant, with regards to security breaches. I mean, from my understanding of that article, a NPE bug on its own is not enough to allow an exploit, but other bugs/exploits need to be be present. (in that particular case, a straight-flush of them it seems). On the other hand, buffer overflows bugs nearly always make possible an exploit, correct? -- Bruno Medeiros - Software Engineer
Nov 25 2010
next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/25/2010 10:28 AM, Bruno Medeiros wrote:
 I think Walter's point remains true: null pointers bugs are an order of
 magnitude less important, if not downright insignificant, with regards
 to security breaches.

 I mean, from my understanding of that article, a NPE bug on its own is
 not enough to allow an exploit, but other bugs/exploits need to be be
 present. (in that particular case, a straight-flush of them it seems).
 On the other hand, buffer overflows bugs nearly always make possible an
 exploit, correct?

From a language designer's perspective, I think programmer association of semantic meaning to null (along with null's weak typing) is a more serious problem than npe vulnerabilities.
Nov 25 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 25/11/2010 16:55, Daniel Gibson wrote:
 On Thu, Nov 25, 2010 at 5:28 PM, Bruno Medeiros
 <brunodomedeiros+spam com.gmail>  wrote:
 On 05/11/2010 18:52, Daniel Gibson wrote:
 Walter Bright schrieb:
 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it. Cheers, - Daniel

I think Walter's point remains true: null pointers bugs are an order of magnitude less important, if not downright insignificant, with regards to security breaches.

No, that wasn't his point - he thought it was *impossible* to exploit null pointers ("Null pointers do not result in memory corruption, buffer overflows, and security breaches.") and I merely pointed out that this is not correct. I didn't say anything about significance for average applications :-)

Yes, Walter's statement that it is impossible for a null pointer to cause a security vulnerability is (likely) incorrect. But his point at large, considering the discussion that preceded the comment, was that null pointers are utterly insignificant with regards to security vulnerabilities. And I agree with that, and because of that I'm suprised and curious to understand why Hoare mentioned (in the abstract on the link posted originally), that null pointers have caused "innumerable vulnerabilities.
 I mean, from my understanding of that article, a NPE bug on its own is not
 enough to allow an exploit, but other bugs/exploits need to be be present.

Well it could be used by a non-privileged user to get root privileges. If you only have "friendly" non-privileged users you need an exploit to make them execute the kernel exploit, of course. But I agree that this kind of bug is not as relevant as others (for most users) - you won't have it in regular programs but only in kernels I guess. (Of course it could work in regular programs as well, but you won't get more privileges then you had before. Also I may be completely wrong on this and maybe there is some way to gain something by using this kind of exploit on regular programs.)

By "exploit", I didn't mean to necessarily imply privilege escalation. I meant arbitrary code execution, with or without privilege escalation. (I don't know if this usage of the term is common in the security community, maybe not) So, going back, is it correct to say that an NPE bug on its own is not enough to allow arbitrary code execution, but that other vulnerabilities are necessary? -- Bruno Medeiros - Software Engineer
Nov 26 2010
next sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 17:28, Bruno Medeiros wrote:
 And I agree with that, and because of that I'm suprised and curious to
 understand why Hoare mentioned (in the abstract on the link posted
 originally), that null pointers have caused "innumerable vulnerabilities.

Hum, cool, I just found out that this link: http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare has the actual presentation on it, so I'm gonna take a look. -- Bruno Medeiros - Software Engineer
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 17:54, Bruno Medeiros wrote:
 On 26/11/2010 17:28, Bruno Medeiros wrote:
 And I agree with that, and because of that I'm suprised and curious to
 understand why Hoare mentioned (in the abstract on the link posted
 originally), that null pointers have caused "innumerable vulnerabilities.

Hum, cool, I just found out that this link: http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare has the actual presentation on it, so I'm gonna take a look.

I've seen the presentation, but he doesn't explain how a null pointer access would have caused a vulnerability. I'm going to assume that in all likelihood this applied to older computer architectures and/or OSes that didn't handle null pointer access that gracefully (1965 is way back...). But not so much to modern ones. Or that the vulnerability wasn't an actual arbitrary code execution, but some other system failure caused by the program crashing. In any case this side-topic was just a minor curiosity, it's not really relevant for D. But on his talk as a whole, the general point he made was interesting, he expressed the desire for languages to have more safety and checking, preferably on compile-time, if possible, and if not, on runtime at least (rather than have the program corrupt data, or execute crap). He mentioned that the big argument against this at that time was performance penalties, but that even so a lot of the people/companies were happy with the checks that were introduced (like array bounds checking), even if initially it didn't seem like a good idea. -- Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Bruno Medeiros schrieb:
 On 25/11/2010 16:55, Daniel Gibson wrote:
 On Thu, Nov 25, 2010 at 5:28 PM, Bruno Medeiros
 <brunodomedeiros+spam com.gmail>  wrote:
 On 05/11/2010 18:52, Daniel Gibson wrote:
 Walter Bright schrieb:
 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it. Cheers, - Daniel

I think Walter's point remains true: null pointers bugs are an order of magnitude less important, if not downright insignificant, with regards to security breaches.

No, that wasn't his point - he thought it was *impossible* to exploit null pointers ("Null pointers do not result in memory corruption, buffer overflows, and security breaches.") and I merely pointed out that this is not correct. I didn't say anything about significance for average applications :-)

Yes, Walter's statement that it is impossible for a null pointer to cause a security vulnerability is (likely) incorrect. But his point at large, considering the discussion that preceded the comment, was that null pointers are utterly insignificant with regards to security vulnerabilities. And I agree with that, and because of that I'm suprised and curious to understand why Hoare mentioned (in the abstract on the link posted originally), that null pointers have caused "innumerable vulnerabilities.

I don't know for sure, but I guess there have been multiple vulnerabilities in kernel code (of different operating systems) caused by null pointers. So it may not affect most programmers (that don't do kernel stuff), but there are still a lot of (?) vulnerabilities caused by that. And those are bad, because you mess within the kernel, so you can do *anything*.
 
 I mean, from my understanding of that article, a NPE bug on its own 
 is not
 enough to allow an exploit, but other bugs/exploits need to be be 
 present.

Well it could be used by a non-privileged user to get root privileges. If you only have "friendly" non-privileged users you need an exploit to make them execute the kernel exploit, of course. But I agree that this kind of bug is not as relevant as others (for most users) - you won't have it in regular programs but only in kernels I guess. (Of course it could work in regular programs as well, but you won't get more privileges then you had before. Also I may be completely wrong on this and maybe there is some way to gain something by using this kind of exploit on regular programs.)

By "exploit", I didn't mean to necessarily imply privilege escalation. I meant arbitrary code execution, with or without privilege escalation. (I don't know if this usage of the term is common in the security community, maybe not)

Yes. I guess (haven't dug too far into this) that NP dereferences are mostly used to exploit the kernel, because you mmap() to NULL in the unprivileged userland program and this mmap() affects kernel code. I don't know if it's possible to have mmap() affect userland programs other than the one that called it. If it isn't possible you'd have to make the programm mmap() stuff to NULL which means you've already exploited it, so no need to use some NP dereference hack.
 
 So, going back, is it correct to say that an NPE bug on its own is not 
 enough to allow arbitrary code execution, but that other vulnerabilities 
 are necessary?
 

I don't think it's correct: You may have a "bad" user on your system (e.g. pseudo-public server like in universities) who executes code that exploits the NPE bug and gains root/kernel privileges that way. You only need another vulnerability if there are no "bad users" on your system => the only way to execute bad code is via an exploit (e.g. in your webbrowser or one of its plugins etc) But, to make it clear: I'm no expert on this, I just read a few reports on that null pointer related kernel exploit. Cheers, - Daniel
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:20, Daniel Gibson wrote:
 So, going back, is it correct to say that an NPE bug on its own is not
 enough to allow arbitrary code execution, but that other
 vulnerabilities are necessary?

I don't think it's correct: You may have a "bad" user on your system (e.g. pseudo-public server like in universities) who executes code that exploits the NPE bug and gains root/kernel privileges that way. You only need another vulnerability if there are no "bad users" on your system => the only way to execute bad code is via an exploit (e.g. in your webbrowser or one of its plugins etc)

You're still thinking in terms of compromising a machine. I didn't mean specifically that, forget users and hosts, I was thinking of a more general and simpler case which relates to a process only: can the input to some program P cause arbitrary code execution, when it shouldn't? Like, can an image file cause my image viewer to run arbitrary code, can a zip file to the same for my zip extractor, etc.. (the same if the program is a server process and the input comes across the network) -- Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 11/26/2010 10:28, Bruno Medeiros wrote:
 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.

I really hate this way of thinking. Security vulnerabilities are binary - either they exist or they don't. Every security vulnerability seems minor until it is exploited. Yes, some security vulnerabilities are more likely to be exploited than others. But instead of rationalizing about how significant each individual security vulnerability is, isn't it better to just fix all of them? (I know, I'm a hopeless idealist.) -- Rainer Deyke - rainerd eldwood.com
Nov 26 2010
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Rainer Deyke schrieb:
 On 11/26/2010 10:28, Bruno Medeiros wrote:
 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.

I really hate this way of thinking. Security vulnerabilities are binary - either they exist or they don't. Every security vulnerability seems minor until it is exploited. Yes, some security vulnerabilities are more likely to be exploited than others. But instead of rationalizing about how significant each individual security vulnerability is, isn't it better to just fix all of them? (I know, I'm a hopeless idealist.)

Of course you should fix all of them, but the (sensible) language level support to prevent them may vary. e.g. buffer overflows are a huge problem, so D has builtin index checking for arrays and such. Null Pointer dereference exploits are - as far as I know - only relevant in kernel code and if mmaping to NULL is possible. So it's probably not a common problem in D code and people who write code that may be affected can take care themselves. Cheers, - Daniel
Nov 26 2010
parent %u <webnews digitalmars.com> writes:
Daniel Gibson Wrote:

 Rainer Deyke schrieb:
 On 11/26/2010 10:28, Bruno Medeiros wrote:
 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.

I really hate this way of thinking. Security vulnerabilities are binary - either they exist or they don't. Every security vulnerability seems minor until it is exploited. Yes, some security vulnerabilities are more likely to be exploited than others. But instead of rationalizing about how significant each individual security vulnerability is, isn't it better to just fix all of them? (I know, I'm a hopeless idealist.)

Of course you should fix all of them, but the (sensible) language level support to prevent them may vary. e.g. buffer overflows are a huge problem, so D has builtin index checking for arrays and such. Null Pointer dereference exploits are - as far as I know - only relevant in kernel code and if mmaping to NULL is possible. So it's probably not a common problem in D code and people who write code that may be affected can take care themselves.

Even if eliminating unnecessary NPEs or segfaults doesn't prevent many significant security problems, it can improve the user experience in tremendous ways. I'm an embedded / qt / android software engineer working on Linux. My (graphical) tools experience random segfaults in the range of tens of times per a single work day. Reloading the same tools over and over again and restoring the lost data may actually waste almost 10% of my daily work time. Imagine if the industry lost 10% of its income because of sloppy tools written in too low level languages, would that make the feature significant enough? Instead of using automatic GUI builders and such, the tools we use are often hand written C or C++. Why these happen? We forget to initialize or fail to track the lifetime properly.
Nov 26 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 27/11/2010 03:29, Rainer Deyke wrote:
 On 11/26/2010 10:28, Bruno Medeiros wrote:
 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.

I really hate this way of thinking. Security vulnerabilities are binary - either they exist or they don't. Every security vulnerability seems minor until it is exploited. Yes, some security vulnerabilities are more likely to be exploited than others. But instead of rationalizing about how significant each individual security vulnerability is, isn't it better to just fix all of them? (I know, I'm a hopeless idealist.)

You missed the point. The point wasn't that a vulnerability caused by a null pointer access was less serious or significant than a vulnerabitiy caused by a buffer overrun. Once a vulnerability exists, it should be fixed regardless, yes. The point was that if you have a null pointer access *bug*, that bug is incredibly less likely to create a *vulnerability* than a buffer overrun *bug*. Note that "creating a vulnerability" means "making it *possible* to exploit the program", it does not mean "someone actually exploiting the vulnerability". -- Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/05/2010 01:39 PM, Walter Bright wrote:
 It's infinitely worse. Null pointers do not result in memory corruption,
 buffer overflows, and security breaches.

Not true. Null pointer dereference exploits are difficult, but very real.
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Ellery Newcomer wrote:
 On 11/05/2010 01:39 PM, Walter Bright wrote:
 It's infinitely worse. Null pointers do not result in memory corruption,
 buffer overflows, and security breaches.


Is that like worrying that an airliner will crash into your house and explode?
Nov 05 2010
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/05/2010 02:24 PM, Walter Bright wrote:
 Ellery Newcomer wrote:
 On 11/05/2010 01:39 PM, Walter Bright wrote:
 It's infinitely worse. Null pointers do not result in memory corruption,
 buffer overflows, and security breaches.


Is that like worrying that an airliner will crash into your house and explode?

Don't know. Has anyone ever crashed into my house with an airliner and lived to tell about it?
Nov 05 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Fri, 05 Nov 2010 14:43:48 +0300, Gary Whatmore <no spam.sp> wrote:

 bearophile Wrote:

 Plus a nice Microsoft site that allows you to try it in interactive  
 way, this is very good:
 http://www.rise4fun.com/SpecSharp

D can do that too. We had those interactive versions in the newsrgoup. We saw no value in them.
 Spec# adds only few things to C# 2.0:
 - Non-nullable types;

The day D implements non-nullable types is the day I burn my copy of TDPL and stop using D. Why keep you pushin this crap. I don't want to hear about it. It doesn't improve my productivity at all.

Is anyone FORCING you to use non-nullable references? What's the big deal?
Nov 05 2010
prev sibling next sibling parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 12:43 PM, Gary Whatmore wrote:
 bearophile Wrote:
 - A way to list what attributes are modified in a method (similar to my
 outer).

The compiler should do this itself.

Doesn't make sense.
 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571

Walter, please close this as wontfix. We don't need those. These extra runtime checks will slow down my code. I know myself when my pointer is null. - G.W.

How, exactly, do you know when your references are null? Without runtime checks, of course.
Nov 05 2010
parent reply Gary Whatmore <no spam.sp> writes:
Pelle Månsson Wrote:

 On 11/05/2010 12:43 PM, Gary Whatmore wrote:
 bearophile Wrote:
 - A way to list what attributes are modified in a method (similar to my
 outer).

The compiler should do this itself.

Doesn't make sense.
 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571

Walter, please close this as wontfix. We don't need those. These extra runtime checks will slow down my code. I know myself when my pointer is null. - G.W.

How, exactly, do you know when your references are null? Without runtime checks, of course.

Good code design makes the null pointer exceptions go away. I carefully ponder what code goes where. Simple as that. That language just introduces a boatload of unnecessary cruft in forms of runtime null checks. I don't need to know the exact temporal location of nulls, it's enough if the code takes care of handling it at run time.
Nov 05 2010
next sibling parent =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 02:48 PM, Gary Whatmore wrote:
 Pelle Månsson Wrote:

 On 11/05/2010 12:43 PM, Gary Whatmore wrote:
 bearophile Wrote:
 - A way to list what attributes are modified in a method (similar to my
 outer).

The compiler should do this itself.

Doesn't make sense.
 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571

Walter, please close this as wontfix. We don't need those. These extra runtime checks will slow down my code. I know myself when my pointer is null. - G.W.

How, exactly, do you know when your references are null? Without runtime checks, of course.

Good code design makes the null pointer exceptions go away. I carefully ponder what code goes where. Simple as that. That language just introduces a boatload of unnecessary cruft in forms of runtime null checks. I don't need to know the exact temporal location of nulls, it's enough if the code takes care of handling it at run time.

Say you write a library, with a class and a function. Something like this: class C { /* stuff */ } void foo(C[] cs) { foreach (c; cs) { // do stuff with c } } How do you handle null, in this case?
Nov 05 2010
prev sibling parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Gary Whatmore wrote:
 Pelle M=EF=BF=BDnsson Wrote:
=20
 On 11/05/2010 12:43 PM, Gary Whatmore wrote:
 bearophile Wrote:
 - A way to list what attributes are modified in a method (similar to=




 The compiler should do this itself.

 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=3D4571




null.
   - G.W.

runtime checks, of course.

Good code design makes the null pointer exceptions go away. I carefully=

es a boatload of unnecessary cruft in forms of runtime null checks. I don= 't need to know the exact temporal location of nulls, it's enough if the = code takes care of handling it at run time. Who said anything about runtime checks? Non-nullable types are enforced at compile-time anyway... Jerome --=20 mailto:jeberger free.fr http://jeberger.free.fr Jabber: jeberger jabber.fr
Nov 05 2010
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 09:51:07 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:

 FWIW, I got word that nullable types are considered failure in C#'s 
 design team. They were added in a haste and now the designers regret 
 that.

But did they refer to nullable pointers or the ability to make regular value types (integers, etc.) nullable? These are actually two different things. <http://msdn.microsoft.com/en-us/library/1t3y8s4s(VS.80).aspx> -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 05 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 05 Nov 2010 09:48:50 -0400
Gary Whatmore <no spam.sp> wrote:

 Pelle M=C3=A5nsson Wrote:
=20
 How, exactly, do you know when your references are null? Without=20
 runtime checks, of course.

Good code design makes the null pointer exceptions go away. I carefully p=

boatload of unnecessary cruft in forms of runtime null checks. I don't nee= d to know the exact temporal location of nulls, it's enough if the code tak= es care of handling it at run time. Good code design makes type errors go away. Just have a clear model of the = app, carefully match your types with true notions of the domain. Simple as = that. Who gets type errors, anyway? Do you? I don't. It's just unnecessary load over the compiler dev team, isn't it? Let's get = rid of that cruft! Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 05 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Gary Whatmore wrote:
 bearophile Wrote:
 
 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp

D can do that too. We had those interactive versions in the newsrgoup. We saw no value in them.

Here it is: http://codepad.org/ It's been around a while, and nobody cares. Such a capability is marketing puffery, and not useful.
Nov 05 2010
next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Walter Bright Wrote:

 Gary Whatmore wrote:
 bearophile Wrote:
 
 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp

D can do that too. We had those interactive versions in the newsrgoup. We saw no value in them.

Here it is: http://codepad.org/ It's been around a while, and nobody cares. Such a capability is marketing puffery, and not useful.

Sadly that site uses some Unknown version of D1. http://www.ideone.com is using dmd 2.042 though. I don't these sites are very good what you want to learn a new language, but the are nice to demonstrate code snippets or test something out when you do know the language.
Nov 05 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:

 Here it is:
 
 http://codepad.org/
 
 It's been around a while, and nobody cares. Such a capability is marketing 
 puffery, and not useful.

There are two sites that allow to compile D code online: codepad (D1) and ideone (D2). Even if ideone site is just "marketing puffery" for D, recent discussions on this newsgroup have shown that Andrei is willing to do something for marketing purposes. And this kind of ideone marketing is not going to worsen the API of Phobos, it's just a different way to use D compiler. So it doesn't harm. If you take a look at the Go site you see a way to try code: http://golang.org/ Or even LLVM compiler: http://llvm.org/demo/ I have recently discussed with ideone devs to suggest a new feature, and it seems their site is having enough visitors, so it seems some people care about the site. I have personally found ideone and codepad sites useful, because they allow you to try snippets of code in languages you don't know yet. So you can try a new language in a very fast and painless way, without committing to install a compiler/interpreter to your PC. Today web and being online are very important things, and ideone/codepad allow even compiled languages to be tried online by young people used to javascript and browsers. I have used ideone to try small programs in Scala, Haskell, and other languages. I have used ideone to compile small snippets of D2 code in places where I don't have a D2 compiler available. If you go take a visit to the IRC #D channel you see an tool based on codepad that allows to try snippets of code from the IRC channel itself. A tool often used by feep and others. So you are wrong, ideone/codepad are used by people, are good marketing, allow people to try D before installing the compiler and they don't harm D in any way. What I have suggested in another post is not to put a online demo window in the D site, but to just add a *link* to ideone and codepad in the D home page, so people may try D online. Bye, bearophile
Nov 05 2010
next sibling parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 11/5/10 10:36 AM, bearophile wrote:
 What I have suggested in another post is not to put a online demo window in
the D site, but to just add a *link* to ideone and codepad in the D home page,
so people may try D online.

When I started the redesign of the D site, one of my first ideas was to add links for all code snippets to a copy of the code on codepad (I didn't know about ideone), so that people could see them in action. Later I decided that this might not have as much value as I thought, since many of the snippets depended on command-line arguments and I didn't see a way to provide them. Nonetheless I still think that providing these kind of links would have value to newcomers wanting to learn and experiment with the language.
Nov 05 2010
parent bearophile <bearophileHUGS lycos.com> writes:
David Gileadi:

 Later I decided that this might not have as much value as I thought, 
 since many of the snippets depended on command-line arguments and I 
 didn't see a way to provide them.

ideone allows you to give command line arguments/input too to D programs (but you can't change compilation switches yet, I have suggested that too to the ideone authors). Bye, bearophile
Nov 05 2010
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:
 If you take a look at the Go site you see a way to try code:
 http://golang.org/

Yes, I know about that site and the accolades heaped on Go for having it. Meanwhile, codepad has had it for years, and comeaucomputing has had it for at least 20 years now (for Comeau C++), and nobody cared. I agree they do no harm, but once you're past the gee-whiz factor, they aren't of much use.
Nov 05 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:

 I agree they do no harm, but once you're past the gee-whiz factor, they aren't 
 of much use.

Yet, I will keep using ideone almost daily :-) Bye, bearophile
Nov 05 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Gary Whatmore wrote:
 - Checked exceptions (like Java ones).

Oh god.. what were they smoking?

Checked exceptions are one of those ideas that look great on paper but are an utter failure in practice. As Bruce Eckel pointed out, they are *worse* than useless and *cause* bugs to be inserted into the code.
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:

 Checked exceptions are one of those ideas that look great on paper but are an 
 utter failure in practice. As Bruce Eckel pointed out, they are *worse* than 
 useless and *cause* bugs to be inserted into the code.

(Just to avoid possible misunderstandings: I have never suggested to add checked exceptions to D). I agree that checked exceptions are a pain in a general purpose language. But Spec# isn't a general purpose language, it's designed to be a high integrity language, where the user is supposed to endure some pain in the hope to produce statically verified (and less buggy) programs. So while checked exceptions are probably a bad idea for a handy general purpose language, the authors of Spec# have thought that for the special purposes of Spec# those exceptions are justified. I don't know if they are right (maybe they are wrong, surely not everything in Spec# design is perfect, despite it generally looks like a really well though out language). But you need to keep in account the quite special purpose of Spec# before judging if adding checked exceptions to Spec# is a bad idea. Bye, bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:
 Walter Bright:
 
 Checked exceptions are one of those ideas that look great on paper but are
 an utter failure in practice. As Bruce Eckel pointed out, they are *worse*
 than useless and *cause* bugs to be inserted into the code.

(Just to avoid possible misunderstandings: I have never suggested to add checked exceptions to D).

I just have a hard time seeing that Spec# is an advanced language incorporating the latest in comp sci thought when it adds checked exceptions.
 I agree that checked exceptions are a pain in a general purpose language. But
 Spec# isn't a general purpose language, it's designed to be a high integrity
 language, where the user is supposed to endure some pain in the hope to
 produce statically verified (and less buggy) programs. So while checked
 exceptions are probably a bad idea for a handy general purpose language, the
 authors of Spec# have thought that for the special purposes of Spec# those
 exceptions are justified. I don't know if they are right (maybe they are
 wrong, surely not everything in Spec# design is perfect, despite it generally
 looks like a really well though out language). But you need to keep in
 account the quite special purpose of Spec# before judging if adding checked
 exceptions to Spec# is a bad idea.

I think you misunderstand why checked exceptions are such a bad idea. It's not just that they are inconvenient and annoying. They decrease security by *hiding* bugs. That is the opposite of what you'd want in a high security language. http://www.mindview.net/Etc/Discussions/CheckedExceptions Spec# and Cyclone both fall into the trap of making the right thing to do hard, and the wrong thing easy.
Nov 05 2010
next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Walter Bright Wrote:

 I think you misunderstand why checked exceptions are such a bad idea. It's not 
 just that they are inconvenient and annoying. They decrease security by
*hiding* 
 bugs. That is the opposite of what you'd want in a high security language.

Not only is swallowing the exception a problem, but exceptions aren't added liberally in the code since every exception thrown/renamed effects all calling code. I'm still not in the habit of asserting assumptions.
Nov 05 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 05/11/2010 18:52, Walter Bright wrote:
 I think you misunderstand why checked exceptions are such a bad idea.
 It's not just that they are inconvenient and annoying. They decrease
 security by *hiding* bugs. That is the opposite of what you'd want in a
 high security language.

 http://www.mindview.net/Etc/Discussions/CheckedExceptions

Just to clarify: Checked Exceptions are not a source of bugs per se. What is a source of bugs is catch-hiding an exception temporarily and then forgetting to change the code later (that's the case Bruce mentions in the article). But with discipline you can avoid this: just don't catch-hide exceptions. One never catch-hides exceptions by mistake, it is always conscious (unlike other bugs like off-by-ones, logic errors, etc.). For example in Java I *always* wrap exceptions I don't care about in a RuntimeException, (using the adapter Bruce presented in that article, actually). Is it annoying and/or unnecessary? Well, I'm not making a statement about that, just that it will only actually cause bugs if you are lazy. -- Bruno Medeiros - Software Engineer
Nov 25 2010
prev sibling next sibling parent retard <re tard.com.invalid> writes:
Fri, 05 Nov 2010 13:36:18 -0400, bearophile wrote:
 Walter:
 
 http://codepad.org/
 
 It's been around a while, and nobody cares. Such a capability is
 marketing puffery, and not useful.


 If you go take a visit to the IRC #D channel you see an tool based on
 codepad that allows to try snippets of code from the IRC channel itself.
 A tool often used by feep and others.
 
 So you are wrong, ideone/codepad are used by people, are good marketing,
 allow people to try D before installing the compiler and they don't harm
 D in any way.

It feels like these introvert academic profs and compiler developers with huge beards are actually getting more pragmatic than our BDFL. They regularly visit their IRC channels and are fluently using these online tools. They even adopted open source version control tools and public repositories before Walter did. Just take a look at Scala's trac.. This shouting from the "pragmatic" ivory tower is getting out of touch with reality.
Nov 05 2010
prev sibling next sibling parent retard <re tard.com.invalid> writes:
Fri, 05 Nov 2010 13:40:59 -0400, bearophile wrote:

 Walter Bright:
 
 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.
 
 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html
 
 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs. But Java, C#, and D have solved this problem already, so we can now go on and try to solve other problems. This thread was about a different problem (null-related bugs), currently present in Java, C#, and D too.

One first needs to study these more modern languages before he/she can form any kind of relevant opinions.
 and is missed by the supposedly "safe C" alternatives.

This is probably wrong. I don't know many C alternatives

The primary competition of D is all "safe/better" C clones, and naturally C and C++. D is trying to beat C and C++. To me it seems like the authors fail to see how and why more modern languages succeed.
Nov 05 2010
prev sibling next sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
On Thu, Nov 25, 2010 at 5:28 PM, Bruno Medeiros
<brunodomedeiros+spam com.gmail> wrote:
 On 05/11/2010 18:52, Daniel Gibson wrote:
 Walter Bright schrieb:
 bearophile wrote:
 Walter Bright:

 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs.

It's infinitely worse. Null pointers do not result in memory corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it. Cheers, - Daniel

I think Walter's point remains true: null pointers bugs are an order of magnitude less important, if not downright insignificant, with regards to security breaches.

No, that wasn't his point - he thought it was *impossible* to exploit null pointers ("Null pointers do not result in memory corruption, buffer overflows, and security breaches.") and I merely pointed out that this is not correct. I didn't say anything about significance for average applications :-)
 I mean, from my understanding of that article, a NPE bug on its own is not
 enough to allow an exploit, but other bugs/exploits need to be be present.

Well it could be used by a non-privileged user to get root privileges. If you only have "friendly" non-privileged users you need an exploit to make them execute the kernel exploit, of course. But I agree that this kind of bug is not as relevant as others (for most users) - you won't have it in regular programs but only in kernels I guess. (Of course it could work in regular programs as well, but you won't get more privileges then you had before. Also I may be completely wrong on this and maybe there is some way to gain something by using this kind of exploit on regular programs.)
 (in that particular case, a straight-flush of them it seems). On the other
 hand, buffer overflows bugs nearly always make possible an exploit, correct?

I never said that buffer overflow bugs are better ;-) Cheers, - Daniel
Nov 25 2010
prev sibling parent spir <denis.spir gmail.com> writes:
On Thu, 25 Nov 2010 10:54:39 -0600
Ellery Newcomer <ellery-newcomer utulsa.edu> wrote:

 From a language designer's perspective, I think programmer association=20
 of semantic meaning to null (along with null's weak typing) is a more=20
 serious problem than npe vulnerabilities.

True! An unfortunately rarely pointed issue and somewhat nebulous to expose. I had the idea of an UNDEF value on the implementation side that could not = be used/assigned explicitely by the programmer. It would just allow idioms = like if (x) {} // if x is not undefined... by beeing under the hood logically equal to false, but any operation, even = toString (I mean writeTo ;-) would raise an UndefinedSymbol exception. Sinc= e it's not assignable, a programmer cannot load it with app semantics (eg: = author name unknown, element not found, invalid color code... whatever). A = trial to get the best of both worlds. Just remains the problem of false/UND= EF not beeing distinct in logical expressions. Denis -- -- -- -- -- -- -- vit esse estrany =E2=98=A3 spir.wikidot.com
Nov 25 2010
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/04/2010 03:44 PM, bearophile wrote:
 Spec# is a Microsoft language, its development started in 2003, so it's not a
new language. But both its specs and its implementation are unfinished still.
Designing it is a long&  hard task.

 I have recently found a quite nice introduction to Spec#, "Using the Spec#
Language, Methodology, and Tools to Write Bug-Free Programs" di K. Rustan M.
Leino and Peter Muller (2009):
 http://research.microsoft.com/en-us/um/people/leino/papers/krml189.pdf

 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp

hey, cool stumbled on sing# a while ago and thought it was intriguing, or at least the fact that ms was using it to write an OS kernel
Nov 05 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Ellery Newcomer:

 hey, cool
 
 stumbled on sing# a while ago and thought it was intriguing, or at least 
 the fact that ms was using it to write an OS kernel

It contains a ton of new computer science ideas :-) So it's interesting regardless its applications. (If you don't keep yourself updated with new computer science ideas, you can't keep being a productive programmer for many years). Bye, bearophile
Nov 05 2010
parent reply Gary Whatmore <no spam.sp> writes:
bearophile Wrote:

 Ellery Newcomer:
 
 hey, cool
 
 stumbled on sing# a while ago and thought it was intriguing, or at least 
 the fact that ms was using it to write an OS kernel

It contains a ton of new computer science ideas :-) So it's interesting regardless its applications. (If you don't keep yourself updated with new computer science ideas, you can't keep being a productive programmer for many years).

Are you saying that Walter Bright or anyone else here isn't as productive as you are because we haven't read about the latest language research done between 1980 and 2010? Keeping yourself updated with new ideas also means pragmatic real world books, not just ivory tower research papers that usually aren't freely accessible without an expensive subscription: http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882/ http://www.amazon.com/Domain-Specific-Languages-Addison-Wesley-Signature-Martin/dp/0321712943/ http://www.amazon.com/D-Programming-Language-Andrei-Alexandrescu/dp/0321635361/ http://www.amazon.com/Lean-Architecture-Agile-Software-Development/dp/0470684208/ http://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959/ http://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/dp/020161622X/ http://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/dp/0123814723/ http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672/ http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530/ http://www.amazon.com/Version-Control-Git-Collaborative-Development/dp/0596520123/ http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052/ Do I need to say more?
Nov 05 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:

 Are you saying that Walter Bright or anyone else here isn't as productive as
you are because we haven't read about the latest language research done between
1980 and 2010?

I have found that I need to play with some "new" (*) CS ideas new and then, if I want to keep my mind flexible and open, and keep my programming skills sharp in the long term. (*) I am aware that most of those "new" ideas are up to 30-40 years old. But often a new implementation too is an interesting thing. Bye, bearophile
Nov 05 2010
prev sibling parent retard <re tard.com.invalid> writes:
Fri, 05 Nov 2010 09:53:50 -0400, Gary Whatmore wrote:

 bearophile Wrote:
 
 Ellery Newcomer:
 
 hey, cool
 
 stumbled on sing# a while ago and thought it was intriguing, or at
 least the fact that ms was using it to write an OS kernel

It contains a ton of new computer science ideas :-) So it's interesting regardless its applications. (If you don't keep yourself updated with new computer science ideas, you can't keep being a productive programmer for many years).

Are you saying that Walter Bright or anyone else here isn't as productive as you are because we haven't read about the latest language research done between 1980 and 2010? Keeping yourself updated with new ideas also means pragmatic real world books, not just ivory tower research papers that usually aren't freely accessible without an expensive subscription: http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/

 http://www.amazon.com/Domain-Specific-Languages-Addison-Wesley-

 http://www.amazon.com/D-Programming-Language-Andrei-Alexandrescu/

 http://www.amazon.com/Lean-Architecture-Agile-Software-Development/

 http://www.amazon.com/Mythical-Man-Month-Software-Engineering-

 http://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/

 http://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/

 http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/

 http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530/
 http://www.amazon.com/Version-Control-Git-Collaborative-Development/

 http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/

 
 Do I need to say more?

Those all books deal with tools and non-academic best practices. Probably useful information, but it's not necessarily built on previous (written) knowledge about the domain. The more academic papers discuss why some algoritm works, how to measure it analytically, how language constructs are built from smaller pieces etc. You should at least read few of those to get a better view of the literature. Programming language research has indeed found new techniques and results since the idea of OOP came some 30 to 40 years ago. Nothing suggests that OOP is the be-all and end-all of programming language design. See the links at http://harmful.cat-v.org/software/OO_programming/ Even D is slowly adopting real lambdas, algebraic data types, tuples, and all kinds of functional features.
Nov 05 2010
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
bearophile Wrote:

 Spec# adds only few things to C# 2.0:
 - Non-nullable types;

It's hard to tell, whether they fix anything. When you cast nullable to non-nullable, you get your runtime exception as usual, if you if out access to nullable (e.g. in delayed method), you get your runtime exception again or rather logical bug.
Nov 05 2010
parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 02:39 PM, Kagamin wrote:
 bearophile Wrote:

 Spec# adds only few things to C# 2.0:
 - Non-nullable types;

It's hard to tell, whether they fix anything. When you cast nullable to non-nullable, you get your runtime exception as usual, if you if out access to nullable (e.g. in delayed method), you get your runtime exception again or rather logical bug.

Getting the error early is actually a lot better than getting the error late.
Nov 05 2010
next sibling parent reply Gary Whatmore <no spam.sp> writes:
Pelle Månsson Wrote:

 On 11/05/2010 02:39 PM, Kagamin wrote:
 bearophile Wrote:

 Spec# adds only few things to C# 2.0:
 - Non-nullable types;

It's hard to tell, whether they fix anything. When you cast nullable to non-nullable, you get your runtime exception as usual, if you if out access to nullable (e.g. in delayed method), you get your runtime exception again or rather logical bug.

Getting the error early is actually a lot better than getting the error late.

Getting the error early means that less code compiles and that makes the rapid development fail and turns it into a waterfall misery. It's important to make your tests run quickly in the background. One reason I prefer Python is that it let's me run even (semantically) buggy code, because syntactical correctness is enough. It really improves productivity.
Nov 05 2010
parent =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 02:56 PM, Gary Whatmore wrote:
 Pelle Månsson Wrote:

 On 11/05/2010 02:39 PM, Kagamin wrote:
 bearophile Wrote:

 Spec# adds only few things to C# 2.0:
 - Non-nullable types;

It's hard to tell, whether they fix anything. When you cast nullable to non-nullable, you get your runtime exception as usual, if you if out access to nullable (e.g. in delayed method), you get your runtime exception again or rather logical bug.

Getting the error early is actually a lot better than getting the error late.

Getting the error early means that less code compiles and that makes the rapid development fail and turns it into a waterfall misery. It's important to make your tests run quickly in the background. One reason I prefer Python is that it let's me run even (semantically) buggy code, because syntactical correctness is enough. It really improves productivity.

Yes, let's turn off compiler errors entirely!
Nov 05 2010
prev sibling parent reply Kagamin <spam here.lot> writes:
Pelle Månsson Wrote:

 Getting the error early is actually a lot better than getting the error 
 late.

OK, but it doesn't reduce the number of bugs. You had an error with nullables and you still has error with non-nullables.
Nov 05 2010
parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 03:04 PM, Kagamin wrote:
 Pelle Månsson Wrote:

 Getting the error early is actually a lot better than getting the error
 late.

OK, but it doesn't reduce the number of bugs. You had an error with nullables and you still has error with non-nullables.

But in the non-nullable version you actually know where the bug is, namely where you assign the null to the thing that shouldn't be null. The segfault can come from any unrelated part of the program whereto your null has slipped, at any later point in time.
Nov 05 2010
parent reply Don <nospam nospam.com> writes:
Pelle Månsson wrote:
 On 11/05/2010 03:04 PM, Kagamin wrote:
 Pelle Månsson Wrote:

 Getting the error early is actually a lot better than getting the error
 late.

OK, but it doesn't reduce the number of bugs. You had an error with nullables and you still has error with non-nullables.

But in the non-nullable version you actually know where the bug is, namely where you assign the null to the thing that shouldn't be null. The segfault can come from any unrelated part of the program whereto your null has slipped, at any later point in time.

I've always found it very easy to work out where a null came from. What I would hope from a non-nullable type is to eliminate rare code paths where a null can occur, which might not show up in testing.
Nov 05 2010
next sibling parent dennis luehring <dl.soluz gmx.net> writes:
Am 05.11.2010 15:27, schrieb Don:
 Pelle Månsson wrote:
  On 11/05/2010 03:04 PM, Kagamin wrote:
  Pelle Månsson Wrote:

  Getting the error early is actually a lot better than getting the error
  late.

OK, but it doesn't reduce the number of bugs. You had an error with nullables and you still has error with non-nullables.

But in the non-nullable version you actually know where the bug is, namely where you assign the null to the thing that shouldn't be null. The segfault can come from any unrelated part of the program whereto your null has slipped, at any later point in time.

I've always found it very easy to work out where a null came from. What I would hope from a non-nullable type is to eliminate rare code paths where a null can occur, which might not show up in testing.

or better -> "how many UNTESTED null-able situations out there?" i think an enormous amount of code that just explodes when an null runs through it good example is c# - because of (nearly) everything is nullable in c# is see code like if( x != null ){ if( y != null ){ if( blub != null ... checks all day long, and if i don't see code like this does not mean that it is safer then...
Nov 05 2010
prev sibling parent Roman Ivanov <isroman.DEL ETE.km.ru> writes:
On 11/5/2010 10:27 AM, Don wrote:
 Pelle MÃ¥nsson wrote:
 On 11/05/2010 03:04 PM, Kagamin wrote:
 Pelle MÃ¥nsson Wrote:

 Getting the error early is actually a lot better than getting the error
 late.

OK, but it doesn't reduce the number of bugs. You had an error with nullables and you still has error with non-nullables.

But in the non-nullable version you actually know where the bug is, namely where you assign the null to the thing that shouldn't be null. The segfault can come from any unrelated part of the program whereto your null has slipped, at any later point in time.

I've always found it very easy to work out where a null came from.

I hadn't. It really depends on what kind of code you're dealing with. I've seen null-related bugs that took many, many hours to trace, because the code was using several third-party libraries. For example, library A can return a null in some obscure case without documenting that behavior. Library B can interpret null as a meaningful value and try to do something. You code sits in between, does some logic and passes information from A to B. Stuff like that is _very_ difficult to debug, and having non-nullable types would make it much, much easier.
 What
 I would hope from a non-nullable type is to eliminate rare code paths
 where a null can occur, which might not show up in testing.

Nov 05 2010
prev sibling next sibling parent Leandro Lucarella <luca llucax.com.ar> writes:
Daniel Gibson, el  5 de noviembre a las 19:52 me escribiste:
 Walter Bright schrieb:
bearophile wrote:
Walter Bright:

The $10 billion mistake was C's conversion of arrays to pointers when
passing to a function.

http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

Sadly, there's an ongoing failure to recognize this, as it is never
addressed in any of the revisions to the C or C++ standards,

I agree, that's a very bad problem, probably worse than null-related bugs.

It's infinitely worse. Null pointers do not result in memory corruption, buffer overflows, and security breaches.

Not entirely true: Null Pointer dereferences *have* been used for security breaches, see for example: http://lwn.net/Articles/342330/ The problem is that one can mmap() to 0/NULL so it can be dereferenced without causing a crash. Of course this is also a problem of the OS, it shouldn't allow mmap()ing to NULL in the first place (it's now forbidden by default on Linux and FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.

And then, you can corrupt memory with something like: struct S { int[1_000_000_000] data; int far_data; } S* s = null; s.far_data = 5; If you are unlucky enough to end up in a valid address. That might not be a practical example, of course, but theoretically null pointer could lead to memory corruption. -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- <mazzi> gmail is down? <Luca> waiting for mail.google.com.... <mazzi> ya vendí todas mis acciones de google en los mercados asiaticos <Luca> se viene la ecatombe <mazzi> mal <mazzi> es como que te corten el porno en una tarde al pedo en tu casa
Nov 05 2010
prev sibling parent Leandro Lucarella <luca llucax.com.ar> writes:
Jesse Phillips, el  5 de noviembre a las 18:38 me escribiste:
 Leandro Lucarella Wrote:
 
 And then, you can corrupt memory with something like:
 
 struct S {
 	int[1_000_000_000] data;
 	int far_data;
 }
 
 S* s = null;
 s.far_data = 5;
 
 If you are unlucky enough to end up in a valid address. That might not
 be a practical example, of course, but theoretically null pointer could
 lead to memory corruption.

Interesting, I get this: test.d(13): Error: index 1000000000 overflow for static array I'm not exactly sure what you are trying to demonstrate with it. Are you filling up the stack such that the OS tries to assign to a proper data field? How is it still not trying to access location 0 causing an Access Violation?

s.far_data = 5 is the same as *(s + S.far_data.sizeof) = 5, where S.far_data.sizeof is 4_000_000_000 (i.e. 4 GB, I put a couple of extra 0 than I should :). The point is, you're writing to a big address, which might be mapped to your process, resulting in corrupted memory. I think the error you are receiving is a compiler limitation, not a language feature, but I might be wrong... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Pa' ella cociné, pa' ella lavé, pa' ella soñe Paella completa, $2,50 Pero, la luz mala me tira, y yo? yo soy ligero pa'l trote La luz buena, está en el monte, allá voy, al horizonte
Nov 05 2010