www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - array slice setting not bounds checked!

reply derick_eddington nospam.yashmoo.com writes:
import std.stdio;

void main ()
{
ubyte[] uba = new ubyte[123];

uba[-2 .. length+5] = 7;      // not bounds checked!

//uba[-2 .. length+5][] = 7;  // but this does throw ArrayBoundsError...

ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+5];

foreach (size_t i, ubyte ub; wtf)
writef("%.3d:%d ", i, ub);

writefln();
}
Apr 04 2005
parent reply derick_eddington nospam.yashmoo.com writes:
That should have been:
ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5

but same point...


In article <d2sa8a$518$1 digitaldaemon.com>, derick_eddington nospam.yashmoo.com
says...
import std.stdio;

void main ()
{
ubyte[] uba = new ubyte[123];

uba[-2 .. length+5] = 7;      // not bounds checked!

//uba[-2 .. length+5][] = 7;  // but this does throw ArrayBoundsError...

ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+5];

foreach (size_t i, ubyte ub; wtf)
writef("%.3d:%d ", i, ub);

writefln();
}

Apr 04 2005
next sibling parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

derick_eddington nospam.yashmoo.com schrieb am Mon, 4 Apr 2005 21:16:13 +0000
(UTC):
 That should have been:
 ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5

 but same point...


 In article <d2sa8a$518$1 digitaldaemon.com>,
derick_eddington nospam.yashmoo.com
 says...
import std.stdio;

void main ()
{
ubyte[] uba = new ubyte[123];

uba[-2 .. length+5] = 7;      // not bounds checked!

//uba[-2 .. length+5][] = 7;  // but this does throw ArrayBoundsError...

ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+5];

foreach (size_t i, ubyte ub; wtf)
writef("%.3d:%d ", i, ub);

writefln();
}


http://digitalmars.com/d/arrays.html # A program may not rely on array bounds checking happening. Unless the quote some lines lower is implemented, the quote above should be changed to "can rely on" ? # Insertion of array bounds checking code at runtime should be turned # on and off with a compile time switch. Thomas -----BEGIN PGP SIGNATURE----- iD8DBQFCUbtJ3w+/yD4P9tIRAmSJAKDIq6UX/bEMpWl/Z36SEhZ5TYEw8ACgzleJ vQo0cL7+DmGEBSttpuG7Pzw= =w5eE -----END PGP SIGNATURE-----
Apr 04 2005
parent reply derick_eddington place.com writes:
In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...
http://digitalmars.com/d/arrays.html
# A program may not rely on array bounds checking happening.

Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?

#  Insertion of array bounds checking code at runtime should be turned
#  on and off with a compile time switch.

Thomas

I remember reading that, but if bounds checking is turned on and the double slice bounds checks, I think the other should. I like the idea of having the option of turning on and off bounds checking, but I would think bounds checking should also be a feature that you can rely on to work comprehensively and consistently when it is turned on and be a feature to use in release/production code. Why only go half way?
Apr 04 2005
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
derick_eddington place.com wrote:
 In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...
 
http://digitalmars.com/d/arrays.html
# A program may not rely on array bounds checking happening.

Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?


By "may not rely on", it means that a program that needs ABC turned on in order to work correctly is incorrectly written. As such, it will not work when compiled in release mode.
#  Insertion of array bounds checking code at runtime should be turned
#  on and off with a compile time switch.


That switch is -release. At least in the current DMD design.
 I remember reading that, but if bounds checking is turned on and the double
 slice bounds checks, I think the other should.  I like the idea of having the
 option of turning on and off bounds checking, but I would think bounds checking
 should also be a feature that you can rely on to work comprehensively and
 consistently when it is turned on

Entirely agreed.
 and be a feature to use in release/production code.  Why only go half way?

Disagreed. ABC is left out of release builds for the same reason DBC is - because it is intended for checking that the program is working correctly, and having it in production code would slow the program down. As such, programs are supposed to be written so that array indexes will never be out of bounds. Stewart. -- My e-mail is valid but not my primary mailbox. Please keep replies on the 'group where everyone may benefit.
Apr 05 2005
parent reply derick_eddington yamoooo.com writes:
In article <d2uj5q$2hoh$1 digitaldaemon.com>, Stewart Gordon says...
derick_eddington place.com wrote:
 In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...
 
http://digitalmars.com/d/arrays.html
# A program may not rely on array bounds checking happening.

Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?


By "may not rely on", it means that a program that needs ABC turned on in order to work correctly is incorrectly written. As such, it will not work when compiled in release mode.
#  Insertion of array bounds checking code at runtime should be turned
#  on and off with a compile time switch.


That switch is -release. At least in the current DMD design.
 I remember reading that, but if bounds checking is turned on and the double
 slice bounds checks, I think the other should.  I like the idea of having the
 option of turning on and off bounds checking, but I would think bounds checking
 should also be a feature that you can rely on to work comprehensively and
 consistently when it is turned on

Entirely agreed.
 and be a feature to use in release/production code.  Why only go half way?

Disagreed. ABC is left out of release builds for the same reason DBC is - because it is intended for checking that the program is working correctly, and having it in production code would slow the program down. As such, programs are supposed to be written so that array indexes will never be out of bounds. Stewart.

I think having the switch which disables ABC and DBC be called -release is fine. What I meant was that if you want to have ABC turned on in your production code, you should be able to rely on it; not in the sense of using it to control program flow (like the arrays.html example) but rely on it to catch any and every out-of-bounds (this would mean comphrensive ABC that could be used to control program flow if you wanted, but I agree that's not good design). Saying one should be a good enough programmer that you never allow overflow is a great ideal and one I advocate but the plethora of overflow vulnerabilities says to me it's not being widely achieved. If you'd like to trade performance for assurance, I think you should be able to; especially for programs where DBC and ABC won't have a large impact on performance, and for important programs that are exposed to a lot of untrusted input and use, I think not using -release for your production version and being able to count on DBC and ABC would be a great option and huge selling point for D. If I'm running a financial app exposed to anyone and I can afford more hardware to get whatever performance I need, knowing I can rely on DBC and ABC would make D very attractive to me.
Apr 05 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 6 Apr 2005 00:51:58 +0000 (UTC), <derick_eddington yamoooo.com>  
wrote:
 In article <d2uj5q$2hoh$1 digitaldaemon.com>, Stewart Gordon says...
 derick_eddington place.com wrote:
 In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...

 http://digitalmars.com/d/arrays.html
 # A program may not rely on array bounds checking happening.

 Unless the quote some lines lower is implemented, the quote above  
 should
 be changed to "can rely on" ?


By "may not rely on", it means that a program that needs ABC turned on in order to work correctly is incorrectly written. As such, it will not work when compiled in release mode.
 #  Insertion of array bounds checking code at runtime should be turned
 #  on and off with a compile time switch.


That switch is -release. At least in the current DMD design.
 I remember reading that, but if bounds checking is turned on and the  
 double
 slice bounds checks, I think the other should.  I like the idea of  
 having the
 option of turning on and off bounds checking, but I would think bounds  
 checking
 should also be a feature that you can rely on to work comprehensively  
 and
 consistently when it is turned on

Entirely agreed.
 and be a feature to use in release/production code.  Why only go half  
 way?

Disagreed. ABC is left out of release builds for the same reason DBC is - because it is intended for checking that the program is working correctly, and having it in production code would slow the program down. As such, programs are supposed to be written so that array indexes will never be out of bounds. Stewart.

I think having the switch which disables ABC and DBC be called -release is fine. What I meant was that if you want to have ABC turned on in your production code, you should be able to rely on it; not in the sense of using it to control program flow (like the arrays.html example) but rely on it to catch any and every out-of-bounds (this would mean comphrensive ABC that could be used to control program flow if you wanted, but I agree that's not good design). Saying one should be a good enough programmer that you never allow overflow is a great ideal and one I advocate but the plethora of overflow vulnerabilities says to me it's not being widely achieved. If you'd like to trade performance for assurance, I think you should be able to; especially for programs where DBC and ABC won't have a large impact on performance, and for important programs that are exposed to a lot of untrusted input and use, I think not using -release for your production version and being able to count on DBC and ABC would be a great option and huge selling point for D. If I'm running a financial app exposed to anyone and I can afford more hardware to get whatever performance I need, knowing I can rely on DBC and ABC would make D very attractive to me.

See the other thread of this thread, I beleive you've found a bug (or a strange 'feature'). Making this thread of this thread err.. pointless? (perhaps?) Regan
Apr 05 2005
prev sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
derick_eddington yamoooo.com wrote:
<snip>
 Saying one should be a good enough programmer that you never allow
 overflow is a great ideal and one I advocate but the plethora of
 overflow vulnerabilities says to me it's not being widely achieved.

Exactly. That's the point of ABC - it gives you the chance to fix the bugs causing overflow.
 If you'd like to trade performance for assurance, I think you should 
 be able to; especially for programs where DBC and ABC won't have a 
 large impact on performance, and for important programs that are 
 exposed to a lot of untrusted input and use, I think not using 
 -release for your production version and being able to count on DBC 
 and ABC would be a great option and huge selling point for D.  If I'm 
 running a financial app exposed to anyone and I can afford more 
 hardware to get whatever performance I need, knowing I can rely on 
 DBC and ABC would make D very attractive to me.

If you want to release your programs with the ABC and DBC still in, nobody's going to stop you. But do prepare to be thrown stones at by anyone who reads your code and finds that you rely on it. Stewart. -- My e-mail is valid but not my primary mailbox. Please keep replies on the 'group where everyone may benefit.
Apr 06 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5

This is not a bug. Creating an array from a pointer does not do any bounds checking ever since a pointer does not store any bounds information. If you want to perform bounds checking you need to slice [a .. b] on an array and not a pointer. In fact one should even talk about slicing a pointer since that implies a pointer has a range - instead one should talk about defining an array.
Apr 04 2005
parent reply derick_eddington yahsoo.com writes:
What I consider the bug is the line:

uba[-2 .. length+5] = 7;      // not bounds checked!

which is allowing an overflowed slice-setting of a previously declared ubyte[].
wtf is for creating an array on the overflowed range to print it out and see
that 7 was indeed set before and after uba's correct range.  I understand that
slicing a pointer can't and shouldn't do bounds checking.

In article <d2sm9q$icv$1 digitaldaemon.com>, Ben Hinkle says...
 ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5

This is not a bug. Creating an array from a pointer does not do any bounds checking ever since a pointer does not store any bounds information. If you want to perform bounds checking you need to slice [a .. b] on an array and not a pointer. In fact one should even talk about slicing a pointer since that implies a pointer has a range - instead one should talk about defining an array.

Apr 05 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 6 Apr 2005 00:11:23 +0000 (UTC), <derick_eddington yahsoo.com>  
wrote:
 What I consider the bug is the line:

 uba[-2 .. length+5] = 7;      // not bounds checked!

 which is allowing an overflowed slice-setting of a previously declared  
 ubyte[].

Indeed. I believe you have found a bug (or at least something odd), another example: import std.stdio; void main() { int[] test; int[] part; test.length = 10; test[] = 1; part = test[2..5]; //part[-1] = 0; //array bounds error part[-1 .. 7] = 2; //no array bounds error; foreach(int i; test) writefln(i); } Output: 1 2 2 2 2 2 2 2 2 1 It appears not to check bounds for a slice with upper/lower bounds if the lower bound is negative. Is this intentional? Regan
Apr 05 2005
parent derick_eddington yamoooo.com writes:
Slice-setting with a positive lower bound and out-of-bounds upper bound also
doesn't check, so I don't think it's having a negative lower bound that's the
reason.  I think it's a problem with slice-setting.

Also, if you "double slice-set" the out-of-bounds slice it does throw.
ie:  
int[] ia = new int[4];
ia[2 .. 7][] = 2;  // detects
which I think is because it's evaluating the accessing-slice expression first
which checks.



In article <opsosct1cv23k2f5 nrage.netwin.co.nz>, Regan Heath says...
On Wed, 6 Apr 2005 00:11:23 +0000 (UTC), <derick_eddington yahsoo.com>  
wrote:
 What I consider the bug is the line:

 uba[-2 .. length+5] = 7;      // not bounds checked!

 which is allowing an overflowed slice-setting of a previously declared  
 ubyte[].

Indeed. I believe you have found a bug (or at least something odd), another example: import std.stdio; void main() { int[] test; int[] part; test.length = 10; test[] = 1; part = test[2..5]; //part[-1] = 0; //array bounds error part[-1 .. 7] = 2; //no array bounds error; foreach(int i; test) writefln(i); } Output: 1 2 2 2 2 2 2 2 2 1 It appears not to check bounds for a slice with upper/lower bounds if the lower bound is negative. Is this intentional? Regan

Apr 05 2005