www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Static arrays inside struct and class - bug?

reply "NX" <no-reply mail.com> writes:
I wonder if the followings are compiler bugs:

class stuff_class
{
    byte[1024*1024*16] arr; // Error: index 16777216 overflow for 
static array
}

struct stuff
{
    byte[1024*1024*16] arr; // Error: index 16777216 overflow for 
static array
}

My project has just stopped for this reason, I was trying to hack 
into another process memory with something similar to this:
    stuff data;
    ReadProcessMemory(Proc, (void*)0xA970F4, &data, stuff.sizeof, 
null);
Target program is written in C++ and because of this limitation 
I'm not able to write equivalent code and here I'm stuck.
Aug 01 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 1 August 2015 at 17:22:40 UTC, NX wrote:

 I wonder if the followings are compiler bugs:


No, it is by design, the idea is to keep static arrays smallish 
so null references will be caught by the processor. (An overly 
large static array could allow indexing it through a null pointer 
to potentially reach another object.)

The easiest workaround is to just dynamically allocate such huge 
arrays:

byte[] arr = new byte[](1024*1024*16);
ReadProcessMemory(Proc, 0xdeadbeef, arr.ptr, arr.length, null);

The arr.ptr and arr.length are the key arguments there.
Aug 01 2015
parent reply "NX" <no-reply mail.com> writes:
On Saturday, 1 August 2015 at 17:29:54 UTC, Adam D. Ruppe wrote:

 On Saturday, 1 August 2015 at 17:22:40 UTC, NX wrote:

 I wonder if the followings are compiler bugs:


 No, it is by design, the idea is to keep static arrays smallish 
 so null references will be caught by the processor. (An overly 
 large static array could allow indexing it through a null 
 pointer to potentially reach another object.)

 The easiest workaround is to just dynamically allocate such 
 huge arrays:

 byte[] arr = new byte[](1024*1024*16);
 ReadProcessMemory(Proc, 0xdeadbeef, arr.ptr, arr.length, null);

 The arr.ptr and arr.length are the key arguments there.


Sorry, I can't see _the_ point in that. I understand that could 
be a problem if it was a "global" array but this scenery is 
completely wrong in my view. I'm already going to dynamically 
allocate it and my problem is actually a lot complex than what I 
showed there, I not even allowed to do this:

struct stuff
{
    byte[1024*1024*16] arr; // Error: index 16777216 overflow for 
static array
}
//...
stuff* data = new stuff;
ReadProcessMemory(Proc, (void*)0xA970F4, data, stuff.sizeof, 
null);

Here 
(https://gist.github.com/NightmareX1337/6408287d7823c8a4ba20) is 
the real issue if anyone want to see the real-world problem with 
long lines of code
Aug 01 2015
next sibling parent "NX" <no-reply mail.com> writes:
Typo:
*scenario
Aug 01 2015
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 1 August 2015 at 18:07:51 UTC, NX wrote:

 Sorry, I can't see _the_ point in that.


Yeah, especially since you can jsut break up the array and get 
the same effect anyway...

so like if you don't want to dynamically allocate the memory, you 
could also try:

  byte[1024*1024*8] arr1;
  byte[1024*1024*8] arr2;

If they are right next to each other they will still be 
continuous in memory and then you work around the limit.
Aug 01 2015
prev sibling next sibling parent reply Daniel Kozak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
V Sat, 01 Aug 2015 18:07:50 +0000
NX via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> napsáno:


 On Saturday, 1 August 2015 at 17:29:54 UTC, Adam D. Ruppe wrote:

 On Saturday, 1 August 2015 at 17:22:40 UTC, NX wrote:

 I wonder if the followings are compiler bugs:


 No, it is by design, the idea is to keep static arrays smallish 
 so null references will be caught by the processor. (An overly 
 large static array could allow indexing it through a null 
 pointer to potentially reach another object.)

 The easiest workaround is to just dynamically allocate such 
 huge arrays:

 byte[] arr = new byte[](1024*1024*16);
 ReadProcessMemory(Proc, 0xdeadbeef, arr.ptr, arr.length, null);

 The arr.ptr and arr.length are the key arguments there.


 
 Sorry, I can't see _the_ point in that. I understand that could 
 be a problem if it was a "global" array but this scenery is 
 completely wrong in my view. I'm already going to dynamically 
 allocate it and my problem is actually a lot complex than what I 
 showed there, I not even allowed to do this:
 
 struct stuff
 {
     byte[1024*1024*16] arr; // Error: index 16777216 overflow for 
 static array
 }
 //...
 stuff* data = new stuff;
 ReadProcessMemory(Proc, (void*)0xA970F4, data, stuff.sizeof, 
 null);
 
 Here 
 (https://gist.github.com/NightmareX1337/6408287d7823c8a4ba20) is 
 the real issue if anyone want to see the real-world problem with 
 long lines of code


Still same problem, You can`t allocate more then 16M on stack. Use
dynamic allocation
Aug 01 2015
parent reply "NX" <no-reply mail.com> writes:
On Saturday, 1 August 2015 at 18:47:00 UTC, Daniel Kozak wrote:

 Still same problem, You can`t allocate more then 16M on stack. 
 Use dynamic allocation


I don't think "new MyStruct" allocates on stack, actually 
allocating ~16MB on stack will immediatelly crash the program 
which is not the case with NewExpression.
Aug 01 2015
parent reply Daniel Kozak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
V Sat, 01 Aug 2015 19:16:16 +0000
NX via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> napsáno:


 On Saturday, 1 August 2015 at 18:47:00 UTC, Daniel Kozak wrote:

 Still same problem, You can`t allocate more then 16M on stack. 
 Use dynamic allocation


 
 I don't think "new MyStruct" allocates on stack, actually 
 allocating ~16MB on stack will immediatelly crash the program 
 which is not the case with NewExpression.


My fault It is not on stack, but still it is a static allocation
Aug 01 2015
parent reply "NX" <no-reply mail.com> writes:
On Saturday, 1 August 2015 at 19:33:26 UTC, Daniel Kozak wrote:

 My fault It is not on stack, but still it is a static allocation


I think you're misusing "static allocation" and "static 
declaration" for each other.
Aug 01 2015
parent Daniel Kozak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
V Sat, 01 Aug 2015 20:20:14 +0000
NX via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> napsáno:


 On Saturday, 1 August 2015 at 19:33:26 UTC, Daniel Kozak wrote:

 My fault It is not on stack, but still it is a static allocation


 
 I think you're misusing "static allocation" and "static 
 declaration" for each other.


Yep ;-)
Aug 01 2015
prev sibling parent reply "Daniel Kozak" <kozzi11 gmail.com> writes:
On Saturday, 1 August 2015 at 18:07:51 UTC, NX wrote:

 On Saturday, 1 August 2015 at 17:29:54 UTC, Adam D. Ruppe wrote:

 Sorry, I can't see _the_ point in that. I understand that could 
 be a problem if it was a "global" array but this scenery is 
 completely wrong in my view. I'm already going to dynamically 
 allocate it


No you don't. You still use static allocation for array
Aug 01 2015
parent reply "NX" <no-reply mail.com> writes:
On Saturday, 1 August 2015 at 18:50:09 UTC, Daniel Kozak wrote:

 No you don't. You still use static allocation for array


Can clarify why does that happen and I still suspect it's a 
static allocation it would increase output exe if it was really 
that static..?
Aug 01 2015
parent Daniel Kozak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
V Sat, 01 Aug 2015 19:21:36 +0000
NX via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> napsáno:


 On Saturday, 1 August 2015 at 18:50:09 UTC, Daniel Kozak wrote:

 No you don't. You still use static allocation for array


 
 Can clarify why does that happen and I still suspect it's a 
 static allocation it would increase output exe if it was really 
 that static..?


No it would not increase output exe. Problem is with definition:
type[size] val; // static declaration so compilere check max 16M.

But you are right, in your case it could be improved and such
declaration could work.


because:

S {
  byte[16*1024*1024*1024] arr;
}

void main() {
    auto s = new S();
    if (s is null) {
      // error cannont allocate enought memory
    }
   
}

but:

void main() {
  byte[16*1024*1024*1024] arr; // impossible to check if is allocated
}

Maybe you can open an enhancment on issues.dlang.org
Aug 01 2015
prev sibling parent "BBasile" <bb.temp gmx.com> writes:
On Saturday, 1 August 2015 at 17:22:40 UTC, NX wrote:

 I wonder if the followings are compiler bugs:

 class stuff_class
 {
    byte[1024*1024*16] arr; // Error: index 16777216 overflow 
 for static array
 }

 struct stuff
 {
    byte[1024*1024*16] arr; // Error: index 16777216 overflow 
 for static array
 }

 My project has just stopped for this reason, I was trying to 
 hack into another process memory with something similar to this:
    stuff data;
    ReadProcessMemory(Proc, (void*)0xA970F4, &data, 
 stuff.sizeof, null);
 Target program is written in C++ and because of this limitation 
 I'm not able to write equivalent code and here I'm stuck.


There a limit for static array size. This limits is exactly 16MB 
so 1024*1024*16.
Remove one element:

byte[1024*1024*16-1] arr;
Aug 01 2015