digitalmars.D - Can dynamic arrays move in memory?
- Brad Beveridge (27/27) Apr 27 2005 I already know the answer - yes. The question should really be
- Jarrett Billingsley (9/11) Apr 27 2005 The current GC doesn't, but you're absolutely right - it should be set i...
- Norbert Nemec (9/18) Apr 28 2005 Allowing memory to be moved by the GC would be a perfect source of
- Ben Hinkle (5/17) Apr 27 2005 [snip]
- Brad Beveridge (5/9) Apr 27 2005 Ah. At least I don't feel too bad about missing this message - it was a...
- Benji Smith (14/20) Apr 28 2005 Hmmmmmmmm.
- Russ Lewis (6/38) Apr 28 2005 There is another thing to consider, which is similar, but not quite the
- bobef (7/39) May 01 2005 Locking/unlocking is much needed for every type of variable, I believe,
I already know the answer - yes. The question should really be "Can dynamic arrays move in memory if you don't resize them?" If they can't - fine, but it should be part of the specification & not up to the compiler implementation to decide. If they can, then I think D has a big problem - which I will illustrate. I am playing with OpenGL, and I have a bug - it may not be because arrays move, but if they could move then you would certainly run into problems. Assume that the GC can move array references at will: 1) Create a dynamic array called points. 2) Pass points into a C library - in this case OpenGL 3) Create a dynamic array of normals 4) Pass normals into C lib (openGL) 5) Call a library function that does something with these arrays. Depending on how the GC behaves - and if arrays can move - then the call to 5) is pretty undefined. The C library is pointing at stale memory locations. I get the impression that future (compacting) GCs may be able to move arrays at will, even if you don't resize the array that moves. If this is the case I would like to propose "lock" and "unlock" properties for arrays. When an array is locked you cannot resize it (runtime error), but you can be sure that the GC will not move it for you - ie, it is just a plain chunk of memory that C can mess about with. After the array is unlocked, then any C code can no longer trust their pointers. Thoughts? Brad
Apr 27 2005
"Brad Beveridge" <brad somewhere.net> wrote in message news:d4pgjl$2gho$1 digitaldaemon.com...If they can't - fine, but it should be part of the specification & not up to the compiler implementation to decide.The current GC doesn't, but you're absolutely right - it should be set in stone whether the GC is allowed to move memory or not. Hopefully the decision is _not_ to allow it. Most C/C++ libs require that the memory does not move. Things like associative arrays with class refs as keys would not work properly (unless the class refs were doubly indirect - which would just be stupid). And unless you're really really REALLY pressed for memory, not copying is usually just fine.
Apr 27 2005
Norbert Nemec <Norbert Nemec-online.de> Jarrett Billingsley schrieb:"Brad Beveridge" <brad somewhere.net> wrote in message news:d4pgjl$2gho$1 digitaldaemon.com...Allowing memory to be moved by the GC would be a perfect source of subtle bugs, far worse than uninitialized variables in C: Imagine some code that erraneously depends on memory staying in some fixed position. The code will work for years until somebody actually implements a GC that exploits that possibility. Now the code will suddenly begin to behave absolutely unpredictable and crash in non-reproducible ways! And this possibly so rarely that people will not even believe there is a bug...If they can't - fine, but it should be part of the specification & not up to the compiler implementation to decide.The current GC doesn't, but you're absolutely right - it should be set in stone whether the GC is allowed to move memory or not.
Apr 28 2005
"Brad Beveridge" <brad somewhere.net> wrote in message news:d4pgjl$2gho$1 digitaldaemon.com...I already know the answer - yes. The question should really be "Can dynamic arrays move in memory if you don't resize them?"[snip]I get the impression that future (compacting) GCs may be able to move arrays at will, even if you don't resize the array that moves. If this is the case I would like to propose "lock" and "unlock" properties for arrays. When an array is locked you cannot resize it (runtime error), but you can be sure that the GC will not move it for you - ie, it is just a plain chunk of memory that C can mess about with. After the array is unlocked, then any C code can no longer trust their pointers. Thoughts? BradOn the wish-list. see http://www.digitalmars.com/d/archives/digitalmars/D/9385.html
Apr 27 2005
Ben Hinkle wrote:On the wish-list. see http://www.digitalmars.com/d/archives/digitalmars/D/9385.htmlAh. At least I don't feel too bad about missing this message - it was a while ago :) Thanks Ben Brad
Apr 27 2005
Brad Beveridge wrote:I already know the answer - yes. The question should really be "Can dynamic arrays move in memory if you don't resize them?" ... If they can, then I think D has a big problem...Hmmmmmmmm. But if the GC can't move arrays, then won't it eventually degrade as memory becomes fragmented? It seems like, if the GC were allowed to move arrays, it would be able to keep a reasonably sized pool of contiguous memory available for allocation. The most recent incarnation of the Java GC has improved its performance quite a bit by creating memory "ergonomics", one of which is determining when (and if) the compiler is allowed to move objects from its "young" object pool to its "mature" object pool. Perhaps the lock & unlock idea proposed by Brad would work well to solve any problems when interacting with C libraries. But it seems like a job for the compiler/runtime. Not for the programmer to worry about. --BenjiSmith
Apr 28 2005
There is another thing to consider, which is similar, but not quite the same: if you don't keep a D reference to one of those arrays, then D might garbage collect the array away before GL uses it. So, you either have to keep a D reference to those arrays hanging around until GL is done, or you have to disable the GC in that window of time. Brad Beveridge wrote:I already know the answer - yes. The question should really be "Can dynamic arrays move in memory if you don't resize them?" If they can't - fine, but it should be part of the specification & not up to the compiler implementation to decide. If they can, then I think D has a big problem - which I will illustrate. I am playing with OpenGL, and I have a bug - it may not be because arrays move, but if they could move then you would certainly run into problems. Assume that the GC can move array references at will: 1) Create a dynamic array called points. 2) Pass points into a C library - in this case OpenGL 3) Create a dynamic array of normals 4) Pass normals into C lib (openGL) 5) Call a library function that does something with these arrays. Depending on how the GC behaves - and if arrays can move - then the call to 5) is pretty undefined. The C library is pointing at stale memory locations. I get the impression that future (compacting) GCs may be able to move arrays at will, even if you don't resize the array that moves. If this is the case I would like to propose "lock" and "unlock" properties for arrays. When an array is locked you cannot resize it (runtime error), but you can be sure that the GC will not move it for you - ie, it is just a plain chunk of memory that C can mess about with. After the array is unlocked, then any C code can no longer trust their pointers. Thoughts? Brad
Apr 28 2005
Locking/unlocking is much needed for every type of variable, I believe, not only for arrays. And this is because all the world uses C/C++ which doesn't have a GC... And since D depends on C/C++ so much and it always will it needs such properties... Just for example - Win32. Win32 will always lay on C and D will always have to support Win32 (or at least for next 15 years)... Brad Beveridge wrote:I already know the answer - yes. The question should really be "Can dynamic arrays move in memory if you don't resize them?" If they can't - fine, but it should be part of the specification & not up to the compiler implementation to decide. If they can, then I think D has a big problem - which I will illustrate. I am playing with OpenGL, and I have a bug - it may not be because arrays move, but if they could move then you would certainly run into problems. Assume that the GC can move array references at will: 1) Create a dynamic array called points. 2) Pass points into a C library - in this case OpenGL 3) Create a dynamic array of normals 4) Pass normals into C lib (openGL) 5) Call a library function that does something with these arrays. Depending on how the GC behaves - and if arrays can move - then the call to 5) is pretty undefined. The C library is pointing at stale memory locations. I get the impression that future (compacting) GCs may be able to move arrays at will, even if you don't resize the array that moves. If this is the case I would like to propose "lock" and "unlock" properties for arrays. When an array is locked you cannot resize it (runtime error), but you can be sure that the GC will not move it for you - ie, it is just a plain chunk of memory that C can mess about with. After the array is unlocked, then any C code can no longer trust their pointers. Thoughts? Brad
May 01 2005