www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - HANDLE and GC

reply pinzo gmail.com writes:
Hello all.
I have just read through the docs and found this statements regarding th=
e  =

GC:
  * Do not store into pointers values that may point into the garbage  =

collected heap.
  * Do not store magic values into pointers, other than null.

But now, looking at std.c.windows.windows I noticed that Win32 HANDLEs  =

(that is HANDLE, HWND, HFILE, etc) are defined:

alias void* HANDLE;

Since HANDLEs are just arbitrary values, and usually don't even point to=
  =

user memory, won't that raise undefined behavior.

If it's true, I think that just changing the definition to something lik=
e:

typedef size_t HANDLE;

would do the trick, since size_t should have the same size and alignment=
  =

requirements that a pointer.

What do you think?
Sep 01 2005
next sibling parent reply Chris Sauls <ibisbasenji gmail.com> writes:
pinzo gmail.com wrote:
 Hello all.
 I have just read through the docs and found this statements regarding 
 the  GC:
  * Do not store into pointers values that may point into the garbage  
 collected heap.
  * Do not store magic values into pointers, other than null.
 
 But now, looking at std.c.windows.windows I noticed that Win32 HANDLEs  
 (that is HANDLE, HWND, HFILE, etc) are defined:
 
 alias void* HANDLE;
 
 Since HANDLEs are just arbitrary values, and usually don't even point 
 to  user memory, won't that raise undefined behavior.

Presumably, going by the spec docs you quoted.
 If it's true, I think that just changing the definition to something like:
 
 typedef size_t HANDLE;
 
 would do the trick, since size_t should have the same size and 
 alignment  requirements that a pointer.
 
 What do you think?

I think you've got the right idea, except I might debate the merits of using 'typedef' over 'alias' in this case, and I'd probably use 'ptrdiff_t' rather than 'size_t'. -- Chris Sauls
Sep 01 2005
parent pinzo gmail.com writes:
On Thu, 01 Sep 2005 21:22:30 +0200, Chris Sauls <ibisbasenji gmail.com> =
 =

wrote:

 I think you've got the right idea, except I might debate the merits of=

 using 'typedef' over 'alias' in this case, and I'd probably use  =

 'ptrdiff_t' rather than 'size_t'.

Well, I said size_t just because is more familiar. Anyway it's guarantee= d = that they are the same size, isn't it? And the size is what matters... About 'typedef' or 'alias', errr well, the whole point of typedef is to = = get type-safeness, so the following get a compiler error: typedef size_t HWND; int n; ShowWindow(n, SW_SHOW) // error cannot convert int to HANDLE. Curiously enough, using a constant compiles fine: ShowWindow(1729, SW_SHOW) // OK (?) That's great for the 0, and not so great for other numbers. But the true advantage of typedef over alias comes when you have 20 = different types of handles, all of them with a lot of functions incompatibles between them (HWND, = HANDLE, HGLOBAL, HINSTANCE, HGDIOBJ, HKEY...) It's too easy to get messed up, and call DestroyObject with a HANDLE, fo= r = example. typedef would mark this as an error.
Sep 01 2005
prev sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES). What 
you don't want to do is store an arbitrary value into a pointer type since 
it might accidentally point to some GC region.

<pinzo gmail.com> wrote in message news:op.swfmpvdn0r3bey pinzopc2...
Hello all.
I have just read through the docs and found this statements regarding the
GC:
  * Do not store into pointers values that may point into the garbage
collected heap.
  * Do not store magic values into pointers, other than null.
Sep 01 2005
parent reply pinzo gmail.com writes:
On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  =

wrote:

 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES)=

 What
 you don't want to do is store an arbitrary value into a pointer type  =

 since
 it might accidentally point to some GC region.

The problem here is that HANDLE values, in general, are not pointers! Th= ey = are arbitrary values cast to (void*) (well, maybe pointers from kernel space or from CSRSS = space, etc.) So they could point just anywhere, including GC-data.
Sep 01 2005
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 01 Sep 2005 23:26:16 +0200, pinzo gmail.com wrote:

 On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  
 wrote:
 
 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES).  
 What
 you don't want to do is store an arbitrary value into a pointer type  
 since
 it might accidentally point to some GC region.

The problem here is that HANDLE values, in general, are not pointers! They are arbitrary values cast to (void*) (well, maybe pointers from kernel space or from CSRSS space, etc.) So they could point just anywhere, including GC-data.

Actually it turns out that the term "handle" can refer to at least two types of data items. One, as you say, is a theoretically arbitrary value but is usually an index into any array, and the other is a pointer to a pointer. In many Windows API and Macintosh contexts, a handle is a pointer to a pointer. The idea being that applications use the handle to deference the target data and that the operating system is free to move the target data about. However, in any Unix contexts, a handle is an array index. In both cases though, a handle is an indirect reference to data. -- Derek Parnell Melbourne, Australia 2/09/2005 8:08:27 AM
Sep 01 2005
parent reply pinzo <pinzo_member pathlink.com> writes:
In article <1ri252j8s1xuw.1in47u3z42esx.dlg 40tude.net>, Derek Parnell says...

Actually it turns out that the term "handle" can refer to at least two
types of data items. One, as you say, is a theoretically arbitrary value
but is usually an index into any array, and the other is a pointer to a
pointer.

I think your definition of "handle" is too restrictive. A "handle" is just an 'opaque' value used to reference an object, in the context of a library. A handle can be 'implemented' as a pointer, a pointer to a pointer, an index into an array, or any other way you can imagine. A different thing is how the handle value is seen by the programmer. Win32 handles are declared as (void*), but the fact is that it isn't documented how exactly they are implemented. Of course, we can guess, I have done some investigation :-) * Some are direct pointers: HMENU, HACCEL, HINSTANCE, HANDLE from FindFirstFile * Some are pointers to pointers: HGLOBAL * Some are indexes into an array: SOCKET? * Some are pointers into the address space of 'another' process: HWND * Some are pointers into the kernel space: HANDLE of kernel objects What I am trying to say is that the last two of them are *not* valid pointers in the current address space, even when they are cast to void*. And so they can theoretically trash the GC. Of course, don't trust the list above if you want to be minimally portable, because this is totally undocumented. So, to be in the safe side, just say no to pointer handles.
Sep 01 2005
next sibling parent "Ben Hinkle" <ben.hinkle gmail.com> writes:
 * Some are direct pointers: HMENU, HACCEL, HINSTANCE, HANDLE from 
 FindFirstFile
 * Some are pointers to pointers: HGLOBAL
 * Some are indexes into an array: SOCKET?
 * Some are pointers into the address space of 'another' process: HWND
 * Some are pointers into the kernel space: HANDLE of kernel objects

 What I am trying to say is that the last two of them are *not* valid 
 pointers in
 the current address space, even when they are cast to void*. And so they 
 can
 theoretically trash the GC.

Only invalid pointers into the GC regions can "trash" the GC (I put quotes around trash because the GC won't get trashed but your fake pointer might). The GC knows what memory regions it owns and if a pointer points to something outside its control it ignores it. So pointing out of the address space is ok - as are values like (HANDLE)-1 that will never point to a valid GC region.
 Of course, don't trust the list above if you want to be minimally 
 portable,
 because this is totally undocumented.
 So, to be in the safe side, just say no to pointer handles.

Sep 02 2005
prev sibling next sibling parent reply Mike Parker <aldacron71 yahoo.com> writes:
pinzo wrote:


 What I am trying to say is that the last two of them are *not* valid pointers
in
 the current address space, even when they are cast to void*. And so they can
 theoretically trash the GC.

Since they are not GC-allocated resources, how are they going to trash the DC? From the GC docs: "Pointers in D can be broadly divided into two categories: those that point to garbage collected memory, and those that do not. Examples of the latter are pointers created by calls to C's malloc(), pointers received from C library routines, pointers to static data, pointers to objects on the stack, etc. For those pointers, anything that is legal in C can be done with them." The two points you began the thread with apply to GCed pointers, which Win32 HANDLES are not.
Sep 02 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <df9e92$e1a$1 digitaldaemon.com>, Mike Parker says...
pinzo wrote:

 What I am trying to say is that the last two of them are *not* valid pointers
in
 the current address space, even when they are cast to void*. And so they can
 theoretically trash the GC.

Since they are not GC-allocated resources, how are they going to trash the DC? From the GC docs: "Pointers in D can be broadly divided into two categories: those that point to garbage collected memory, and those that do not. Examples of the latter are pointers created by calls to C's malloc(), pointers received from C library routines, pointers to static data, pointers to objects on the stack, etc. For those pointers, anything that is legal in C can be done with them." The two points you began the thread with apply to GCed pointers, which Win32 HANDLES are not.

With the current mark/scan implementation, the worst that could happen is memory not being recycled when it should be (because the GC mistakes a handle value for a pointer into GC memory). With a generational (ie. copying) GC, the handle value could be changed if it happens to coincide with the address of a valid GC memory block. Sean
Sep 02 2005
parent reply rodrigorivascosta gmail.com writes:
On Sat, 03 Sep 2005 02:03:43 +0200, Sean Kelly <sean f4.ca> wrote:

 In article <df9e92$e1a$1 digitaldaemon.com>, Mike Parker says...
 With the current mark/scan implementation, the worst that could happen=

 is memory
 not being recycled when it should be (because the GC mistakes a handle=

 value for
 a pointer into GC memory).  With a generational (ie. copying) GC, the =

 handle
 value could be changed if it happens to coincide with the address of a=

 valid GC
 memory block.

Well, to be on the safe side of the life, I strongly suggest to change a= ll = handle types to size_t or similar, the cost of this change is practically zero,= = and the potential problems and incompatibilities that can arise with this or= = future GC implementations are too great.
Sep 04 2005
parent pinzo correo.nu writes:
On Sun, 04 Sep 2005 23:30:42 +0200, <rodrigorivascosta gmail.com> wrote:=


 Well, to be on the safe side of the life, I strongly suggest to change=

 all handle
 types to size_t or similar, the cost of this change is practically zer=

 and
 the potential problems and incompatibilities that can arise with this =

 future
 GC implementations are too great.

I have just messed up my newsreader program. That was myself, of course.= --- Pinzo.
Sep 04 2005
prev sibling parent "Walter Bright" <newshound digitalmars.com> writes:
"pinzo" <pinzo_member pathlink.com> wrote in message
news:df8pve$2nfe$1 digitaldaemon.com...
 What I am trying to say is that the last two of them are *not* valid

 the current address space, even when they are cast to void*.

That's true.
 And so they can
 theoretically trash the GC.

In the case of Windows handles, that's not likely because the only values that can trash the GC are values that happen to fall within range of the GC's allocated memory pool. The non-pointer values I've seen in Windows handles are all less than 64K, which is never, by design, memory allocated to a process, hence cannot point into the GC pool.
 Of course, don't trust the list above if you want to be minimally

 because this is totally undocumented.
 So, to be in the safe side, just say no to pointer handles.

For any new design, that is correct. You can do a legacy special case exception for Windows handles, though.
Sep 08 2005
prev sibling parent reply Shammah Chancellor <Shammah_member pathlink.com> writes:
In article <op.swf0h2q40r3bey pinzopc2>, pinzo gmail.com says...
On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  =

wrote:

 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES)=

 What
 you don't want to do is store an arbitrary value into a pointer type  =

 since
 it might accidentally point to some GC region.

The problem here is that HANDLE values, in general, are not pointers! Th= ey = are arbitrary values cast to (void*) (well, maybe pointers from kernel space or from CSRSS = space, etc.) So they could point just anywhere, including GC-data.

The definition of a "handle" is a pointer to a pointer. -Sha
Sep 01 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Shammah Chancellor" <Shammah_member pathlink.com> wrote in message 
news:df878o$251n$1 digitaldaemon.com...
 In article <op.swf0h2q40r3bey pinzopc2>, pinzo gmail.com says...
On Thu, 01 Sep 2005 22:37:56 +0200, Ben Hinkle <bhinkle mathworks.com>  =

wrote:

 Storing pointers to non-GC data is fine (ie malloc'ed data or HANDLES)=

 What
 you don't want to do is store an arbitrary value into a pointer type  =

 since
 it might accidentally point to some GC region.

The problem here is that HANDLE values, in general, are not pointers! Th= ey = are arbitrary values cast to (void*) (well, maybe pointers from kernel space or from CSRSS = space, etc.) So they could point just anywhere, including GC-data.

The definition of a "handle" is a pointer to a pointer.

In Win32 API HANDLE is void* but there are places where things like -1 are cast to HANDLE. I'm not aware of what non-pointers are cast to HANDLES but I've never looked into it. The "pointer to pointer" is what Macs used ages ago and from what I understand the Mac OS X doesn't use handles anymore. In MATLAB, ironically, a handle is a double :-) That's because back when MATLAB first got graphics objects (called Handle Graphics) the double was the only datatype in MATLAB. It has advantages and disadvantages but I'm mentioning it just for those trivia-minded folks out there who are interested in uses of the word "handle".
 -Sha

 

Sep 01 2005
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 1 Sep 2005 20:56:51 -0400, Ben Hinkle wrote:

[snip]
 
 In Win32 API HANDLE is void* but there are places where things like -1 are 
 cast to HANDLE. I'm not aware of what non-pointers are cast to HANDLES but 
 I've never looked into it.

<trivia> Windows API assumes that no valid pointer will ever point anywhere in the range of addresses 0x00000000 to 0x0000001F, and 0xFFFFFFFD to 0xFFFFFFFF. And thus it allows Handles to contain integer values in the range -3 to 31 inclusive. These are usually reserved values for various things, such as system defined Brushes, Fonts, etc... and error codes. </trivia> -- Derek (skype: derek.j.parnell) Melbourne, Australia 2/09/2005 11:35:49 AM
Sep 01 2005
parent pinzo <pinzo_member pathlink.com> writes:
In article <keoxw1cdrwjs.2hi280n7hudh.dlg 40tude.net>, Derek Parnell says...

Windows API assumes that no valid pointer will ever point anywhere in the
range of addresses 0x00000000 to 0x0000001F, and 0xFFFFFFFD to 0xFFFFFFFF.

<more_trivia> Actually, Win32 assumes that the range 0x00000000 to 0x0000FFFF is never valid memory, and thus is never referenced. This is what enables the MAKEINTRESOURCE hack. </more_trivia>
Sep 01 2005
prev sibling next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 1 Sep 2005 20:56:51 -0400, Ben Hinkle wrote:

 The definition of a "handle" is a pointer to a pointer.


This usage was extensively used in the Amiga system too. In the Amiga, there was only one fixed address in the entire system. The address 0x00000004 contained the address of where the core library was loaded into RAM. From that knowledge, you could locate everything else in the system. They chose that address instead of zero because a common mistake for coders was to write something to RAM using a null pointer and doing that would have wiped out the operating system :D -- Derek (skype: derek.j.parnell) Melbourne, Australia 2/09/2005 11:43:36 AM
Sep 01 2005
parent "Walter Bright" <newshound digitalmars.com> writes:
"Derek Parnell" <derek psych.ward> wrote in message
news:akshp04hduz4$.16z4tpc4g2lzg$.dlg 40tude.net...
 This usage was extensively used in the Amiga system too. In the Amiga,
 there was only one fixed address in the entire system. The address
 0x00000004 contained the address of where the core library was loaded into
 RAM. From that knowledge, you could locate everything else in the system.
 They chose that address instead of zero because a common mistake for

 was to write something to RAM using a null pointer and doing that would
 have wiped out the operating system :D

The biggest mistake in the Intel 8088 design was to put the interrupt table in the bottom 64Kb of memory, guaranteeing that any NULL or offset-from-NULL pointer write would trash the operating system. This means that EVERY time your program crashed, the only thing to do was reboot regardless, otherwise you risked scrambling your hard disk. The system PROM should have been mapped to the bottom 64Kb (instead of the top 64Kb). The biggest productivity booster for the old DOS days was when OS/2 1.1 came out, which ran 16 bit programs in protected mode. Oh heaven, a program could crash and I didn't have to reboot! I never did develop on DOS after that - all the programs were built, debugged, and tested on OS/2 1.1, and only the last step was getting them running under DOS.
Sep 08 2005
prev sibling parent "Walter Bright" <newshound digitalmars.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message
news:df880n$25p2$1 digitaldaemon.com...
 In Win32 API HANDLE is void* but there are places where things like -1 are
 cast to HANDLE. I'm not aware of what non-pointers are cast to HANDLES but
 I've never looked into it. The "pointer to pointer" is what Macs used ages
 ago and from what I understand the Mac OS X doesn't use handles anymore.

 MATLAB, ironically, a handle is a double :-) That's because back when

 first got graphics objects (called Handle Graphics) the double was the

 datatype in MATLAB. It has advantages and disadvantages but I'm mentioning
 it just for those trivia-minded folks out there who are interested in uses
 of the word "handle".

16 bit DMC++ supports a "handle" pointer type, which is the combination of an EMM page number, and EMM page offset. A runtime conversion would be done automagically to map the EMM page into memory and then compute an actual pointer into it. char __handle *p; Ah, those were the daze <g>.
Sep 08 2005