www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Question about garbage collection specification

reply "rsw0x" <anonymous anonymous.com> writes:
http://dlang.org/garbage.html

Do not take advantage of alignment of pointers to store bit flags 
in the low order bits:
p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

if this restriction is actually imposed - why does 
std.bitmanip.tagged{ClassRef,Pointer} even exist?
Jun 13 2015
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Sat, 13 Jun 2015 11:32:18 +0000, rsw0x wrote:


 http://dlang.org/garbage.html
=20
 Do not take advantage of alignment of pointers to store bit flags in the
 low order bits:
 p =3D cast(void*)(cast(int)p | 1);  // error: undefined behavior
=20
 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


'cause D is not another fashionable scripting language where you have to=20
eat what you got. one can avoid using GC in his code, or intermix GC with=20
other allocators, or rewrite the whole GC or the whole runtime. writing=20
stupid boilerplate code for tagged pointers and nan boxing is simply=20
tedious, it belongs to library.

in other words: you can use `DirEntries` and `remove` to kill all files=20
in your home directory, but it's not the only use for that functions, so=20
they should not be removed from Phobos as "useless" or "dangerous".=
Jun 13 2015
next sibling parent reply "rsw0x" <anonymous anonymous.com> writes:
On Saturday, 13 June 2015 at 12:26:49 UTC, ketmar wrote:

 On Sat, 13 Jun 2015 11:32:18 +0000, rsw0x wrote:


 http://dlang.org/garbage.html
 
 Do not take advantage of alignment of pointers to store bit 
 flags in the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior
 
 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 'cause D is not another fashionable scripting language where 
 you have to
 eat what you got. one can avoid using GC in his code, or 
 intermix GC with
 other allocators, or rewrite the whole GC or the whole runtime. 
 writing
 stupid boilerplate code for tagged pointers and nan boxing is 
 simply
 tedious, it belongs to library.

 in other words: you can use `DirEntries` and `remove` to kill 
 all files
 in your home directory, but it's not the only use for that 
 functions, so
 they should not be removed from Phobos as "useless" or 
 "dangerous".


There's not even a warning on them that they're violating the 
garbage collector specification, don't you think that's a little 
important?
Jun 13 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 13 June 2015 at 12:28:15 UTC, rsw0x wrote:

 There's not even a warning on them that they're violating the 
 garbage collector specification, don't you think that's a 
 little important?


https://github.com/D-Programming-Language/phobos/pull/3411
Jun 13 2015
prev sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Sat, 13 Jun 2015 12:28:13 +0000, rsw0x wrote:


 There's not even a warning on them that they're violating the garbage
 collector specification, don't you think that's a little important?


i believe that if one needs to do such things, he is knowledgeable enough=20
to foresee the possible consequences. this is not what casual programmer=20
does, so one is expected to know some internals.

yet warning may be helpful too, Marc submitted a PR for that. ;-)=
Jun 13 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 13 June 2015 at 12:26:49 UTC, ketmar wrote:

 'cause D is not another fashionable scripting language where 
 you have to
 eat what you got. one can avoid using GC in his code, or 
 intermix GC with
 other allocators, or rewrite the whole GC or the whole runtime. 
 writing
 stupid boilerplate code for tagged pointers and nan boxing is 
 simply
 tedious, it belongs to library.


I haven't read something that ridiculous in a while. Even if 
you'd be right, OP has a point in the fact that this should not 
be present in the standard lib without some big red sign.

Hopefully, This will work just fine with the GC. The GC spec is 
needlessly restrictive.
Jun 13 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Sat, 13 Jun 2015 22:09:52 +0000, deadalnix wrote:


 I haven't read something that ridiculous in a while.


you're welcome.=
Jun 13 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does 
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


That seems like an arbitrary limitation. This will create an 
interior pointer, which the GC needs to recognize anyway.

The doc need to be updated.
Jun 13 2015
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/13/15 6:07 PM, deadalnix wrote:

 On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit flags in
 the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 That seems like an arbitrary limitation. This will create an interior
 pointer, which the GC needs to recognize anyway.

 The doc need to be updated.


void *x = new int;
ptr += 1;

Is this illegal? If not, then why is the above illegal? If so, thousands 
of lines of code are in violation -- any kind of serialization library 
for example.

It's exactly the same thing. We can probably change taggedPointer to do 
this instead, if it's such a big deal. We can change it to a void 
pointer, and only cast it to T* when retrieving the real pointer.

But it seems kind of silly to outlaw the obvious implementation. I think 
we should revert the warning and fix the docs.

-Steve
Jun 13 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 13 June 2015 at 22:07:26 UTC, deadalnix wrote:

 On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does 
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 That seems like an arbitrary limitation. This will create an 
 interior pointer, which the GC needs to recognize anyway.

 The doc need to be updated.


I see David Nadlinger already said so in the PR, but that's how I 
understand it:

AFAIU the purpose of this restriction is to allow the GC to take 
advantage of alignment: a pointer with the lowest bit set cannot 
point to a 4-byte aligned struct, for example. The GC doc [1] 
mentions alignment several times, among them:
"Do not misalign pointers if those pointers may point into the GC 
heap"

As for arbitrary pointer arithmetic being allowed, I guess that's 
because the language doesn't distinguish between GC and non-GC 
pointers. And note that it is un- safe anyway.

[1] http://dlang.org/garbage.html
Jun 14 2015
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Sunday, 14 June 2015 at 11:03:00 UTC, Marc Schütz wrote:

 On Saturday, 13 June 2015 at 22:07:26 UTC, deadalnix wrote:

 On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does 
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 That seems like an arbitrary limitation. This will create an 
 interior pointer, which the GC needs to recognize anyway.

 The doc need to be updated.


 I see David Nadlinger already said so in the PR, but that's how 
 I understand it:

 AFAIU the purpose of this restriction is to allow the GC to 
 take advantage of alignment: a pointer with the lowest bit set 
 cannot point to a 4-byte aligned struct, for example. The GC 
 doc [1] mentions alignment several times, among them:
 "Do not misalign pointers if those pointers may point into the 
 GC heap"

 As for arbitrary pointer arithmetic being allowed, I guess 
 that's because the language doesn't distinguish between GC and 
 non-GC pointers. And note that it is un- safe anyway.

 [1] http://dlang.org/garbage.html


Wouldn't it make sense to do so?

Active Oberon, Modula-3 and .NET make such difference in their 
pointer types.

I would imagine it could help in terms of what is possible for GC 
improvements.

--
Paulo
Jun 14 2015
next sibling parent reply "rsw0x" <anonymous anonymous.com> writes:
On Sunday, 14 June 2015 at 11:36:59 UTC, Paulo Pinto wrote:

 On Sunday, 14 June 2015 at 11:03:00 UTC, Marc Schütz wrote:

 On Saturday, 13 June 2015 at 22:07:26 UTC, deadalnix wrote:

 On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined 
 behavior

 if this restriction is actually imposed - why does 
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 That seems like an arbitrary limitation. This will create an 
 interior pointer, which the GC needs to recognize anyway.

 The doc need to be updated.


 I see David Nadlinger already said so in the PR, but that's 
 how I understand it:

 AFAIU the purpose of this restriction is to allow the GC to 
 take advantage of alignment: a pointer with the lowest bit set 
 cannot point to a 4-byte aligned struct, for example. The GC 
 doc [1] mentions alignment several times, among them:
 "Do not misalign pointers if those pointers may point into the 
 GC heap"

 As for arbitrary pointer arithmetic being allowed, I guess 
 that's because the language doesn't distinguish between GC and 
 non-GC pointers. And note that it is un- safe anyway.

 [1] http://dlang.org/garbage.html


 Wouldn't it make sense to do so?

 Active Oberon, Modula-3 and .NET make such difference in their 
 pointer types.

 I would imagine it could help in terms of what is possible for 
 GC improvements.

 --
 Paulo


But a pointer can already point to a one byte aligned part of an 
object and keep it alive because internal pointers must be 
supported. The only alignment enforced is that pointers must be 
aligned. This is an arbitrary restriction.
Jun 14 2015
parent "rsw0x" <anonymous anonymous.com> writes:
On Sunday, 14 June 2015 at 11:57:35 UTC, rsw0x wrote:

 On Sunday, 14 June 2015 at 11:36:59 UTC, Paulo Pinto wrote:

 On Sunday, 14 June 2015 at 11:03:00 UTC, Marc Schütz wrote:

 On Saturday, 13 June 2015 at 22:07:26 UTC, deadalnix wrote:

 On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined 
 behavior

 if this restriction is actually imposed - why does 
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 That seems like an arbitrary limitation. This will create an 
 interior pointer, which the GC needs to recognize anyway.

 The doc need to be updated.


 I see David Nadlinger already said so in the PR, but that's 
 how I understand it:

 AFAIU the purpose of this restriction is to allow the GC to 
 take advantage of alignment: a pointer with the lowest bit 
 set cannot point to a 4-byte aligned struct, for example. The 
 GC doc [1] mentions alignment several times, among them:
 "Do not misalign pointers if those pointers may point into 
 the GC heap"

 As for arbitrary pointer arithmetic being allowed, I guess 
 that's because the language doesn't distinguish between GC 
 and non-GC pointers. And note that it is un- safe anyway.

 [1] http://dlang.org/garbage.html


 Wouldn't it make sense to do so?

 Active Oberon, Modula-3 and .NET make such difference in their 
 pointer types.

 I would imagine it could help in terms of what is possible for 
 GC improvements.

 --
 Paulo


 But a pointer can already point to a one byte aligned part of 
 an object and keep it alive because internal pointers must be 
 supported. The only alignment enforced is that pointers must be 
 aligned. This is an arbitrary restriction.


Woops, replied to the wrong person. My bad.


And yes, you're right that the GC could improve if we went 
towards "d pointers" like Go did.
Jun 14 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 14 June 2015 at 11:36:59 UTC, Paulo Pinto wrote:

 On Sunday, 14 June 2015 at 11:03:00 UTC, Marc Schütz wrote:

 As for arbitrary pointer arithmetic being allowed, I guess 
 that's because the language doesn't distinguish between GC and 
 non-GC pointers. And note that it is un- safe anyway.

 [1] http://dlang.org/garbage.html


 Wouldn't it make sense to do so?

 Active Oberon, Modula-3 and .NET make such difference in their 
 pointer types.

 I would imagine it could help in terms of what is possible for 
 GC improvements.


Definitely, but I don't see how it could reasonably be added to 
the language now.
Jun 14 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/14/15 8:44 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Sunday, 14 June 2015 at 11:36:59 UTC, Paulo Pinto wrote:

 On Sunday, 14 June 2015 at 11:03:00 UTC, Marc Schütz wrote:

 As for arbitrary pointer arithmetic being allowed, I guess that's
 because the language doesn't distinguish between GC and non-GC
 pointers. And note that it is un- safe anyway.

 [1] http://dlang.org/garbage.html


 Wouldn't it make sense to do so?

 Active Oberon, Modula-3 and .NET make such difference in their pointer
 types.

 I would imagine it could help in terms of what is possible for GC
 improvements.


 Definitely, but I don't see how it could reasonably be added to the
 language now.


We can create new types of pointers, remove void * from the language, 
and get rid of unions, and yes, *ONLY THEN* we can have a GC that takes 
advantage of this.

These are painful, massively breaking changes. It's not going to happen.

-Steve
Jun 15 2015
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Mon, 15 Jun 2015 08:01:58 -0400, Steven Schveighoffer wrote:


 We can create new types of pointers, remove void * from the language,
 and get rid of unions, and yes, *ONLY THEN* we can have a GC that takes
 advantage of this.
=20
 These are painful, massively breaking changes. It's not going to happen.


i'm dreaming of making unions that contains GC objects invalid. it simply=20
begs for troubles!=
Jun 15 2015
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/14/15 7:02 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Saturday, 13 June 2015 at 22:07:26 UTC, deadalnix wrote:

 On Saturday, 13 June 2015 at 11:32:20 UTC, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit flags in
 the low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 That seems like an arbitrary limitation. This will create an interior
 pointer, which the GC needs to recognize anyway.

 The doc need to be updated.


 I see David Nadlinger already said so in the PR, but that's how I
 understand it:

 AFAIU the purpose of this restriction is to allow the GC to take
 advantage of alignment: a pointer with the lowest bit set cannot point
 to a 4-byte aligned struct, for example.


Huh? Of course it can!

struct S
{
   ubyte[4] arr;
   int x;
}

auto s = new S;
auto p = &s.arr[1]; // points at an S in memory.

I don't see how GC can take any advantage of this.

-Steve
Jun 15 2015
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 13-Jun-2015 14:32, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit flags in the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


AFAIK the restriction was that pointers _themselves_ have to be stored 
at word-aligned addresses. This allows GC to scan memory cheaper w/o 
considering if some misaligned address may contain a pointer.

-- 
Dmitry Olshansky
Jun 15 2015
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/15/15 8:08 AM, Dmitry Olshansky wrote:

 On 13-Jun-2015 14:32, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit flags in the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 AFAIK the restriction was that pointers _themselves_ have to be stored
 at word-aligned addresses. This allows GC to scan memory cheaper w/o
 considering if some misaligned address may contain a pointer.


That doesn't make sense. Why would you want to do this?

The only rational thing I can think of is that you wouldn't want to 
store the result in an *actual* int pointer (lest it be used thinking it 
was valid without masking out the lower bits). But the example is 
storing it in a void *...

-Steve
Jun 15 2015
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 15-Jun-2015 15:49, Steven Schveighoffer wrote:

 On 6/15/15 8:08 AM, Dmitry Olshansky wrote:

 On 13-Jun-2015 14:32, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit flags in the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 AFAIK the restriction was that pointers _themselves_ have to be stored
 at word-aligned addresses. This allows GC to scan memory cheaper w/o
 considering if some misaligned address may contain a pointer.


 That doesn't make sense. Why would you want to do this?


What exactly? Storing pointer at misaligned memory location - no good 
reason whatsoever. As in how it may happen - explicitly tightly packed 
structs with align(1) might end up doing this.

 From http://dlang.org/phobos/core_memory.html which seems to be more 
recent:

Implementations are free to assume that GC pointers are only stored on 
word boundaries. Unaligned pointers may be ignored entirely.


 The only rational thing I can think of is that you wouldn't want to
 store the result in an *actual* int pointer (lest it be used thinking it
 was valid without masking out the lower bits). But the example is
 storing it in a void *...


The example doesn't show what memory location that p refers to thus it's 
not possible to say if it's valid.


-- 
Dmitry Olshansky
Jun 15 2015
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/15/15 11:18 AM, Dmitry Olshansky wrote:

 On 15-Jun-2015 15:49, Steven Schveighoffer wrote:

 On 6/15/15 8:08 AM, Dmitry Olshansky wrote:

 On 13-Jun-2015 14:32, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit flags in
 the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 AFAIK the restriction was that pointers _themselves_ have to be stored
 at word-aligned addresses. This allows GC to scan memory cheaper w/o
 considering if some misaligned address may contain a pointer.


 That doesn't make sense. Why would you want to do this?


 What exactly? Storing pointer at misaligned memory location - no good
 reason whatsoever. As in how it may happen - explicitly tightly packed
 structs with align(1) might end up doing this.

  From http://dlang.org/phobos/core_memory.html which seems to be more
 recent:

 Implementations are free to assume that GC pointers are only stored on
 word boundaries. Unaligned pointers may be ignored entirely.


Right, and that's already assumed in (and would be ignored by) the 
current GC. What's NOT assumed (and I have no idea how we could make 
this assumption) is that interior pointers that point at parts of a 
larger struct/type/etc aren't valid GC pointers, and so the GC can 
ignore those too.

For example:

void *p = &someGCInt;
p++; // equivalent to p = cast(void *)(cast(size_t)p | 1);

The documentation above implies that if p is the only pointer to 
someGCInt, the GC may collect it. I think this is not possible for us to 
claim, even if we wanted to, given D's type system.


 The only rational thing I can think of is that you wouldn't want to
 store the result in an *actual* int pointer (lest it be used thinking it
 was valid without masking out the lower bits). But the example is
 storing it in a void *...


 The example doesn't show what memory location that p refers to thus it's
 not possible to say if it's valid.


The whole premise (as stated in the doc right before the example) is 
that you know the low order bits are 0 :) this is the complete basis of 
std.bitmanip.taggedPtr. It's quite genius I think, and the fact that it 
works with GC (and IMO, any possible GC you could create for D) without 
issue is awesome. We should not be attaching a scarlet comment to the 
docs to warn people away from it.

-Steve
Jun 15 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 15 June 2015 at 12:49:25 UTC, Steven Schveighoffer 
wrote:

 On 6/15/15 8:08 AM, Dmitry Olshansky wrote:

 On 13-Jun-2015 14:32, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 AFAIK the restriction was that pointers _themselves_ have to 
 be stored
 at word-aligned addresses. This allows GC to scan memory 
 cheaper w/o
 considering if some misaligned address may contain a pointer.


 That doesn't make sense. Why would you want to do this?

 The only rational thing I can think of is that you wouldn't 
 want to store the result in an *actual* int pointer (lest it be 
 used thinking it was valid without masking out the lower bits). 
 But the example is storing it in a void *...

 -Steve


I think you miread it.

The memory the pointer points to can be misaligned.
The memory the pointer is in must be aligned.

Note that the second is mandatory is various hardware 
architecture to begin with, so it is not like it is a big 
constraint. It allow the GC to scan faster. It is all good.

The first one do not make any sense as GC need to support 
interior pointers anyway.
Jun 15 2015
prev sibling parent reply "rsw0x" <anonymous anonymous.com> writes:
On Monday, 15 June 2015 at 12:08:31 UTC, Dmitry Olshansky wrote:

 On 13-Jun-2015 14:32, rsw0x wrote:

 http://dlang.org/garbage.html

 Do not take advantage of alignment of pointers to store bit 
 flags in the
 low order bits:
 p = cast(void*)(cast(int)p | 1);  // error: undefined behavior

 if this restriction is actually imposed - why does
 std.bitmanip.tagged{ClassRef,Pointer} even exist?


 AFAIK the restriction was that pointers _themselves_ have to be 
 stored at word-aligned addresses. This allows GC to scan memory 
 cheaper w/o considering if some misaligned address may contain 
 a pointer.


this doesn't make any sense, it's referring to an object p (of 
void*), not the location of p. It's setting the lsb of p to 1 and 
claiming it's undefined behavior, when it's clearly not.

Unless I misunderstand it.
Jun 15 2015
parent reply "Alex Parrill" <initrd.gz gmail.com> writes:
On Monday, 15 June 2015 at 15:33:41 UTC, rsw0x wrote:

 this doesn't make any sense, it's referring to an object p (of 
 void*), not the location of p. It's setting the lsb of p to 1 
 and claiming it's undefined behavior, when it's clearly not.

 Unless I misunderstand it.


`p` is assumed to be a pointer. `cast(int) p` gets the integer 
representation of the pointer (though it should be `uintptr_t` 
but whatever). It then does an integer bitwise or, then converts 
the result back to a pointer.

It's undefined behavior in D because the spec just said so 
(regardless of its defined behavior in C).
Jun 15 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/15/15 11:49 AM, Alex Parrill wrote:

 On Monday, 15 June 2015 at 15:33:41 UTC, rsw0x wrote:

 this doesn't make any sense, it's referring to an object p (of void*),
 not the location of p. It's setting the lsb of p to 1 and claiming
 it's undefined behavior, when it's clearly not.

 Unless I misunderstand it.


 `p` is assumed to be a pointer. `cast(int) p` gets the integer
 representation of the pointer (though it should be `uintptr_t` but
 whatever). It then does an integer bitwise or, then converts the result
 back to a pointer.

 It's undefined behavior in D because the spec just said so (regardless
 of its defined behavior in C).


Where? In the GC document? I don't think that document is correct in 
this regard. Undefined behavior is something different than 
implementation defined behavior, which is what this is. The spec says 
(http://dlang.org/type.html):

"Casting pointers to non-pointers and vice versa is allowed in D, 
however, do not do this for any pointers that point to data allocated by 
the garbage collector."

How can the compiler POSSIBLY know whether a pointer points at GC data 
in order to make it undefined behavior?

I don't see any justification for any of these restrictions. The "to 
allow maximum flexibility" justification needs more explanation, what 
possible feature could you implement given this restriction?

It even allows using unions to do exactly the same thing. There is just 
no way this cannot be allowed.

-Steve
Jun 15 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 15 June 2015 at 16:16:28 UTC, Steven Schveighoffer 
wrote:

 How can the compiler POSSIBLY know whether a pointer points at 
 GC data in order to make it undefined behavior?


It doesn't need to, the undefined behaviour comes from the 
unfulfilled expectations of the GC. For example, a "sufficiently 
advanced GC" with enough knowledge of types, including of local 
vars, could by chance run just at the moment when the only 
instance of the address is stored in an int variable, and 
disregard that variable because it's not a pointer. It is of 
course unlikely that GCs will ever get as much knowledge about 
local variables, or even unnamed temporaries...


 I don't see any justification for any of these restrictions. 
 The "to allow maximum flexibility" justification needs more 
 explanation, what possible feature could you implement given 
 this restriction?

 It even allows using unions to do exactly the same thing. There 
 is just no way this cannot be allowed.


For unions, an advanced GC would know the type and would make 
sure to treat them conservatively. The restrictions are for the 
other cases where the types are references. The GC needs to rely 
on them being pointer if it wants to move them, for example.

But I think you are right that the current restrictions are too 
strong. After some thoughts, I can't really see any realistic 
problem with tagged pointers.
Jun 15 2015
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/15/15 12:37 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Monday, 15 June 2015 at 16:16:28 UTC, Steven Schveighoffer wrote:

 How can the compiler POSSIBLY know whether a pointer points at GC data
 in order to make it undefined behavior?


 It doesn't need to, the undefined behaviour comes from the unfulfilled
 expectations of the GC. For example, a "sufficiently advanced GC" with
 enough knowledge of types, including of local vars, could by chance run
 just at the moment when the only instance of the address is stored in an
 int variable, and disregard that variable because it's not a pointer. It
 is of course unlikely that GCs will ever get as much knowledge about
 local variables, or even unnamed temporaries...


That is clearly not the case in the documentation. 'p' maintains a 
pointer to the data the entire time, before and after it's augmented.

I'm perfectly OK changing the documentation to say "do not use integral 
types as the sole pointer to GC data." I can see how that may cause 
issues with the GC that has more advanced type information.


 But I think you are right that the current restrictions are too strong.
 After some thoughts, I can't really see any realistic problem with
 tagged pointers.


Thanks for the consideration.

-Steve
Jun 15 2015