www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - A question regarding the GC

reply bearophile <bearophileHUGS lycos.com> writes:
If I perform a std.gc.malloc(N), the Phobos GC gives me a pointer P, when I
perform a std.gc.realloc(x, 0) it frees the memory, so the GC must remember how
much N was (or probably its value rounded to the nearest larger multiple of 4,
8 or 16 bytes).
So given the pointer P, how can I ask the Phobos GC to quickly tell me how much
N was? (and I'd like to know how much approximated such N' value is)
This will allow me to avoid saving that N value again into my data structures,
avoiding wasting space to keep N two times in memory.
If the Phobos GC API is currently unable to give me this N' value and
approximations, can't we modify the GC so it can give us such values?

Bye,
bearophile
Jul 19 2008
parent reply torhu <no spam.invalid> writes:
bearophile wrote:
 If I perform a std.gc.malloc(N), the Phobos GC gives me a pointer P, when I
perform a std.gc.realloc(x, 0) it frees the memory, so the GC must remember how
much N was (or probably its value rounded to the nearest larger multiple of 4,
8 or 16 bytes).
 So given the pointer P, how can I ask the Phobos GC to quickly tell me how
much N was? (and I'd like to know how much approximated such N' value is)
 This will allow me to avoid saving that N value again into my data structures,
avoiding wasting space to keep N two times in memory.
 If the Phobos GC API is currently unable to give me this N' value and
approximations, can't we modify the GC so it can give us such values?
 

Are you aware of std.gc.capacity()? If you want to know what sizes the GC rounds to when allocating, you probably need to read the source, which comes with dmd.
Jul 19 2008
parent reply bearophile <bearophileHUGS lycos.com> writes:
torhu:
 Are you aware of std.gc.capacity()?

The docs say:
uint capacity(void* p); Returns capacity (size of the memory block) that p
points to the beginning of. If p does not point into the gc memory pool, or
does not point to the beginning of an allocated memory block, 0 is returned.<

Is it safe to read/write all the bytes of such capacity (even if it's higher than the size of the requested memory block)?
 If you want to know what sizes the 
 GC rounds to when allocating, you probably need to read the source, 
 which comes with dmd.

Okay, but I think a constant property may be added to the GC API. Thank you, bye, bearophile
Jul 20 2008
next sibling parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"bearophile" <bearophileHUGS lycos.com> wrote in message 
news:g5v9je$uos$1 digitalmars.com...
 torhu:
 Are you aware of std.gc.capacity()?

The docs say:
uint capacity(void* p); Returns capacity (size of the memory block) that p 
points to the beginning of. If p does not point into the gc memory pool, 
or does not point to the beginning of an allocated memory block, 0 is 
returned.<

Is it safe to read/write all the bytes of such capacity (even if it's higher than the size of the requested memory block)?

Yes, the GC always allocates memory blocks in sizes that are powers of two, at least up to the size of an OS memory page. After that I think it's rounded up to some number of pages. So yes, a lot of times there will be "holes" after your allocations that you can access with pointer trickery.
Jul 20 2008
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
bearophile wrote:
 torhu:
 Are you aware of std.gc.capacity()?

The docs say:
 uint capacity(void* p); Returns capacity (size of the memory block) that p
points to the beginning of. If p does not point into the gc memory pool, or
does not point to the beginning of an allocated memory block, 0 is returned.<

Is it safe to read/write all the bytes of such capacity (even if it's higher than the size of the requested memory block)?

Since the GC provides a method to obtain this information I believe that implicit permission has been given to use the capacity indicated. After all, the GC wouldn't tell you about capacity it didn't want you to use :-) Sean
Jul 20 2008
parent reply bearophile <bearophileHUGS lycos.com> writes:
Sean Kelly:
 Since the GC provides a method to obtain this information I believe that 
 implicit permission has been given to use the capacity indicated.  After 
 all, the GC wouldn't tell you about capacity it didn't want you to use :-)

I see. (So far I have meant the capacity value as the higher value the GC can realloc the memory block until it needs to copy the block to a new position. I have meant it as little useful value. Now I'll find better ways to use it, the first usage will probably be in the implementation of a templeted unrolled linked list :-) ). Related note: the std.c.malloc() too must store the size of the allocated block somewhere, but I presume such value can't be found in simple ways... Bye and thank you, bearophile
Jul 20 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
bearophile wrote:
 Sean Kelly:
 Since the GC provides a method to obtain this information I believe that 
 implicit permission has been given to use the capacity indicated.  After 
 all, the GC wouldn't tell you about capacity it didn't want you to use :-)

I see. (So far I have meant the capacity value as the higher value the GC can realloc the memory block until it needs to copy the block to a new position. I have meant it as little useful value. Now I'll find better ways to use it, the first usage will probably be in the implementation of a templeted unrolled linked list :-) ). Related note: the std.c.malloc() too must store the size of the allocated block somewhere, but I presume such value can't be found in simple ways...

You'd have to know how the allocator was implemented, or modify it (in the case of an open source allocator like nedmalloc) to add the routine. This is one call that I wish had been in C from the start. Oh well. Sean
Jul 20 2008
parent Sean Kelly <sean invisibleduck.org> writes:
Sean Kelly wrote:
 bearophile wrote:
 Sean Kelly:
 Since the GC provides a method to obtain this information I believe 
 that implicit permission has been given to use the capacity 
 indicated.  After all, the GC wouldn't tell you about capacity it 
 didn't want you to use :-)

I see. (So far I have meant the capacity value as the higher value the GC can realloc the memory block until it needs to copy the block to a new position. I have meant it as little useful value. Now I'll find better ways to use it, the first usage will probably be in the implementation of a templeted unrolled linked list :-) ). Related note: the std.c.malloc() too must store the size of the allocated block somewhere, but I presume such value can't be found in simple ways...

You'd have to know how the allocator was implemented, or modify it (in the case of an open source allocator like nedmalloc) to add the routine. This is one call that I wish had been in C from the start. Oh well.

Oh, for what it's worth... some allocators can claim adjoining free blocks when reallocating. I would not expect the GC to report this memory as available capacity because it isn't technically part of the memory block in question and so may disappear at some point. That is, I would say that capacity() should only return the already reserved space for the block in question. Sean
Jul 20 2008