www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Get size of mem to free by free

reply Mark "J" Twain <Juckleberry Twain.com> writes:
I use malloc to allocate some memory, then free it later. For 
monitoring purposes, I would like to know how much is free'ed by 
free by just knowing the object. Or, rather, given a ptr 
allocated by malloc, bet the block size it allocated from the ptr 
alone.

Some C compilers have special intrinsics and such for this, does 
D have any ability? If not, any hacks? I just need something that 
works.
Aug 05 2016
next sibling parent reply H.Loom <loom grossnet.az> writes:
On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:

 I use malloc to allocate some memory, then free it later. For 
 monitoring purposes, I would like to know how much is free'ed 
 by free by just knowing the object. Or, rather, given a ptr 
 allocated by malloc, bet the block size it allocated from the 
 ptr alone.

 Some C compilers have special intrinsics and such for this, 
 does D have any ability? If not, any hacks? I just need 
 something that works.


You can wrap the C memory allocations functions with a version 
identifier, e.g


version(stat)
{
     __gshared size_t[size_t] sizes;
}

version(stat)
{
     auto malloc(size_t size)
     {
         auto result = std.c.stdlib.malloc;
         sizes[result] = size;
         return result;
     }
}
else alias malloc = std.c.stdlib.malloc;

version(stat)
{
     void free(void* ptr)
     {
         std.c.stdlib.free(ptr);
         sizes.remove(ptr);
     }
}
else alias free = std.c.stdlib.free;


that a bit a DIY but this would work.
Aug 05 2016
parent reply Mark "J" Twain <Juckleberry Twain.com> writes:
On Friday, 5 August 2016 at 20:43:12 UTC, H.Loom wrote:

 On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:

 [...]


 You can wrap the C memory allocations functions with a version 
 identifier, e.g


 version(stat)
 {
     __gshared size_t[size_t] sizes;
 }

 version(stat)
 {
     auto malloc(size_t size)
     {
         auto result = std.c.stdlib.malloc;
         sizes[result] = size;
         return result;
     }
 }
 else alias malloc = std.c.stdlib.malloc;

 version(stat)
 {
     void free(void* ptr)
     {
         std.c.stdlib.free(ptr);
         sizes.remove(ptr);
     }
 }
 else alias free = std.c.stdlib.free;


 that a bit a DIY but this would work.


Yeah, I ended up just storing the sizes in the ptr on malloc. 
Essentially the same but I don't have to keep a global around and 
deal with that.
Aug 05 2016
parent reply H.Loom <loom grossnet.az> writes:
On Friday, 5 August 2016 at 20:54:59 UTC, Mark "J" Twain wrote:

 On Friday, 5 August 2016 at 20:43:12 UTC, H.Loom wrote:

 On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:

 [...]


 You can wrap the C memory allocations functions with a version 
 identifier, e.g


 version(stat)
 {
     __gshared size_t[size_t] sizes;
 }

 version(stat)
 {
     auto malloc(size_t size)
     {
         auto result = std.c.stdlib.malloc(size);
         sizes[result] = size;
         return result;
     }
 }
 else alias malloc = std.c.stdlib.malloc;

 version(stat)
 {
     void free(void* ptr)
     {
         std.c.stdlib.free(ptr);
         sizes.remove(ptr);
     }
 }
 else alias free = std.c.stdlib.free;


 that a bit a DIY but this would work.


 Yeah, I ended up just storing the sizes in the ptr on malloc. 
 Essentially the same but I don't have to keep a global around 
 and deal with that.


you cannot get rid of a global variable to store the sizes. Even 
if it's hidden in an API you didn't write, in which case it's 
just hidden.

Have you looked at experimental allocators ?
Aug 05 2016
parent H.Loom <loom grossnet.az> writes:
On Friday, 5 August 2016 at 21:07:02 UTC, H.Loom wrote:

 On Friday, 5 August 2016 at 20:54:59 UTC, Mark "J" Twain wrote:

 On Friday, 5 August 2016 at 20:43:12 UTC, H.Loom wrote:

 On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain 
 wrote:

 [...]


 You can wrap the C memory allocations functions with a 
 version identifier, e.g


 version(stat)
 {
     __gshared size_t[size_t] sizes;
 }

 version(stat)
 {
     auto malloc(size_t size)
     {
         auto result = std.c.stdlib.malloc(size);
         sizes[result] = size;
         return result;
     }
 }
 else alias malloc = std.c.stdlib.malloc;

 version(stat)
 {
     void free(void* ptr)
     {
         std.c.stdlib.free(ptr);
         sizes.remove(ptr);
     }
 }
 else alias free = std.c.stdlib.free;


 that a bit a DIY but this would work.


 Yeah, I ended up just storing the sizes in the ptr on malloc. 
 Essentially the same but I don't have to keep a global around 
 and deal with that.


 you cannot get rid of a global variable to store the sizes. 
 Even if it's hidden in an API you didn't write, in which case 
 it's just hidden.

 Have you looked at experimental allocators ?


More especially 
https://github.com/dlang/phobos/blob/master/std/experimental/allocator/building_blocks/stats_collector.d

instead of using malloc/free you can use Mallocator (this is the 
same).

Maybe that StatsCollector!Mallocator == win for you.
Aug 05 2016
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 08/05/2016 12:55 PM, Mark J Twain wrote:

 I use malloc to allocate some memory, then free it later. For monitoring
 purposes, I would like to know how much is free'ed by free by just
 knowing the object. Or, rather, given a ptr allocated by malloc, bet the
 block size it allocated from the ptr alone.

 Some C compilers have special intrinsics and such for this, does D have
 any ability? If not, any hacks? I just need something that works.


malloc stores the allocation size and other information right before the 
pointer that it returns. You can do better than I do below if you know 
the exact objects that it stores there. And of course you don't need a 
struct. I just liked the property idea there. :)

import std.stdio;
import core.stdc.stdlib;

struct AllocatedMemory {
     void *p;

     alias SizeType = size_t;    // perhaps ulong?

      property SizeType size() {
         return *cast(SizeType*)(p - SizeType.sizeof);
     }
}

void myFree(void * p) {
     auto a = AllocatedMemory(p);
     writefln("Freeing something like 0x%x bytes", a.size);
}

void main() {
     auto p = malloc(0x12345);
     myFree(p);
}

Prints:

Freeing something like 0x12351 bytes

Ali
Aug 05 2016
parent reply H.Loom <loom grossnet.az> writes:
On Friday, 5 August 2016 at 21:25:41 UTC, Ali Çehreli wrote:

 malloc stores the allocation size and other information right 
 before the pointer that it returns.


TIL. Is it guaranteed for any malloc implementation ? I'm a bit 
surpised to read that. I thought these kind of trick was only 
used for aligned allocations !
Aug 05 2016
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 08/05/2016 03:49 PM, H.Loom wrote:

 On Friday, 5 August 2016 at 21:25:41 UTC, Ali Çehreli wrote:

 malloc stores the allocation size and other information right before
 the pointer that it returns.


 TIL. Is it guaranteed for any malloc implementation ? I'm a bit surpised
 to read that. I thought these kind of trick was only used for aligned
 allocations !


Not guaranteed of course but it's a common technique. Here is an 
implementation:

   https://code.woboq.org/userspace/glibc/malloc/malloc.c.html#1116

As I had suspected, the imprecision in my test code comes from low order 
bits being used as flags (P and M in that source code).

And I think malloc always returns memory properly aligned. After all, 
the user should be able to put anything there.

Ali
Aug 05 2016