www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Memory Allocation

reply Enigma <Enigma dot.com> writes:
I have a memory buffer allocated using different methods. It is 
simply a pointer and a size.

I would like to be able to manage this buffer by treating it as a 
memory pool or heap. I think I can use allocators to do this but 
not sure how.

Effectively I want something like new or malloc but it pulls from 
the memory buffer rather than the program heap.

// Allocated once at program start
void* FancyBuffer = FancyAlloc(1000);


Then when I want to use the buffer I'll do stuff like



auto myptr = FancyMalloc(10);

....

FancyFree(myptr);


or whatever.

The main thing is, I don't want to have to write my own allocator 
to manage this buffer as it seems that D's allocators would do a 
better job.

I imagine that most of the time the buffer will not have more 
than one piece of code using it(no overlapping uses) but since I 
won't be 100% sure, I need allow for the cases where there might 
be overlapping usage. (else I wouldn't ever have to worry about 
"allocating or releasing" from it.

Thanks.
Mar 29
next sibling parent reply Faux Amis <faux amis.com> writes:
On 2017-03-29 21:19, Enigma wrote:
 I have a memory buffer allocated using different methods. It is simply a
 pointer and a size.
Can you maybe just tread it like an array and slice it for allocation?
Mar 29
parent Faux Amis <faux amis.com> writes:
On 2017-03-29 23:30, Faux Amis wrote:
 On 2017-03-29 21:19, Enigma wrote:
 I have a memory buffer allocated using different methods. It is simply a
 pointer and a size.
Can you maybe just tread it like an array and slice it for allocation?
*treat*
Mar 29
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:
 I have a memory buffer allocated using different methods. It is 
 simply a pointer and a size.

 I would like to be able to manage this buffer by treating it as 
 a memory pool or heap. I think I can use allocators to do this 
 but not sure how.

 Effectively I want something like new or malloc but it pulls 
 from the memory buffer rather than the program heap.

 // Allocated once at program start
 void* FancyBuffer = FancyAlloc(1000);


 Then when I want to use the buffer I'll do stuff like



 auto myptr = FancyMalloc(10);

 ....

 FancyFree(myptr);


 or whatever.

 The main thing is, I don't want to have to write my own 
 allocator to manage this buffer as it seems that D's allocators 
 would do a better job.

 I imagine that most of the time the buffer will not have more 
 than one piece of code using it(no overlapping uses) but since 
 I won't be 100% sure, I need allow for the cases where there 
 might be overlapping usage. (else I wouldn't ever have to worry 
 about "allocating or releasing" from it.

 Thanks.
It looks like you are looking for this: http://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_region.html.
Mar 29
parent reply Enigma <Enigma dot.com> writes:
On Wednesday, 29 March 2017 at 21:36:14 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:
 [...]
It looks like you are looking for this: http://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_region.html.
But these seem to require passing a mallocator. I simply want to pass an already allocated region/block/buffer and have the allocators use it. I will allocate and free on my own.
Mar 29
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Wed, Mar 29, 2017 at 11:01:12PM +0000, Enigma via Digitalmars-d-learn wrote:
 On Wednesday, 29 March 2017 at 21:36:14 UTC, Petar Kirov [ZombineDev] wrote:
 On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:
 [...]
It looks like you are looking for this: http://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_region.html.
But these seem to require passing a mallocator. I simply want to pass an already allocated region/block/buffer and have the allocators use it. I will allocate and free on my own.
Huh? Where does it say that a mallocator is required? As far as I can tell, you could simply do this: void[] myBuffer = ...; auto allocator = Region!()(myBuffer); auto p = allocator.allocate(...); The default parent allocator is NullAllocator, which does nothing, so that leaves the management of myBuffer entirely up to you. T -- Life would be easier if I had the source code. -- YHL
Mar 29
parent Enigma <Enigma dot.com> writes:
On Wednesday, 29 March 2017 at 23:26:04 UTC, H. S. Teoh wrote:
 On Wed, Mar 29, 2017 at 11:01:12PM +0000, Enigma via 
 Digitalmars-d-learn wrote:
 On Wednesday, 29 March 2017 at 21:36:14 UTC, Petar Kirov 
 [ZombineDev] wrote:
 [...]
But these seem to require passing a mallocator. I simply want to pass an already allocated region/block/buffer and have the allocators use it. I will allocate and free on my own.
Huh? Where does it say that a mallocator is required? As far as I can tell, you could simply do this: void[] myBuffer = ...; auto allocator = Region!()(myBuffer); auto p = allocator.allocate(...); The default parent allocator is NullAllocator, which does nothing, so that leaves the management of myBuffer entirely up to you. T
Thanks. All the examples show using a size rather than an array so I assumed that was the case.
Mar 30
prev sibling parent reply Gary Willoughby <dev nomad.so> writes:
On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:
 I have a memory buffer allocated using different methods. It is 
 simply a pointer and a size.

 I would like to be able to manage this buffer by treating it as 
 a memory pool or heap. I think I can use allocators to do this 
 but not sure how.
You can re-task the GC to use an arena if that's suitable: https://bitbucket.org/infognition/dstuff/src/1dca752af1021ed5998bb041004465674695113f/gcarena.d?fileviewer=file-view-default
Mar 30
parent Enigma <Enigma dot.com> writes:
On Thursday, 30 March 2017 at 11:22:05 UTC, Gary Willoughby wrote:
 On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:
 I have a memory buffer allocated using different methods. It 
 is simply a pointer and a size.

 I would like to be able to manage this buffer by treating it 
 as a memory pool or heap. I think I can use allocators to do 
 this but not sure how.
You can re-task the GC to use an arena if that's suitable: https://bitbucket.org/infognition/dstuff/src/1dca752af1021ed5998bb041004465674695113f/gcarena.d?fileviewer=file-view-default
cool. thanks.
Mar 30