www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - What's the D way of allocating memory?

reply "Gary Willoughby" <dev nomad.so> writes:
What's the D way of allocating memory? Do you guys just use 
malloc() and free()? Or are there functions in phobos that should 
be used? I just need a buffer of variable length allocating every 
now and then.
Aug 07 2013
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 7 August 2013 at 15:21:56 UTC, Gary Willoughby 
wrote:

 What's the D way of allocating memory? Do you guys just use 
 malloc() and free()? Or are there functions in phobos that 
 should be used? I just need a buffer of variable length 
 allocating every now and then.


Want it to be managed by GC: use `new`
Want to do manual management: use `malloc`
Aug 07 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 7 August 2013 at 15:21:56 UTC, Gary Willoughby 
wrote:

 What's the D way of allocating memory? Do you guys just use 
 malloc() and free()? Or are there functions in phobos that 
 should be used? I just need a buffer of variable length 
 allocating every now and then.


Allocating some garbage collected memory:

an array:
SomeType[] a = new SomeType[somePositiveInteger];
or
SomeType[][] m = new SomeType[][](somePosInt, someOtherPosInt);

a class
SomeClass b = new SomeClass(/*constructor args*/)



Sure, you can use malloc and free, but then of course you have to 
manage everything yourself
Aug 07 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 7 August 2013 at 15:21:56 UTC, Gary Willoughby 
wrote:

 What's the D way of allocating memory? Do you guys just use 
 malloc() and free()? Or are there functions in phobos that 
 should be used? I just need a buffer of variable length 
 allocating every now and then.


I usually use "new": It takes care of T.sieof, it initializes my 
memory to T.init, and conveniently returns a slice. Furthermore, 
the slice has "append" info, which is always a nice plus. This is 
really the equivalent of C++'s new.

If you want to do "semi-manual" memory management, you can use 
GC.malloc, which works like normal malloc, but is *still* managed 
by the GC. I'd actually recommend "GC.qalloc" over "GC.malloc": 
it's the same function, but qalloc actually returns more info, 
such as the total amount of memory *actually* allocated. This is 
very useful when you need a buffer of arbitrary size, as you get 
the most out of your allocation. The memory can also be eagerly 
marked as unused with "GC.free", although you'll still have to 
wait for a collect for it to actually be freed.

Finally, you have C's malloc and free. I don't have much to say 
about these that you shouldn't already know.

If you need a non-initialized buffer, but don't want to bother 
with a hand written [GC.]malloc, you can use phobos' 
std.array.uninitializedArray and 
std.array.minimallyInitializedArray. These conveniently return a 
slice of type T, containing N elements, but without actually 
initializing them. It's convenient.
Aug 07 2013
parent reply "Gary Willoughby" <dev nomad.so> writes:
On Wednesday, 7 August 2013 at 17:47:38 UTC, monarch_dodra wrote:

 If you want to do "semi-manual" memory management, you can use 
 GC.malloc, which works like normal malloc, but is *still* 
 managed by the GC. I'd actually recommend "GC.qalloc" over 
 "GC.malloc": it's the same function, but qalloc actually 
 returns more info, such as the total amount of memory 
 *actually* allocated. This is very useful when you need a 
 buffer of arbitrary size, as you get the most out of your 
 allocation. The memory can also be eagerly marked as unused 
 with "GC.free", although you'll still have to wait for a 
 collect for it to actually be freed.


Interesting. How do i access GC.malloc?
Aug 07 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 08/07/2013 11:40 AM, Gary Willoughby wrote:

 On Wednesday, 7 August 2013 at 17:47:38 UTC, monarch_dodra wrote:

 If you want to do "semi-manual" memory management, you can use
 GC.malloc, which works like normal malloc, but is *still* managed by
 the GC. I'd actually recommend "GC.qalloc" over "GC.malloc": it's the
 same function, but qalloc actually returns more info, such as the
 total amount of memory *actually* allocated. This is very useful when
 you need a buffer of arbitrary size, as you get the most out of your
 allocation. The memory can also be eagerly marked as unused with
 "GC.free", although you'll still have to wait for a collect for it to
 actually be freed.


 Interesting. How do i access GC.malloc?


import core.memory;

void main()
{
     auto p = GC.malloc(42);
}

But question to others: I wouldn't want garbage filled memory, right? So 
I should consider GC.calloc first.

Ali
Aug 07 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 7 August 2013 at 18:56:40 UTC, Ali Çehreli wrote:

 import core.memory;

 void main()
 {
     auto p = GC.malloc(42);
 }

 But question to others: I wouldn't want garbage filled memory, 
 right? So I should consider GC.calloc first.

 Ali


Depends on why you think garbage is a problem I guess.

If it is because of false pointers, yet you aren't storing any 
pointers, then simply allocate using GC.BlkAttr.NO_SCAN. Then, 
garbage won't be a problem.

Otherwise, yeah, calloc should do the trick. Unfortunatly, calloc 
doesn't return much other than a void*, so depending on what you 
are doing, a qalloc + memset might be better.

Also, keep in mind that 0-initialization is not T.init 
initialization. This means that calloc'ed memory is not 
"initialized" per say, it is just zero'ed memory. An emplace is 
necessary before attempting, say an assignment.
Aug 07 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 08/07/2013 01:16 PM, monarch_dodra wrote:


 On Wednesday, 7 August 2013 at 18:56:40 UTC, Ali Çehreli wrote:



 But question to others: I wouldn't want garbage filled memory, right?
 So I should consider GC.calloc first.





 Depends on why you think garbage is a problem I guess.

 If it is because of false pointers


Precisely.


, yet you aren't storing any pointers, then simply allocate
 using GC.BlkAttr.NO_SCAN. Then, garbage won't be a problem.


But the attribute is in effect throughout the lifetime of that memory 
block, right?


 Otherwise, yeah, calloc should do the trick. Unfortunatly, calloc
 doesn't return much other than a void*, so depending on what you are
 doing, a qalloc + memset might be better.


qalloc seems better but just like in C and C++, when comparing GC.malloc 
and GC.calloc, the default choice should always be GC.calloc.

So, GC.malloc is preferable only if the data that I am going to place in 
there will never need GC scanning. Because it provides zeroed-out 
memory, GC.calloc seems to be slower but I hear that that is not a 
problem on modern systems. (I have vague recollections that the system 
has already zeroed-out memory; but I am not sure.)


 Also, keep in mind that 0-initialization is not T.init initialization.
 This means that calloc'ed memory is not "initialized" per say, it is
 just zero'ed memory. An emplace is necessary before attempting, say an
 assignment.


Exactly. I wouldn't expect more from a function that returns void*. :)

Ali
Aug 07 2013
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 7 August 2013 at 22:03:16 UTC, Ali Çehreli wrote:

 On 08/07/2013 01:16 PM, monarch_dodra wrote:


 On Wednesday, 7 August 2013 at 18:56:40 UTC, Ali Çehreli


 wrote:


 But question to others: I wouldn't want garbage filled




 memory, right?

 So I should consider GC.calloc first.





 Depends on why you think garbage is a problem I guess.

 If it is because of false pointers


 Precisely.


, yet you aren't storing any pointers, then simply allocate
 using GC.BlkAttr.NO_SCAN. Then, garbage won't be a problem.


 But the attribute is in effect throughout the lifetime of that 
 memory block, right?


I *think*, however, GC.realloc and GC.extend take a BlkAttr 
attribute, and I'm not sure how that works if they don't match...
Aug 08 2013
prev sibling parent reply "Gary Willoughby" <dev nomad.so> writes:
On Wednesday, 7 August 2013 at 22:03:16 UTC, Ali Çehreli wrote:

 qalloc seems better but just like in C and C++, when comparing 
 GC.malloc and GC.calloc, the default choice should always be 
 GC.calloc.


As a side note, i was just checking this out in core.memory and i 
was surprised:

     /**
      * Requests an aligned block of managed memory from the 
garbage collector,
      * which is initialized with all bits set to zero.  This 
memory may be
      * deleted at will with a call to free, or it may be 
discarded and cleaned
      * up automatically during a collection run.  If allocation 
fails, this
      * function will call onOutOfMemory which is expected to 
throw an
      * OutOfMemoryError.
      *
      * Params:
      *  sz = The desired allocation size in bytes.
      *  ba = A bitmask of the attributes to set on this block.
      *
      * Returns:
      *  A reference to the allocated memory or null if 
insufficient memory
      *  is available.
      *
      * Throws:
      *  OutOfMemoryError on allocation failure.
      */
     static void* calloc( size_t sz, uint ba = 0 ) pure nothrow
     {
         return gc_calloc( sz, ba );
     }

calloc is marked as nothrow but apparently throws 
OutOfMemoryError on allocation failure? ..eh? O_o
Aug 08 2013
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 8 August 2013 at 18:35:06 UTC, Gary Willoughby wrote:

 calloc is marked as nothrow but apparently throws 
 OutOfMemoryError on allocation failure? ..eh? O_o


http://dlang.org/function.html
Nothrow Functions:
Nothrow functions do not throw any exceptions derived from class 
Exception.

Error is not Exception (they are both Throwable though), Errors 
are considered non-recoverable and leaving program in invalid 
state. They are allowed even in nothrow functions.
Aug 08 2013
parent "Gary Willoughby" <dev nomad.so> writes:
On Thursday, 8 August 2013 at 18:53:02 UTC, Dicebot wrote:

 On Thursday, 8 August 2013 at 18:35:06 UTC, Gary Willoughby 
 wrote:

 calloc is marked as nothrow but apparently throws 
 OutOfMemoryError on allocation failure? ..eh? O_o


 http://dlang.org/function.html
 Nothrow Functions:
 Nothrow functions do not throw any exceptions derived from 
 class Exception.

 Error is not Exception (they are both Throwable though), Errors 
 are considered non-recoverable and leaving program in invalid 
 state. They are allowed even in nothrow functions.


Ah right thanks.
Aug 08 2013
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Aug 08, 2013 at 08:35:04PM +0200, Gary Willoughby wrote:
[...]

 calloc is marked as nothrow but apparently throws OutOfMemoryError
 on allocation failure? ..eh? O_o


nothrow means Exception objects won't be thrown.

OutOfMemoryError is a Throwable, but not an Exception, so it is excepted
from nothrow. The idea is that these Throwable Errors are serious
runtime failures that indicate the program must terminate.

It's a very bad idea to catch an Error unless you know what you're
doing. And even then, you cannot safely resume program execution because
the fact that it can get thrown from nothrow functions means that some
cleanup code may have not been executed (nothrow functions do not
register stack unwinding handlers), so your program is potentially in an
inconsistent state. The only sane thing to do upon catching an Error is
to perform any last-ditch backups of important data, then terminate the
program.


T

-- 
The best compiler is between your ears. -- Michael Abrash
Aug 08 2013