www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is there an easy way to convert a pointer to malloc'd memory to an

reply Gary Willoughby <dev nomad.so> writes:
I have a T* pointer to the start of a malloc'd chunk of memory, 
the type T and the number of T's stored in the chunk.

Is there an efficient way of converting this information to a D 
array of type T[] or even T[n]?
May 24 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:

 I have a T* pointer to the start of a malloc'd chunk of memory, 
 the type T and the number of T's stored in the chunk.

 Is there an efficient way of converting this information to a D 
 array of type T[] or even T[n]?


Slice it:

T[] = t[0 .. n];
May 24 2016
next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:

 On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:

 I have a T* pointer to the start of a malloc'd chunk of 
 memory, the type T and the number of T's stored in the chunk.

 Is there an efficient way of converting this information to a 
 D array of type T[] or even T[n]?


 Slice it:

 T[] = t[0 .. n];


  You can do that??? I thought slices weren't allowed on raw 
pointers. You just blew my mind away!
May 24 2016
next sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, May 24, 2016 19:52:17 Era Scarecrow via Digitalmars-d-learn wrote:

 On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:

 On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:

 I have a T* pointer to the start of a malloc'd chunk of
 memory, the type T and the number of T's stored in the chunk.

 Is there an efficient way of converting this information to a
 D array of type T[] or even T[n]?


 Slice it:

 T[] = t[0 .. n];


   You can do that??? I thought slices weren't allowed on raw
 pointers. You just blew my mind away!


A dynamic array is basically just

struct Array(T)
{
    size_t length;
    T* ptr;
}

It can point to any memory. Much as its the default, a dynamic array does
not need to be backed by GC-allocated memory. It's just that unlike when
it's backed by GC-allocated memory, you have to worry about keeping track of
whether that memory is currently being used and when it needs to be freed,
and the dynamic array is guranteed to not have enough capacity to grow in
place, so any kind of array operation that would require growing the array
(e.g. calling reserve or appending) would be guaranteed to reallocate,
whereas when the dynamic array is backed by GC-allocated memory, it may or
may not have to reallocate when you do something like append to it or call
reserve. But aside from worrying about keeping track of its original memory,
having a dynamic array backed by malloc-ed memory is basically semantically
identical to having it backed by the GC. T[] really doesn't care what memory
it refers to. It works either way as long as the memory it refers to
continues to be valid.

- Jonathan M Davis
May 24 2016
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 24 May 2016 at 19:52:17 UTC, Era Scarecrow wrote:

  You can do that??? I thought slices weren't allowed on raw 
 pointers. You just blew my mind away!


Slicing and indexing are both allowed (actually, that's a way to 
disable bounds checking on an individual expression, see tip: 
http://arsdnet.net/this-week-in-d/dec-06.html ) on pointers, 
though it isn't considered  safe.

Be careful though: it is still manually managed, so if you pass 
it to some function that keeps a slice to your slice, you open 
yourself up to a use-after-free bug!
May 25 2016
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 25 May 2016 at 12:06:58 UTC, Adam D. Ruppe wrote:

 Slicing and indexing are both allowed (actually, that's a way 
 to disable bounds checking on an individual expression, see 
 tip: http://arsdnet.net/this-week-in-d/dec-06.html ) on 
 pointers, though it isn't considered  safe.

 Be careful though: it is still manually managed, so if you pass 
 it to some function that keeps a slice to your slice, you open 
 yourself up to a use-after-free bug!


  If I knew about this trick in the past, it's been years so it 
would have been easy to forget. Still awesome to rediscover. 
Rereading a lot of stuff.

  As for passing slices to functions, I generally only do that 
when I know it won't hold onto it (or try to resize it or 
anything weird), which then lets me use stack memory rather than 
heap memory for temporaries.
May 25 2016
prev sibling parent reply Gary Willoughby <dev nomad.so> writes:
On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:

 On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:

 I have a T* pointer to the start of a malloc'd chunk of 
 memory, the type T and the number of T's stored in the chunk.

 Is there an efficient way of converting this information to a 
 D array of type T[] or even T[n]?


 Slice it:

 T[] = t[0 .. n];


That! ...is amazing!
May 24 2016
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 24 May 2016 20:58:14 +0000
schrieb Gary Willoughby <dev nomad.so>:


 On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:

 On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:  

 I have a T* pointer to the start of a malloc'd chunk of 
 memory, the type T and the number of T's stored in the chunk.

 Is there an efficient way of converting this information to a 
 D array of type T[] or even T[n]?  


 Slice it:

 T[] = t[0 .. n];  


 
 That! ...is amazing!


How did you _ever_ read structs from files without this
knowledge?

-- 
Marco
May 25 2016
prev sibling parent ZombineDev <petar.p.kirov gmail.com> writes:
On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:

 I have a T* pointer to the start of a malloc'd chunk of memory, 
 the type T and the number of T's stored in the chunk.

 Is there an efficient way of converting this information to a D 
 array of type T[] or even T[n]?


BTW, the simplest way to use C malloc to allocate and initialize 
a D array is with 
http://dlang.org/phobos/std_experimental_allocator#makeArray:

struct MyObj { /* ... */ }

{
     MyObj[] arr = Mallocator.instance
         .makeArray!MyObj(10, ctorArgs);
     scope (exit) Mallocator.instance.dispose(arr);

     // use arr...

} // arr is freed at the end of the scope

Which is rougly equivalent to:

void* p = malloc(10 * MyObj.sizeof);
T[] arr = (cast(T*)p)[0 .. 10];
foreach (ref elem; arr)
     emplace(elem, ctorArgs);

The benfit of using make* and dispose is that they allow you to 
change the allocator if you so need. You can also use the GC:
int[] arr = GCAllocator.instance.makeArray!int(128);
// no need to free arr

The common idiom is that the user of a function that produces an 
array should supply the allocator as a parameter through which 
the function will make allocations. This useful when you know for 
example that the allocation would be small and short-lived in 
which case it would better to use a stack allocator, rather than 
a heap allocator.
May 25 2016