www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - safe containers with std.experimental.allocator

reply bitwise <bitwise.pvt gmail.com> writes:
When std.experimental.allocator was created, I was under the 
impression it was meant to be used to make containers. But since 
allocate() returns void[], casts are required before using that 
memory, which is unsafe.

Is my only option to wrap the casts in an  trusted helper 
function? or am I missing something?

  Thanks
Jan 20
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
 [...]
I can see that 'TypedAllocator' exists, but with a different interface than the rest of the allocators. Why wouldn't all allocators just have a common interface? Why wouldn't Mallocator take a template parameter allowing usage with 'ubyte' or 'void' for uninitialized memory via 'allocate' or 'allocateUninitialized' functions? For my own purposes, I would need my containers to have a default allocator(like Mallocator), and support an aligned allocator(doesn't seem to be present in std.experimental).
Jan 20
parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 21 Jan 2017 04:01:06 +0000, bitwise wrote:

 On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
 [...]
I can see that 'TypedAllocator' exists, but with a different interface than the rest of the allocators. Why wouldn't all allocators just have a common interface?
TypedAllocator combines several allocators, using a series of Policies to determine which one to use for a given type. The interface for allocators is mostly given by IAllocator and mostly implemented with UFCS functions defined in std.experimental.allocator.package. The one aspect that isn't given with IAllocator (because OOP doesn't support it well) is that not all allocators have to support aligned allocations. TypedAllocator overrides some of those default implementations in order to use its Policies. It doesn't expose the untyped variants so that you can't get around its Policies. Unfortunately, there's no way to require that a TypedAllocator return an aligned allocation aside from using an aligned allocator for the relevant policy.
Jan 21
prev sibling next sibling parent reply Eugene Wissner <belka caraus.de> writes:
On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks
System functions that do memory allocation, give you a memory block as a void*. So a cast is anyway required to cast to the required data type. Allocators don't cast from void* because you would need to implement the same logic in each allocator. Therefore there are functions like make/makeArray/dispose that take an allocator, cast and initialize data. As for safity part. Allocators can't provide the same safity as the garbage collector anyway, because if you use manual memory management you get all the problems like you can pass to deallocate() a wrong pointer or to free a memory block twice. But with the right abstractions and enough testing, you can reduce such problems and make the code more trusted. Alligned allocator isn't present in the std.experimental.allocator because IAllocator has a method alignedAllocate() (or similar) - so any allocator can be designed to provide aligned allocations.
Jan 20
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 05:06:07 UTC, Eugene Wissner 
wrote:
 Alligned allocator isn't present in the 
 std.experimental.allocator because IAllocator has a method 
 alignedAllocate()
I hadn't noticed this, but I immediately see two problems: 1) there is no alignedDeallocate() which is needed because aligned allocators usually prepend metadata containing a pointer to the actual start of the memory to be passed to free(). So deallocate() can't know if the memory came from allocate() or alignedAllocate(), and hence does not now how to properly free the memory. 2) The whole point of using an allocator is that is allocates memory in a different way, but provides a standard interface. It shouldn't be up to a container to know whether to call allocate() or alignedAllocate(). A container should simply call allocate() of whatever allocator it was given.
Jan 20
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 21 Jan 2017 05:52:49 +0000, bitwise wrote:

 On Saturday, 21 January 2017 at 05:06:07 UTC, Eugene Wissner wrote:
 Alligned allocator isn't present in the std.experimental.allocator
 because IAllocator has a method alignedAllocate()
I hadn't noticed this, but I immediately see two problems: 1) there is no alignedDeallocate() which is needed because aligned allocators usually prepend metadata containing a pointer to the actual start of the memory to be passed to free(). So deallocate() can't know if the memory came from allocate() or alignedAllocate(), and hence does not now how to properly free the memory.
Check the code. CAllocatorImpl's implementation of alignedAllocate returns null if the implementation doesn't support alignedAllocate. (CAllocatorImpl is the provided implementation of IAllocator.)
 2) The whole point of using an allocator is that is allocates memory in
 a different way, but provides a standard interface. It shouldn't be up
 to a container to know whether to call allocate()
 or alignedAllocate(). A container should simply call allocate() of
 whatever allocator it was given.
If you need aligned memory, you need to use alignedAllocate. You could unconditionally wrap the given allocator using BitmappedBlock, but this adds unnecessary overhead if the user passed in an allocator that supports aligned allocation.
Jan 20
prev sibling parent reply Eugene Wissner <belka caraus.de> writes:
On Saturday, 21 January 2017 at 05:52:49 UTC, bitwise wrote:
 I hadn't noticed this, but I immediately see two problems:

 1) there is no alignedDeallocate() which is needed because 
 aligned allocators usually prepend metadata containing a 
 pointer to the actual start of the memory to be passed to 
 free(). So deallocate() can't know if the memory came from 
 allocate() or alignedAllocate(), and hence does not now how to 
 properly free the memory.

 2) The whole point of using an allocator is that is allocates 
 memory in a different way, but provides a standard interface. 
 It shouldn't be up to a container to know whether to call 
 allocate() or alignedAllocate(). A container should simply call 
 allocate() of whatever allocator it was given.
yes, agreed. alignedAllocate() seems then to make the interface complexer without reason.
Jan 20
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/21/2017 01:06 AM, Eugene Wissner wrote:
 On Saturday, 21 January 2017 at 05:52:49 UTC, bitwise wrote:
 I hadn't noticed this, but I immediately see two problems:

 1) there is no alignedDeallocate() which is needed because aligned
 allocators usually prepend metadata containing a pointer to the actual
 start of the memory to be passed to free(). So deallocate() can't know
 if the memory came from allocate() or alignedAllocate(), and hence
 does not now how to properly free the memory.

 2) The whole point of using an allocator is that is allocates memory
 in a different way, but provides a standard interface. It shouldn't be
 up to a container to know whether to call allocate() or
 alignedAllocate(). A container should simply call allocate() of
 whatever allocator it was given.
yes, agreed. alignedAllocate() seems then to make the interface complexer without reason.
alignedAllocate provides access to OS/clib-provided primitives for aligned allocation. Those don't require a special deallocation function, see e.g. http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei
Jan 21
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 16:28:16 UTC, Andrei Alexandrescu 
wrote:
 alignedAllocate provides access to OS/clib-provided primitives 
 for aligned allocation. Those don't require a special 
 deallocation function, see e.g. 
 http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei
This makes sense, but then shouldn't alignedAllocate() be a free function that could be used to make an aligned allocator? Again, the point of an allocator is to provide a standard interface for memory allocation. The user of which shouldn't have to know how that memory was allocated, and I can't think of a case where this would be desired either.
Jan 21
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/21/2017 12:14 PM, bitwise wrote:
 On Saturday, 21 January 2017 at 16:28:16 UTC, Andrei Alexandrescu wrote:
 alignedAllocate provides access to OS/clib-provided primitives for
 aligned allocation. Those don't require a special deallocation
 function, see e.g.
 http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei
This makes sense, but then shouldn't alignedAllocate() be a free function that could be used to make an aligned allocator?
You mean an aligned allocation? But then the actual method, and whether it is supported at all or not, depends on the allocator.
 Again, the point of an allocator is to provide a standard interface for
 memory allocation. The user of which shouldn't have to know how that
 memory was allocated, and I can't think of a case where this would be
 desired either.
I don't understand this. It's possible there's a confusion at a different level, and we're discussing the consequences. To reboot: * Some applications need memory aligned at special powers of two, most are okay with more generic alignments such as word-level. See the "alignment" enum that each allocator must define. * Some allocators do offer allocations at unusual powers of two by means of alignedAllocate. Don't call it naively! Only applications that need such things (e.g. page-aligned, cache-line-aligned, etc) should need those. * Those allocators must be able to deallocate memory allocated with allocate() or alignedAllocate() with a single primitive deallocate(). Andrei
Jan 21
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 17:26:35 UTC, Andrei Alexandrescu 
wrote:
 On 01/21/2017 12:14 PM, bitwise wrote:
 On Saturday, 21 January 2017 at 16:28:16 UTC, Andrei 
 Alexandrescu wrote:
 alignedAllocate provides access to OS/clib-provided 
 primitives for
 aligned allocation. Those don't require a special deallocation
 function, see e.g.
 http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei
This makes sense, but then shouldn't alignedAllocate() be a free function that could be used to make an aligned allocator?
You mean an aligned allocation? But then the actual method, and whether it is supported at all or not, depends on the allocator.
 Again, the point of an allocator is to provide a standard 
 interface for
 memory allocation. The user of which shouldn't have to know 
 how that
 memory was allocated, and I can't think of a case where this 
 would be
 desired either.
I don't understand this. It's possible there's a confusion at a different level, and we're discussing the consequences. To reboot: * Some applications need memory aligned at special powers of two, most are okay with more generic alignments such as word-level. See the "alignment" enum that each allocator must define. * Some allocators do offer allocations at unusual powers of two by means of alignedAllocate. Don't call it naively! Only applications that need such things (e.g. page-aligned, cache-line-aligned, etc) should need those. * Those allocators must be able to deallocate memory allocated with allocate() or alignedAllocate() with a single primitive deallocate(). Andrei
About alignedMalloc: In C++ for example, I may want to use a vector full some SIMD type: class alignas(16) Vec4 { union { struct { float x, y, z, w; }; __m128 m; }; }; std::vector<Vec4> points = { ... }; In C++ however, 'new' does not respect over-alignment: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0035r2.html Even if new respected alignment, there is no gauruntee all containers, STL or otherwise, would use 'new' as opposed to malloc by default(maybe one day?) So I use a custom aligned allocator: template <class T, int ALIGN> class AlignedAllocator { T* allocate(size_type n) { return (T*)_aligned_malloc(ALIGN, n * sizeof(T)); } }; SIMD operations(aligned load and store) can now safely be used on the contents of the std::vector<Vec4>. std::vector knows nothing about the alignment of the memory it uses. It only knows to call allocate() of whatever allocator it's given. If I had an allocator with a function 'alignedAllocate' it wouldn't do any good. I believe this is the _correct_ design, and that a container _shouldn't_ have to know about where from, or what kind of memory it's getting. Considering the above use case, alignedAllocate() is redundant, and possibly confusing. About missing alignedDeallocate: while aligned_alloc(), which works in combination with regular 'free()', is supposed to be standard as of C++11, it's still not supported in visual studio 2015. Instead, one must use _aligned_malloc, and _aligned_free. Passing memory from _aligned_malloc to the regular version of free() causes a crash. Thus, different deallocation methods are needed for both. Also, there's homegrown aligned_allocate functions like the following, which require special deallocation functions because of the exta metadata prepended to the memory: https://github.com/dlang/phobos/blob/366f6e4e66abe96bca9fd69d03042e08f787d040/std/experimental/allocator/mallocator.d#L134-L134 I suppose you could use aligned allocation for _all_ allocations, even allocations with default alignment, but that would add extra metadata(at least 8 bytes) to _all_ allocations even when its unnecessary. So a solution could be to include the alignment as a template parameter of Mallocator, or provide an second AlignedMallocator(uint). The allocate() function of either option would return aligned memory if the 'alignment' template parameter was non-default. Then, the idea of memory alignment would be abstracted away from the containers themselves. struct Mallocator(uint alignment = platformAlignment){}){} or struct AlignedMallocator(uint alignment = platformAlignment){}){}
Jan 21
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/21/17 5:44 PM, bitwise wrote:
 About alignedMalloc:

 In C++ for example, I may want to use a vector full some SIMD type:

 class alignas(16) Vec4 {
     union {
         struct { float x, y, z, w; };
         __m128 m;
     };
 };

 std::vector<Vec4> points = { ... };

 In C++ however, 'new' does not respect over-alignment:
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0035r2.html

 Even if new respected alignment, there is no gauruntee all containers,
 STL or otherwise, would use 'new' as opposed to malloc by default(maybe
 one day?)

 So I use a custom aligned allocator:

 template <class T, int ALIGN>
 class AlignedAllocator {
     T* allocate(size_type n) {
         return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
     }
 };

 SIMD operations(aligned load and store) can now safely be used on the
 contents of the std::vector<Vec4>.

 std::vector knows nothing about the alignment of the memory it uses. It
 only knows to call allocate() of whatever allocator it's given. If I had
 an allocator with a function 'alignedAllocate' it wouldn't do any good.
 I believe this is the _correct_ design, and that a container _shouldn't_
 have to know about where from, or what kind of memory it's getting.
I understand. That's a questionable design. It only works by virtue of a long-distance convention between the rigged allocator and the element type of the vector.
 Considering the above use case, alignedAllocate() is redundant, and
 possibly confusing.
Well, you just made use of it in the rigged allocator.
 About missing alignedDeallocate:

 while aligned_alloc(), which works in combination with regular 'free()',
 is supposed to be standard as of C++11, it's still not supported in
 visual studio 2015. Instead, one must use _aligned_malloc, and
 _aligned_free. Passing memory from _aligned_malloc to the regular
 version of free() causes a crash. Thus, different deallocation methods
 are needed for both. Also, there's homegrown aligned_allocate functions
 like the following, which require special deallocation functions because
 of the exta metadata prepended to the memory:
 https://github.com/dlang/phobos/blob/366f6e4e66abe96bca9fd69d03042e08f787d040/std/experimental/allocator/mallocator.d#L134-L134


 I suppose you could use aligned allocation for _all_ allocations, even
 allocations with default alignment, but that would add extra metadata(at
 least 8 bytes) to _all_ allocations even when its unnecessary.

 So a solution could be to include the alignment as a template parameter
 of Mallocator, or provide an second AlignedMallocator(uint). The
 allocate() function of either option would return aligned memory if the
 'alignment' template parameter was non-default. Then, the idea of memory
 alignment would be abstracted away from the containers themselves.

 struct Mallocator(uint alignment = platformAlignment){}){}
 or
 struct AlignedMallocator(uint alignment = platformAlignment){}){}
It seems a matter of time until aligned_alloc gets implemented on Windows. Andrei
Jan 21
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei Alexandrescu 
wrote:
 On 1/21/17 5:44 PM, bitwise wrote:
 About alignedMalloc:

 In C++ for example, I may want to use a vector full some SIMD 
 type:

 class alignas(16) Vec4 {
     union {
         struct { float x, y, z, w; };
         __m128 m;
     };
 };

 std::vector<Vec4> points = { ... };

 In C++ however, 'new' does not respect over-alignment:
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0035r2.html

 Even if new respected alignment, there is no gauruntee all 
 containers,
 STL or otherwise, would use 'new' as opposed to malloc by 
 default(maybe
 one day?)

 So I use a custom aligned allocator:

 template <class T, int ALIGN>
 class AlignedAllocator {
     T* allocate(size_type n) {
         return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
     }
 };

 SIMD operations(aligned load and store) can now safely be used 
 on the
 contents of the std::vector<Vec4>.

 std::vector knows nothing about the alignment of the memory it 
 uses. It
 only knows to call allocate() of whatever allocator it's 
 given. If I had
 an allocator with a function 'alignedAllocate' it wouldn't do 
 any good.
 I believe this is the _correct_ design, and that a container 
 _shouldn't_
 have to know about where from, or what kind of memory it's 
 getting.
I understand. That's a questionable design. It only works by virtue of a long-distance convention between the rigged allocator and the element type of the vector.
I don't understand what's questionable about it. I don't see how abstracting the alignment away from the consumer of an allocator is a bad thing.
 Considering the above use case, alignedAllocate() is 
 redundant, and
 possibly confusing.
Well, you just made use of it in the rigged allocator.
I made use of what I would expect to be a non-member helper function. I'm saying that I don't believe alignedAllocate() should be a part of the standard interface of an allocator, and that allocators should be specialized such that allocate() returns memory with whatever alignment is needed.
 About missing alignedDeallocate:

 while aligned_alloc(), which works in combination with regular 
 'free()',
 is supposed to be standard as of C++11, it's still not 
 supported in
 visual studio 2015. Instead, one must use _aligned_malloc, and
 _aligned_free. Passing memory from _aligned_malloc to the 
 regular
 version of free() causes a crash. Thus, different deallocation 
 methods
 are needed for both. Also, there's homegrown aligned_allocate 
 functions
 like the following, which require special deallocation 
 functions because
 of the exta metadata prepended to the memory:
 https://github.com/dlang/phobos/blob/366f6e4e66abe96bca9fd69d03042e08f787d040/std/experimental/allocator/mallocator.d#L134-L134


 I suppose you could use aligned allocation for _all_ 
 allocations, even
 allocations with default alignment, but that would add extra 
 metadata(at
 least 8 bytes) to _all_ allocations even when its unnecessary.

 So a solution could be to include the alignment as a template 
 parameter
 of Mallocator, or provide an second AlignedMallocator(uint). 
 The
 allocate() function of either option would return aligned 
 memory if the
 'alignment' template parameter was non-default. Then, the idea 
 of memory
 alignment would be abstracted away from the containers 
 themselves.

 struct Mallocator(uint alignment = platformAlignment){}){}
 or
 struct AlignedMallocator(uint alignment = 
 platformAlignment){}){}
It seems a matter of time until aligned_alloc gets implemented on Windows.
But how much time? Visual studio always lags behind in standards conformance. Also, there is still the fact that some may need to use home-grown aligned allocation functions like the ones I linked above that prepend metadata to the memory returned, in which case they will need specialized deallocation functions.
Jan 21
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei Alexandrescu 
wrote:
 Andrei
Anyways, design opinions aside, I would be satisfied if an aligned allocator were included in std.allocators that provided aligned heap memory using allocate()/deallocate(). The rest is just details with no practical relevance for my immediate plans. Thanks
Jan 21
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/21/17 10:07 PM, bitwise wrote:
 On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei Alexandrescu wrote:
 Andrei
Anyways, design opinions aside, I would be satisfied if an aligned allocator were included in std.allocators that provided aligned heap memory using allocate()/deallocate(). The rest is just details with no practical relevance for my immediate plans. Thanks
That should be easy and useful as an adapter over another allocator that supports alignedAllocate(). Please create an issue for that. Thanks! -- Andrei
Jan 22
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 13:21:41 UTC, Andrei Alexandrescu 
wrote:
 On 1/21/17 10:07 PM, bitwise wrote:
 On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei 
 Alexandrescu wrote:
 Andrei
Anyways, design opinions aside, I would be satisfied if an aligned allocator were included in std.allocators that provided aligned heap memory using allocate()/deallocate(). The rest is just details with no practical relevance for my immediate plans. Thanks
That should be easy and useful as an adapter over another allocator that supports alignedAllocate(). Please create an issue for that. Thanks! -- Andrei
I would, but I'm not sure I can use these allocators. Ran into this: http://forum.dlang.org/post/akohcwiotlcrodemhexx forum.dlang.org As stated in that thread, I can mark the member instance of the allocator in the container as 'shared', but then, if I use an allocator _without_ 'shared' methods, I get errors as well. I'm not sure if there is a solution for this, but at this point, I try to just avoid 'shared' at all costs. The fact that Mallocator is thread safe should not stop me from using it in a non-thread safe class, but it does.
Jan 24
parent bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 24 January 2017 at 19:38:31 UTC, bitwise wrote:
 I would, but I'm not sure I can use these allocators.

 Ran into this:
 http://forum.dlang.org/post/akohcwiotlcrodemhexx forum.dlang.org

 As stated in that thread, I can mark the member instance of the 
 allocator in the container as 'shared', but then, if I use an 
 allocator _without_ 'shared' methods, I get errors as well.

 I'm not sure if there is a solution for this, but at this 
 point, I try to just avoid 'shared' at all costs. The fact that 
 Mallocator is thread safe should not stop me from using it in a 
 non-thread safe class, but it does.
So I guess there is no solution for this? I've written my own allocators at this point, which is not something I did with great joy.
Jan 25
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 13:21:41 UTC, Andrei Alexandrescu 
wrote:
 -- Andrei
After exploring this idea( safe containers), then looking at std.Array again, it's actually pretty smart. I had concerns about efficiency, but RefCounted seems to force allocation like make_shared<>, eliminating the access overhead I thought was present. It doesn't look like RefCounted will work once allocators are introduced, but you seem to already be working on generalized ref-counted buffers. Well played, sir. I decided to make my own containers classes, in the hopes that one day, I can just slap rc in front of them and be done with the GC.
Jan 26
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 21 Jan 2017 22:44:49 +0000, bitwise wrote:
 So I use a custom aligned allocator:
 
 template <class T, int ALIGN>
 class AlignedAllocator {
      T* allocate(size_type n) {
          return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
      }
 };
 
 SIMD operations(aligned load and store) can now safely be used on the
 contents of the std::vector<Vec4>.
This is fine when you have fine-grained control of which allocators are used for which purpose and you know more about what the code needs than whoever is consuming the allocator. alignedAllocate() is for when the code consuming the allocator knows more about alignment needs than whoever is supplying the allocator, which is more likely to crop up with coarse-grained allocator control.
Jan 21
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 00:13:10 UTC, Chris Wright wrote:
 On Sat, 21 Jan 2017 22:44:49 +0000, bitwise wrote:
 So I use a custom aligned allocator:
 
 template <class T, int ALIGN>
 class AlignedAllocator {
      T* allocate(size_type n) {
          return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
      }
 };
 
 SIMD operations(aligned load and store) can now safely be used 
 on the contents of the std::vector<Vec4>.
alignedAllocate() is for when the code consuming the allocator knows more about alignment needs than whoever is supplying the allocator, which is more likely to crop up with coarse-grained allocator control.
I understand this, but I've given a solid example which supports the former, and haven't seen a counter example which supports the latter.
Jan 21
parent reply Chris Wright <dhasenan gmail.com> writes:
On Sun, 22 Jan 2017 00:53:45 +0000, bitwise wrote:
 I understand this, but I've given a solid example which supports the
 former, and haven't seen a counter example which supports the latter.
Because examples that are easy to generate are small and small examples don't have coarse-grained anything.
Jan 21
parent bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 01:02:09 UTC, Chris Wright wrote:
 On Sun, 22 Jan 2017 00:53:45 +0000, bitwise wrote:
 I understand this, but I've given a solid example which 
 supports the former, and haven't seen a counter example which 
 supports the latter.
Because examples that are easy to generate are small and small examples don't have coarse-grained anything.
Please explain
Jan 21
prev sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks
There's no need for casts. You can just use the high-level wrappers: http://dlang.org/phobos-prerelease/std_experimental_allocator#.make http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeArray http://dlang.org/phobos-prerelease/std_experimental_allocator#.dispose http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeMultidimensionalArray http://dlang.org/phobos-prerelease/std_experimental_allocator#.disposeMultidimensionalArray
Jan 21
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Saturday, 21 January 2017 at 09:56:29 UTC, ZombineDev wrote:
 On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks
There's no need for casts. You can just use the high-level wrappers: http://dlang.org/phobos-prerelease/std_experimental_allocator#.make http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeArray http://dlang.org/phobos-prerelease/std_experimental_allocator#.dispose http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeMultidimensionalArray http://dlang.org/phobos-prerelease/std_experimental_allocator#.disposeMultidimensionalArray
Though, since you are implementing a container that manually manages memory, you won't be able to get away with no trusted parts in your code. http://dlang.org/phobos-prerelease/std_experimental_allocator#.expandArray can help, but you'll still need to call dispose/deallocate somewhere yourself and you're the only that knows when is safe to do this. If you look at how Rust's standard library is implemented, you'll see the same pattern. The users have a safe API, but underneath you'll that the library uses many 'unsafe' blocks where the compiler doesn't see the full picture which only the author of the code knows.
Jan 21
parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 10:15:46 UTC, ZombineDev wrote:
 On Saturday, 21 January 2017 at 09:56:29 UTC, ZombineDev wrote:
 On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:
 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks
There's no need for casts. You can just use the high-level wrappers: http://dlang.org/phobos-prerelease/std_experimental_allocator#.make http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeArray http://dlang.org/phobos-prerelease/std_experimental_allocator#.dispose http://dlang.org/phobos-prerelease/std_experimental_allocator#.makeMultidimensionalArray http://dlang.org/phobos-prerelease/std_experimental_allocator#.disposeMultidimensionalArray
Though, since you are implementing a container that manually manages memory, you won't be able to get away with no trusted parts in your code. http://dlang.org/phobos-prerelease/std_experimental_allocator#.expandArray can help, but you'll still need to call dispose/deallocate somewhere yourself and you're the only that knows when is safe to do this. If you look at how Rust's standard library is implemented, you'll see the same pattern. The users have a safe API, but underneath you'll that the library uses many 'unsafe' blocks where the compiler doesn't see the full picture which only the author of the code knows.
I guess you're right. This is my first time looking at safe in detail. I figured something like malloc() could trusted, but I suppose free cannot. I pictured being able to use an safe allocator with an safe container. How else could one reliably create safe containers, or any safe object for that matter, that allows the substitution of arbitrary and possibly unsafe components? I guess the answer is responsible programming, but that seems counter intuitive given the goals of safe.
Jan 21