www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Containers and arrays with custom memory allocators

reply Ivan <ibaltazar0 yahoo.com> writes:
Hi,

I am a C/C++ programmer interested in using D as a replacement 
for C/C++.

I do care a lot about performance and memory management, so
I want to use my own (or from std.experimental) memory
allocators.

Are there any good tutorials or examples about how to use
custom memory allocators for arrays and existing containers?

Or should I just go ahead and write my own containers that are 
allocator
aware?

Thanks.
Oct 17
next sibling parent Aldo <aldocd4 outlook.com> writes:
On Tuesday, 17 October 2017 at 14:14:19 UTC, Ivan wrote:
 Hi,

 I am a C/C++ programmer interested in using D as a replacement 
 for C/C++.

 I do care a lot about performance and memory management, so
 I want to use my own (or from std.experimental) memory
 allocators.

 Are there any good tutorials or examples about how to use
 custom memory allocators for arrays and existing containers?

 Or should I just go ahead and write my own containers that are 
 allocator
 aware?

 Thanks.
You can check this repo : https://github.com/economicmodeling/containers It contains containers backed by std.experimental.allocator.
Oct 17
prev sibling next sibling parent Fra Mecca <me francescomecca.eu> writes:
On Tuesday, 17 October 2017 at 14:14:19 UTC, Ivan wrote:
 Hi,

 I am a C/C++ programmer interested in using D as a replacement 
 for C/C++.

 I do care a lot about performance and memory management, so
 I want to use my own (or from std.experimental) memory
 allocators.

 Are there any good tutorials or examples about how to use
 custom memory allocators for arrays and existing containers?

 Or should I just go ahead and write my own containers that are 
 allocator
 aware?

 Thanks.
I am still working on it, but given your case it may be useful. https://github.com/FraMecca/D_Libraries_Registry
Oct 18
prev sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 17 October 2017 at 14:14:19 UTC, Ivan wrote:
 Hi,

 I am a C/C++ programmer interested in using D as a replacement 
 for C/C++.

 I do care a lot about performance and memory management, so
 I want to use my own (or from std.experimental) memory
 allocators.

 Are there any good tutorials or examples about how to use
 custom memory allocators for arrays and existing containers?

 Or should I just go ahead and write my own containers that are 
 allocator
 aware?

 Thanks.
I'm currently working on a set of containers as part of my very general repo https://github.com/nordlow/phobos-next/tree/master/src For instance: - https://github.com/nordlow/phobos-next/blob/master/src/basic_array.d - https://github.com/nordlow/phobos-next/blob/master/src/bitarray.d - https://github.com/nordlow/phobos-next/blob/master/src/static_bitarray.d - https://github.com/nordlow/phobos-next/blob/master/src/fixed_array.d - https://github.com/nordlow/phobos-next/blob/master/src/soa.d - https://github.com/nordlow/phobos-next/blob/master/src/hashmap.d and a lightweight polymorphic container at - https://github.com/nordlow/phobos-next/blob/master/src/variant_arrays.d and a very experimental radix-tree (trie) implementation at - https://github.com/nordlow/phobos-next/blob/master/src/trie.d These are all very performant containers, but currently lack support for std.experimental.allocator. Instead they use C-style allocation (via qualified C-memory management in qcmeman.c). Further they use Rust-like semantics via disabling of copy-construction; ` disable this(this);`. Instead copying (like for EMSI-containers and many others) is forced to be explicit via .dup member and when you want to copy or pass by move use, for instance, import std.algorithm.mutation : move; import basic_array : A = BasicArray; Ai = A!int; auto s = Ai.withLength(10); Ai src, dst; move(src, dst); someFunctionTakingArrayByValue(move(s)); // pass by move The file hashmap.d provides both a HashSet and HashMap implementation in one go using D's compile-time-code-branching-mechanism `static if`. If you want reference semantics (via reference counting) each container can be wrapper in a `std.typecons.RefCounted` for instance as import basic_array : A = BasicArray; alias Ai = A!int; import std.typecons : RefCounted; RefCounted!A x; used here https://github.com/nordlow/phobos-next/blob/master/src/basic_array.d#L1288 Further note that my work has focused heavily on making things ` safe pure nothrow nogc` via DIP-25/DIP-1000 when possible which is not the case with most other D container libraries I've tried. In some cases I might have made a mistake with my trusted taggings. Please correct me if that is the case. Also note that DIP-1000 scope analysis doesn't currently kick in correctly in templated containers because of a bug in the compiler. The bug has been filed at bugzilla and my guess is that it will soon be fixed, as making scope analysis more complete is a high priority, at least for Walter Bright. ~master currently builds with both dmd (debug mode only) and ldc2 (both debug and release mode). I'm currently searching for some part of trie.d that currently makes dmd segfault when compiled in release mode with inlining enabled. I think DMD's inlining is the root of the problem so be careful when using trie.d. My preliminary benchmark at https://github.com/nordlow/phobos-next/blob/master/benchmarks/source/app.d compiled with LDC and inlining enabled shows that `HashMap!(uint/ulong, uint/ulong)`'s `insert()` and `contains()` with FNV64 hash is at least 10 times as fast as D's builtin associative arrays on my Intel Haswell laptop.
Oct 19
next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 19 October 2017 at 08:47:09 UTC, Per Nordlöw wrote:
 The file hashmap.d provides both a HashSet and HashMap 
 implementation in one go using D's 
 compile-time-code-branching-mechanism `static if`.
Correction: I've now broken it up into - https://github.com/nordlow/phobos-next/blob/master/src/hashmap_or_hashset.d - https://github.com/nordlow/phobos-next/blob/master/src/hashset.d - https://github.com/nordlow/phobos-next/blob/master/src/hashmap.d
Oct 19
prev sibling next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 19 October 2017 at 08:47:09 UTC, Per Nordlöw wrote:
 explicit via .dup member and when you want to copy or pass by
should be: explicit via .dup member and when you want to _move_ from one l-value to another l-value or pass by move
Oct 19
prev sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 19 October 2017 at 08:47:09 UTC, Per Nordlöw wrote:
 These are all very performant containers, but currently lack 
 support for std.experimental.allocator.
Support for std.experimental.allocator is planned but currently not a priority. EMSI-containers have an elegant integration and will be an inspiration in the process.
Oct 19