www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - align dynamic array (for avx friendliness) hints? / possible??

reply james.p.leblanc <james.p.leblanc gmail.com> writes:
Concise question:
=================

I would like to use dynamic arrays, not for their
dynamic sizing properties per se' (slicing, appending, etc).
But, more for their memory protection and efficiencies (for
example,using foreach).

However, I must have the start of my array at an avx
friendly 32 byte alignment.

Is this easily acheivable?

Background:
===========

I am interfacing with fftw.  If I use the fftw_malloc, then
I am forced to either:

   1)  copy to/from the allocated arrays to/from my "standard"
       dlang dynamic arrays (loss of efficiency). or ...

   2)  use standard array/pointer mechanisms everywhere(loss
       of memory safely).

My thinking is that I could forego the use of the fftw_malloc,
and simply hand fftw functions my (properly aligned) pointer
of my dlang dynamic array.

All thoughts, comments, hints, greatly appreciated!

James
Aug 03 2021
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 3 August 2021 at 12:33:56 UTC, james.p.leblanc wrote:

 Concise question:
 =================

 I would like to use dynamic arrays, not for their
 dynamic sizing properties per se' (slicing, appending, etc).
 But, more for their memory protection and efficiencies (for
 example,using foreach).

 However, I must have the start of my array at an avx
 friendly 32 byte alignment.

 Is this easily acheivable?

 Background:
 ===========

 I am interfacing with fftw.  If I use the fftw_malloc, then
 I am forced to either:

   1)  copy to/from the allocated arrays to/from my "standard"
       dlang dynamic arrays (loss of efficiency). or ...

   2)  use standard array/pointer mechanisms everywhere(loss
       of memory safely).

 My thinking is that I could forego the use of the fftw_malloc,
 and simply hand fftw functions my (properly aligned) pointer
 of my dlang dynamic array.

 All thoughts, comments, hints, greatly appreciated!

 James


AFAIK, the GC only guarantees an alignment of 16. But you can 
turn any memory allocation into a slice, simply via

```
size_t length = ...;
T* myPtr = cast(T*) fftw_malloc(length * T.sizeof); // or 
aligned_alloc, posix_memalign etc.
T[] mySlice = myPtr[0 .. length];
foreach (ref e; mySlice) ...
// free!
```
Aug 03 2021
parent reply james.p.leblanc <james.p.leblanc gmail.com> writes:
On Tuesday, 3 August 2021 at 16:32:34 UTC, kinke wrote:

 On Tuesday, 3 August 2021 at 12:33:56 UTC, james.p.leblanc 
 wrote:

 Concise question:
 =================

 I would like to use dynamic arrays, not for their
 dynamic sizing properties per se' (slicing, appending, etc).
 But, more for their memory protection and efficiencies (for
 example,using foreach).

 However, I must have the start of my array at an avx
 friendly 32 byte alignment.

 Is this easily acheivable?

 Background:
 ===========

 I am interfacing with fftw.  If I use the fftw_malloc, then
 I am forced to either:

   1)  copy to/from the allocated arrays to/from my "standard"
       dlang dynamic arrays (loss of efficiency). or ...

   2)  use standard array/pointer mechanisms everywhere(loss
       of memory safely).

 My thinking is that I could forego the use of the fftw_malloc,
 and simply hand fftw functions my (properly aligned) pointer
 of my dlang dynamic array.

 All thoughts, comments, hints, greatly appreciated!

 James


 AFAIK, the GC only guarantees an alignment of 16. But you can 
 turn any memory allocation into a slice, simply via

 ```
 size_t length = ...;
 T* myPtr = cast(T*) fftw_malloc(length * T.sizeof); // or 
 aligned_alloc, posix_memalign etc.
 T[] mySlice = myPtr[0 .. length];
 foreach (ref e; mySlice) ...
 // free!
 ```


Dear Kinke,

THANKS IMMENSELY!
-----------------

This is exactly the kind of solution that I was hoping would
be possible (elegant, simply, and clear).

This really is the perfect solution, and opens up many 
possibilities for me.

(Perhaps I had been using the wrong search terms, or perhaps 
everyone already
knows how to use this "conversion of C arrays/pointers to D 
slices" solution...
but I was stumped!)

Again, thanks kindly

**Is there some highly visible place this is already documented?
If not, it would make a great blog post as it would be beneficial
to any D newcomers bridging C/D and needing AVX alignments, etc.**


Best Regards,
James
Aug 03 2021
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 8/3/21 10:50 AM, james.p.leblanc wrote:


 **Is there some highly visible place this is already documented?


For what it's worth, it appears as "slice from pointer" in my index:

 
http://ddili.org/ders/d.en/pointers.html#ix_pointers.slice%20from%20pointer

Admittedly, one needs to know the concept first to think about that 
entry. :/

Ali
Aug 03 2021
parent james.p.leblanc <james.p.leblanc gmail.com> writes:
On Tuesday, 3 August 2021 at 17:57:47 UTC, Ali Çehreli wrote:

 On 8/3/21 10:50 AM, james.p.leblanc wrote:


 **Is there some highly visible place this is already


 documented?

 For what it's worth, it appears as "slice from pointer" in my 
 index:


 http://ddili.org/ders/d.en/pointers.html#ix_pointers.slice%20from%20pointer

 Admittedly, one needs to know the concept first to think about 
 that entry. :/

 Ali


Ali,

Thanks for your message!  (And even a bigger thanks for writing 
your book!)
Your book was one of the first D programming books I read, and 
learned much from
it.

At the time of my reading, I was too much of a new-comer to the 
D-language to see
the importance of the "slice from pointer".  This concept is a 
hidden gem of the
language IMHO!

But, with my recent needs to mix and match native D with the FFTW 
C-functions
in an avx friendly alignment, I was really searching for a way to 
keep the "best
of both worlds".  The "slice from pointer" concepts is a valuable 
one.

I need to go back and read that section of book now!

Best Regards,
James
Aug 03 2021