www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Using shared memory and thread

reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
Dear,

I would like to put a buffer `shared(char[])` into a shared 
memory ( L2 cache ) and provide a start and end index to multiple 
thread in order to perform computation on a piece of original 
buffer.

I read doc from std.concurrency, std.parallelism, core.thead 
(druntime) and vectorization (simd) .  But I found any example or 
unittest which perform such calculus.

1/ So did you have an example somewhere ?
2/ about shared keyword do we ensure that its size is lower than 
L2 or L3 cache using :

```
core.sys.posix.unistd: sysconf, _SC_LEVEL2_CACHE_SIZE, 
_SC_LEVEL3_CACHE_SIZE;
immutable size_t L2_CACHE_SIZE = cast(size_t) 
sysconf(_SC_LEVEL2_CACHE_SIZE);
immutable size_t L3_CACHE_SIZE = cast(size_t) 
sysconf(_SC_LEVEL3_CACHE_SIZE);

```
3/ variable flagged as `shared` does at mean the variable is put 
into L2 cache ?

thanks for your help

best regards
Nov 04 2019
next sibling parent reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Monday, 4 November 2019 at 19:53:29 UTC, bioinfornatics wrote:

 Dear,

 I would like to put a buffer `shared(char[])` into a shared 
 memory ( L2 cache ) and provide a start and end index to 
 multiple thread in order to perform computation on a piece of 
 original buffer.

 [...]


One image reveals more than a thousand words, see my picture :-)
    -> http://pix.toile-libre.org/?img=1572906675.png
Nov 04 2019
parent reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Monday, 4 November 2019 at 21:53:45 UTC, bioinfornatics wrote:

 On Monday, 4 November 2019 at 19:53:29 UTC, bioinfornatics 
 wrote:

 Dear,

 I would like to put a buffer `shared(char[])` into a shared 
 memory ( L2 cache ) and provide a start and end index to 
 multiple thread in order to perform computation on a piece of 
 original buffer.

 [...]


 One image reveals more than a thousand words, see my picture :-)
    -> http://pix.toile-libre.org/?img=1572906675.png


oups got a problem with the alpha layer:
    -> http://pix.toile-libre.org/?img=1572908302.png
Nov 04 2019
parent reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Monday, 4 November 2019 at 22:19:29 UTC, bioinfornatics wrote:

 On Monday, 4 November 2019 at 21:53:45 UTC, bioinfornatics 
 wrote:

 On Monday, 4 November 2019 at 19:53:29 UTC, bioinfornatics 
 wrote:

 Dear,

 I would like to put a buffer `shared(char[])` into a shared 
 memory ( L2 cache ) and provide a start and end index to 
 multiple thread in order to perform computation on a piece of 
 original buffer.

 [...]


 One image reveals more than a thousand words, see my picture 
 :-)
    -> http://pix.toile-libre.org/?img=1572906675.png


 oups got a problem with the alpha layer:
    -> http://pix.toile-libre.org/?img=1572908302.png


here my (not working) snippet code:
    -> https://run.dlang.io/gist/75429dcb8612e00635f7d53aabc4776f

the example do not meet exactly the spec and he do not compile.
Nov 04 2019
parent reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Tuesday, 5 November 2019 at 00:04:05 UTC, bioinfornatics wrote:

 On Monday, 4 November 2019 at 22:19:29 UTC, bioinfornatics 
 wrote:

 On Monday, 4 November 2019 at 21:53:45 UTC, bioinfornatics 
 wrote:

 On Monday, 4 November 2019 at 19:53:29 UTC, bioinfornatics 
 wrote:

 [...]


 One image reveals more than a thousand words, see my picture 
 :-)
    -> http://pix.toile-libre.org/?img=1572906675.png


 oups got a problem with the alpha layer:
    -> http://pix.toile-libre.org/?img=1572908302.png


 here my (not working) snippet code:
    -> https://run.dlang.io/gist/75429dcb8612e00635f7d53aabc4776f

 the example do not meet exactly the spec and he do not compile.


arf the generated link raise a 404, here a gist 
https://gist.github.com/run-dlang/4ffcc0641acef5eedf52a17c4995599b
Nov 04 2019
parent bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Tuesday, 5 November 2019 at 00:07:40 UTC, bioinfornatics wrote:

 On Tuesday, 5 November 2019 at 00:04:05 UTC, bioinfornatics 
 wrote:

 On Monday, 4 November 2019 at 22:19:29 UTC, bioinfornatics 
 wrote:

 On Monday, 4 November 2019 at 21:53:45 UTC, bioinfornatics 
 wrote:

 On Monday, 4 November 2019 at 19:53:29 UTC, bioinfornatics 
 wrote:

 [...]


 One image reveals more than a thousand words, see my picture 
 :-)
    -> http://pix.toile-libre.org/?img=1572906675.png


 oups got a problem with the alpha layer:
    -> http://pix.toile-libre.org/?img=1572908302.png


 here my (not working) snippet code:
    -> 
 https://run.dlang.io/gist/75429dcb8612e00635f7d53aabc4776f

 the example do not meet exactly the spec and he do not compile.


 arf the generated link raise a 404, here a gist


I got some help from `feep` on IRC and he show me that the task 
pool can not to be const as reduce is not.
So now I have this script:
----> 
https://gist.github.com/run-dlang/46c753f7233a624bb5639db615c9634a
And this error:
----> 
/dlang/dmd/linux/bin64/../../src/phobos/std/parallelism.d(2730): 
Error: function std.parallelism.TaskPool.reduce!"a + 
b".reduce!(MapResult!(__lambda3, Result)).reduce cannot get frame 
pointer to onlineapp.countAllLines

I found this thread which talk about this error, but that did not 
help me to fix it: https://github.com/dlang/phobos/pull/4915

Thanks for your help





 https://gist.github.com/run-dlang/4ffcc0641acef5eedf52a17c4995599b
Nov 05 2019
prev sibling parent welkam <wwwelkam gmail.com> writes:
On Monday, 4 November 2019 at 19:53:29 UTC, bioinfornatics wrote:

 3/ variable flagged as `shared` does at mean the variable is 
 put into L2 cache ?


First caching is controlled by CPU not programmer.
Second not all architectures share L2 between cores.
4 jaguar cores share L2 cache in consoles
https://images.anandtech.com/doci/6976/Screen%20Shot%202013-05-23%20at%2012.22.12%20AM_678x452.png

2 bulldozer cores share L2 cache in FX processors
https://s3.amazonaws.com/hs-wordpress/wp-content/uploads/2017/12/13035240/bulldozer_021.jpg

each zen core has its own L2 cache in Ryzen cpu's
https://images.anandtech.com/doci/10591/HC28.AMD.Mike%20Clark.final-page-014.jpg


Third your link to run.dlang is 404

While you cant control cache contents explicitly you can control 
it by changing memory access patterns. When you fetch a peace of 
data CPU will load 64 byte size bucket/chunk that contains your 
data into L1 cache. Since L1 cache size is limited CPU had to 
remove existing data from L1 based on its eviction policy. 
Evicted data goes into L2 cache. That is the way to control what 
is where.

By default all variables(except immutable or static) are thread 
local and can only be accessed by one thread. Shared keyword 
makes data accessible trough multiple threads
Nov 05 2019