www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to work around the infamous dual-context when using delegates

reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
I have this small program here

test.d:
```
import std;
string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}
void main() {
     auto servers = ["s1", "s2", "s3"];
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(user => servers.doSomething(user))(users));
}
```

The first map just works as expected, for the parallel amap though fromo 
(https://dlang.org/phobos/std_parallelism.html) I get the following 
warning with dmd:

```
/Users/.../dlang/dmd-2.096.1/osx/bin/../../src/phobos/std/parallelism.d(1711): 
Deprecation: function `test.main.amap!(string[]).amap` function requires 
a dual-context, which is deprecated
```

for ldc the build fails with:
```
/Users/.../dlang/ldc-1.26.0/bin/../import/std/parallelism.d(1711): 
Deprecation: function `test.main.amap!(string[]).amap` function requires 
a dual-context, which is deprecated
test.d(9):        instantiated from here: `amap!(string[])`
/Users/.../dlang/ldc-1.26.0/bin/../import/std/parallelism.d(1711): 
Error: function `test.main.amap!(string[]).amap` requires a 
dual-context, which is not yet supported by LDC
```


Thanks in advance for you insights,
Christian
May 27 2021
next sibling parent reply sighoya <sighoya gmail.com> writes:
On Thursday, 27 May 2021 at 09:58:40 UTC, Christian Köstlin wrote:

 I have this small program here

 test.d:
 ```
 import std;
 string doSomething(string[] servers, string user) {
     return user ~ servers[0];
 }
 void main() {
     auto servers = ["s1", "s2", "s3"];
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(user => 
 servers.doSomething(user))(users));
 }
 ```




I think it relates to 
https://issues.dlang.org/show_bug.cgi?id=5710

The reason is that amap requires a this pointer of type TaskPool 
and a context pointer to the closure which belongs to main, at 
least because it requires servers. Having both isn't possible due 
to problems in non DMD compilers.

If you rewrite it more statically:
```D
string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}

string closure(string user)
{
     return servers.doSomething(user);
}
auto servers = ["s1", "s2", "s3"];
int main()
{
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(closure)(users));
     return 0;
}
```

PS: Just enable markdown if you want to highlight D code
May 27 2021
next sibling parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
Thanks for the proposed solution. It also works in my slightly bigger 
program (although I do not like to make servers more global).

I tried also the following (which unfortunately also does not work as 
intended):

```D
import std;
string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}

int main()
{
     auto users = ["u1", "u2", "u3"];
     auto servers = ["s1", "s2", "s3"];
     auto usersWithServers = users.map!(user => tuple!("user", 
"servers")(user, servers)).array;
     writeln(map!(userWithServers => 
userWithServers.servers.doSomething(userWithServers.user))(usersWithServers));
     writeln(taskPool.amap!(userWithServers => 
userWithServers.servers.doSomething(userWithServers.user))(usersWithServers));
     return 0;
}
```

Here I try to put the data I need together into one tuple ("manually") 
and then pass it all to amap. Can you explain me, where here a double 
context is needed? Because all data now should be passed as arguments to 
amap?

Kind regards,
Christian
May 27 2021
parent reply sighoya <sighoya gmail.com> writes:
On Thursday, 27 May 2021 at 12:17:36 UTC, Christian Köstlin wrote:

 Can you explain me, where here a double context is needed? 
 Because all data now should be passed as arguments to amap?

 Kind regards,
 Christian


I  believe D's type system isn't smart enough to see independence 
between context and closure, otherwise your original example 
would also work as users and servers are context independent.

What about:

```D
string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}
void main() {
     static servers = ["s1", "s2", "s3"];
     static users = ["u1", "u2", "u3"];
     static lambda = (string user) => servers.doSomething(user);
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(lambda)(users));
}
```
May 27 2021
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 14:48, sighoya wrote:

 On Thursday, 27 May 2021 at 12:17:36 UTC, Christian Köstlin wrote:

 Can you explain me, where here a double context is needed? Because all 
 data now should be passed as arguments to amap?

 Kind regards,
 Christian


 
 I  believe D's type system isn't smart enough to see independence 
 between context and closure, otherwise your original example would also 
 work as users and servers are context independent.
 
 What about:
 
 ```D
 string doSomething(string[] servers, string user) {
      return user ~ servers[0];
 }
 void main() {
      static servers = ["s1", "s2", "s3"];
      static users = ["u1", "u2", "u3"];
      static lambda = (string user) => servers.doSomething(user);
      writeln(map!(user => servers.doSomething(user))(users));
      writeln(taskPool.amap!(lambda)(users));
 }
 ```
 


That looks nice, but unfortunately my data for servers and users in the 
real world is not static but comes from a config file.
May 27 2021
parent reply sighoya <sighoya gmail.com> writes:
On Thursday, 27 May 2021 at 12:58:28 UTC, Christian Köstlin wrote:


 That looks nice, but unfortunately my data for servers and 
 users in the real world is not static but comes from a config 
 file.


Okay, but then parametrizing the static lambda with runtime 
parameters should work. The important fact is that the closure 
needs to be static.
May 27 2021
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 15:00, sighoya wrote:

 On Thursday, 27 May 2021 at 12:58:28 UTC, Christian Köstlin wrote:
 

 That looks nice, but unfortunately my data for servers and users in 
 the real world is not static but comes from a config file.


 
 Okay, but then parametrizing the static lambda with runtime parameters 
 should work. The important fact is that the closure needs to be static.


Ah thanks, now I understand.
So what I came up with now is a combination of the things mentioned:

```D
import std;

string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}
struct UserWithServers {
     string user;
     string[] servers;
}
void main(string[] args) {
     auto servers = args;
     auto users = ["u1", "u2", "u3"];
     auto usersWithServers = users.map!(user => UserWithServers(user, 
servers)).array;

     static fn = function(UserWithServers user) => 
user.servers.doSomething(user.user);
     writeln(taskPool.amap!(fn)(usersWithServers));
}
```

Making also the example a little bit more "realistic" by using dynamic 
data for servers.

I would like to use auto fn, but somehow saying that its a function is 
not enough for dmd. From my understanding a function would never need a 
context?!?

Thanks a lot!
Christian

P.S.: I still do not get how to post formatted snippets with thunderbird 
to the newsgroup/forum :/
May 27 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/27/21 10:13 AM, Christian Köstlin wrote:

 P.S.: I still do not get how to post formatted snippets with thunderbird 
 to the newsgroup/forum :/


It's not possible currently.

I keep posting all my thunderbird posts *as if* the forum will highlight 
them in the hopes that some day it will retroactively work.

But I don't know... I can reformat the posts in my head, so it's not so bad.

-steve
May 27 2021
parent reply CandG <candg example.com> writes:
On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer 
wrote:

 On 5/27/21 10:13 AM, Christian Köstlin wrote:

 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/


 It's not possible currently.


I no longer use thunderbird, but:
  - 
https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac09bf40a933b851e5R338
  - IIRC thunderbird allows changing headers: try adding 
"Content-Type" to the comma-separated list 
"mail.compose.other.header"
  - Then in the composition window make sure Content-Type is set 
to something like "text/plain; markup=markdown"
May 31 2021
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-31 13:40, CandG wrote:

 On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:

 On 5/27/21 10:13 AM, Christian Köstlin wrote:

 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/


 It's not possible currently.


 
 I no longer use thunderbird, but:
   - 
 https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac
9bf40a933b851e5R338 
 
   - IIRC thunderbird allows changing headers: try adding "Content-Type" 
 to the comma-separated list "mail.compose.other.header"
   - Then in the composition window make sure Content-Type is set to 
 something like "text/plain; markup=markdown"


Thanks for the tip, lets see if it works:

```D
void main(string[] args) {
   writeln("Hello World");
}
```

Kind regards,
Christian
May 31 2021
next sibling parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-31 18:50, Christian Köstlin wrote:

 On 2021-05-31 13:40, CandG wrote:

 On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:

 On 5/27/21 10:13 AM, Christian Köstlin wrote:

 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/


 It's not possible currently.


 I no longer use thunderbird, but:
   - 
 https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac
9bf40a933b851e5R338 

   - IIRC thunderbird allows changing headers: try adding 
 "Content-Type" to the comma-separated list "mail.compose.other.header"
   - Then in the composition window make sure Content-Type is set to 
 something like "text/plain; markup=markdown"


 Thanks for the tip, lets see if it works:
 
 ```D
 void main(string[] args) {
    writeln("Hello World");
 }
 ```
 
 Kind regards,
 Christian
 


another try.

```D
void main(string[] args) {
    writeln("Hello World");
}
```
May 31 2021
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/21 12:50 PM, Christian Köstlin wrote:

 On 2021-05-31 13:40, CandG wrote:

 On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:

 On 5/27/21 10:13 AM, Christian Köstlin wrote:

 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/


 It's not possible currently.


 I no longer use thunderbird, but:
   - 
 https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac
9bf40a933b851e5R338 




That requires the "markup=markdown" to be in the Content-Type, which I 
(and several others) tried to modify, but Thunderbird doesn't allow it.


 Thanks for the tip, lets see if it works:
 
 ```D
 void main(string[] args) {
    writeln("Hello World");
 }
 ```


It doesn't work, thunderbird just wants to put its own Content-Type in 
there, and I don't think there's a way to change it.

If we had a way to use a custom header to do it, that would work. In 
fact, I bet it's not too difficult, I might make a PR for it (thanks for 
identifying the function that can do it)

-Steve
May 31 2021
prev sibling parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 13:11, sighoya wrote:

 On Thursday, 27 May 2021 at 09:58:40 UTC, Christian Köstlin wrote:

 I have this small program here

 test.d:
 ```
 import std;
 string doSomething(string[] servers, string user) {
     return user ~ servers[0];
 }
 void main() {
     auto servers = ["s1", "s2", "s3"];
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(user => servers.doSomething(user))(users));
 }
 ```


 
 
 
 I think it relates to https://issues.dlang.org/show_bug.cgi?id=5710
 
 The reason is that amap requires a this pointer of type TaskPool and a 
 context pointer to the closure which belongs to main, at least because 
 it requires servers. Having both isn't possible due to problems in non 
 DMD compilers.
 
 If you rewrite it more statically:
 ```D
 string doSomething(string[] servers, string user) {
      return user ~ servers[0];
 }
 
 string closure(string user)
 {
      return servers.doSomething(user);
 }
 auto servers = ["s1", "s2", "s3"];
 int main()
 {
      auto users = ["u1", "u2", "u3"];
      writeln(map!(user => servers.doSomething(user))(users));
      writeln(taskPool.amap!(closure)(users));
      return 0;
 }
 ```
 
 PS: Just enable markdown if you want to highlight D code


On a second not I needed to make server __gshared in my real program, as 
otherwise its a thread local variable (in the small demo program, this 
did not occur, I guess because the parallel operations we're too fast).

Kind regards,
Christian
May 27 2021
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/27/21 2:58 AM, Christian K=C3=B6stlin wrote:


      writeln(taskPool.amap!(user =3D> servers.doSomething(user))(users=


));

Luckily, parallel() is a free-standing function that does not require a=20
"this context". Is the following a workaround for you?

   auto result =3D new string[users.length];
   users.enumerate.parallel.each!(en =3D> result[en.index] =3D=20
servers.doSomething(en.value));
   writeln(result);

Ali
May 27 2021
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/27/21 9:19 AM, Ali =C3=87ehreli wrote:


  =C2=A0 auto result =3D new string[users.length];
  =C2=A0 users.enumerate.parallel.each!(en =3D> result[en.index] =3D=20
 servers.doSomething(en.value));
  =C2=A0 writeln(result);


I still like the foreach version more:

     auto result =3D new string[users.length];
     foreach (i, user; users.parallel) {
       result[i] =3D servers.doSomething(user);
     }
     writeln(result);

Ali
May 27 2021
parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 18:56, Ali Çehreli wrote:

 On 5/27/21 9:19 AM, Ali Çehreli wrote:
 

    auto result = new string[users.length];
    users.enumerate.parallel.each!(en => result[en.index] = 
 servers.doSomething(en.value));
    writeln(result);


 
 I still like the foreach version more:
 
      auto result = new string[users.length];
      foreach (i, user; users.parallel) {
        result[i] = servers.doSomething(user);
      }
      writeln(result);
 
 Ali
 


Hi Ali,

both of those variants do work for me, thanks a lot!
Still not sure which I prefer (almost too many options now :) ).

I am so happy that I asked in this forum, help is much appreciated!

Christian
May 27 2021