www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to work around the infamous dual-context when using delegates

reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
I have this small program here

test.d:
```
import std;
string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}
void main() {
     auto servers = ["s1", "s2", "s3"];
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(user => servers.doSomething(user))(users));
}
```

The first map just works as expected, for the parallel amap though fromo 
(https://dlang.org/phobos/std_parallelism.html) I get the following 
warning with dmd:

```
/Users/.../dlang/dmd-2.096.1/osx/bin/../../src/phobos/std/parallelism.d(1711): 
Deprecation: function `test.main.amap!(string[]).amap` function requires 
a dual-context, which is deprecated
```

for ldc the build fails with:
```
/Users/.../dlang/ldc-1.26.0/bin/../import/std/parallelism.d(1711): 
Deprecation: function `test.main.amap!(string[]).amap` function requires 
a dual-context, which is deprecated
test.d(9):        instantiated from here: `amap!(string[])`
/Users/.../dlang/ldc-1.26.0/bin/../import/std/parallelism.d(1711): 
Error: function `test.main.amap!(string[]).amap` requires a 
dual-context, which is not yet supported by LDC
```


Thanks in advance for you insights,
Christian
May 27
next sibling parent reply sighoya <sighoya gmail.com> writes:
On Thursday, 27 May 2021 at 09:58:40 UTC, Christian Köstlin wrote:
 I have this small program here

 test.d:
 ```
 import std;
 string doSomething(string[] servers, string user) {
     return user ~ servers[0];
 }
 void main() {
     auto servers = ["s1", "s2", "s3"];
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(user => 
 servers.doSomething(user))(users));
 }
 ```
I think it relates to https://issues.dlang.org/show_bug.cgi?id=5710 The reason is that amap requires a this pointer of type TaskPool and a context pointer to the closure which belongs to main, at least because it requires servers. Having both isn't possible due to problems in non DMD compilers. If you rewrite it more statically: ```D string doSomething(string[] servers, string user) { return user ~ servers[0]; } string closure(string user) { return servers.doSomething(user); } auto servers = ["s1", "s2", "s3"]; int main() { auto users = ["u1", "u2", "u3"]; writeln(map!(user => servers.doSomething(user))(users)); writeln(taskPool.amap!(closure)(users)); return 0; } ``` PS: Just enable markdown if you want to highlight D code
May 27
next sibling parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
Thanks for the proposed solution. It also works in my slightly bigger 
program (although I do not like to make servers more global).

I tried also the following (which unfortunately also does not work as 
intended):

```D
import std;
string doSomething(string[] servers, string user) {
     return user ~ servers[0];
}

int main()
{
     auto users = ["u1", "u2", "u3"];
     auto servers = ["s1", "s2", "s3"];
     auto usersWithServers = users.map!(user => tuple!("user", 
"servers")(user, servers)).array;
     writeln(map!(userWithServers => 
userWithServers.servers.doSomething(userWithServers.user))(usersWithServers));
     writeln(taskPool.amap!(userWithServers => 
userWithServers.servers.doSomething(userWithServers.user))(usersWithServers));
     return 0;
}
```

Here I try to put the data I need together into one tuple ("manually") 
and then pass it all to amap. Can you explain me, where here a double 
context is needed? Because all data now should be passed as arguments to 
amap?

Kind regards,
Christian
May 27
parent reply sighoya <sighoya gmail.com> writes:
On Thursday, 27 May 2021 at 12:17:36 UTC, Christian Köstlin wrote:
 Can you explain me, where here a double context is needed? 
 Because all data now should be passed as arguments to amap?

 Kind regards,
 Christian
I believe D's type system isn't smart enough to see independence between context and closure, otherwise your original example would also work as users and servers are context independent. What about: ```D string doSomething(string[] servers, string user) { return user ~ servers[0]; } void main() { static servers = ["s1", "s2", "s3"]; static users = ["u1", "u2", "u3"]; static lambda = (string user) => servers.doSomething(user); writeln(map!(user => servers.doSomething(user))(users)); writeln(taskPool.amap!(lambda)(users)); } ```
May 27
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 14:48, sighoya wrote:
 On Thursday, 27 May 2021 at 12:17:36 UTC, Christian Köstlin wrote:
 Can you explain me, where here a double context is needed? Because all 
 data now should be passed as arguments to amap?

 Kind regards,
 Christian
I  believe D's type system isn't smart enough to see independence between context and closure, otherwise your original example would also work as users and servers are context independent. What about: ```D string doSomething(string[] servers, string user) {     return user ~ servers[0]; } void main() {     static servers = ["s1", "s2", "s3"];     static users = ["u1", "u2", "u3"];     static lambda = (string user) => servers.doSomething(user);     writeln(map!(user => servers.doSomething(user))(users));     writeln(taskPool.amap!(lambda)(users)); } ```
That looks nice, but unfortunately my data for servers and users in the real world is not static but comes from a config file.
May 27
parent reply sighoya <sighoya gmail.com> writes:
On Thursday, 27 May 2021 at 12:58:28 UTC, Christian Köstlin wrote:

 That looks nice, but unfortunately my data for servers and 
 users in the real world is not static but comes from a config 
 file.
Okay, but then parametrizing the static lambda with runtime parameters should work. The important fact is that the closure needs to be static.
May 27
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 15:00, sighoya wrote:
 On Thursday, 27 May 2021 at 12:58:28 UTC, Christian Köstlin wrote:
 
 That looks nice, but unfortunately my data for servers and users in 
 the real world is not static but comes from a config file.
Okay, but then parametrizing the static lambda with runtime parameters should work. The important fact is that the closure needs to be static.
Ah thanks, now I understand. So what I came up with now is a combination of the things mentioned: ```D import std; string doSomething(string[] servers, string user) { return user ~ servers[0]; } struct UserWithServers { string user; string[] servers; } void main(string[] args) { auto servers = args; auto users = ["u1", "u2", "u3"]; auto usersWithServers = users.map!(user => UserWithServers(user, servers)).array; static fn = function(UserWithServers user) => user.servers.doSomething(user.user); writeln(taskPool.amap!(fn)(usersWithServers)); } ``` Making also the example a little bit more "realistic" by using dynamic data for servers. I would like to use auto fn, but somehow saying that its a function is not enough for dmd. From my understanding a function would never need a context?!? Thanks a lot! Christian P.S.: I still do not get how to post formatted snippets with thunderbird to the newsgroup/forum :/
May 27
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/27/21 10:13 AM, Christian Köstlin wrote:
 P.S.: I still do not get how to post formatted snippets with thunderbird 
 to the newsgroup/forum :/
It's not possible currently. I keep posting all my thunderbird posts *as if* the forum will highlight them in the hopes that some day it will retroactively work. But I don't know... I can reformat the posts in my head, so it's not so bad. -steve
May 27
parent reply CandG <candg example.com> writes:
On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer 
wrote:
 On 5/27/21 10:13 AM, Christian Köstlin wrote:
 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/
It's not possible currently.
I no longer use thunderbird, but: - https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac09bf40a933b851e5R338 - IIRC thunderbird allows changing headers: try adding "Content-Type" to the comma-separated list "mail.compose.other.header" - Then in the composition window make sure Content-Type is set to something like "text/plain; markup=markdown"
May 31
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-31 13:40, CandG wrote:
 On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:
 On 5/27/21 10:13 AM, Christian Köstlin wrote:
 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/
It's not possible currently.
I no longer use thunderbird, but:  - https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac 9bf40a933b851e5R338  - IIRC thunderbird allows changing headers: try adding "Content-Type" to the comma-separated list "mail.compose.other.header"  - Then in the composition window make sure Content-Type is set to something like "text/plain; markup=markdown"
Thanks for the tip, lets see if it works: ```D void main(string[] args) { writeln("Hello World"); } ``` Kind regards, Christian
May 31
next sibling parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-31 18:50, Christian Köstlin wrote:
 On 2021-05-31 13:40, CandG wrote:
 On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:
 On 5/27/21 10:13 AM, Christian Köstlin wrote:
 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/
It's not possible currently.
I no longer use thunderbird, but:   - https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac 9bf40a933b851e5R338   - IIRC thunderbird allows changing headers: try adding "Content-Type" to the comma-separated list "mail.compose.other.header"   - Then in the composition window make sure Content-Type is set to something like "text/plain; markup=markdown"
Thanks for the tip, lets see if it works: ```D void main(string[] args) {   writeln("Hello World"); } ``` Kind regards, Christian
another try. ```D void main(string[] args) { writeln("Hello World"); } ```
May 31
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/31/21 12:50 PM, Christian Köstlin wrote:
 On 2021-05-31 13:40, CandG wrote:
 On Thursday, 27 May 2021 at 14:44:29 UTC, Steven Schveighoffer wrote:
 On 5/27/21 10:13 AM, Christian Köstlin wrote:
 P.S.: I still do not get how to post formatted snippets with 
 thunderbird to the newsgroup/forum :/
It's not possible currently.
I no longer use thunderbird, but:   - https://github.com/CyberShadow/DFeed/commit/2e60edab2aedd173c7ea3712cb9500d90d4b795d#diff-0ecfc518dcbf670fdac54985dd56663a16a0806fd57a05ac 9bf40a933b851e5R338
That requires the "markup=markdown" to be in the Content-Type, which I (and several others) tried to modify, but Thunderbird doesn't allow it.
 Thanks for the tip, lets see if it works:
 
 ```D
 void main(string[] args) {
    writeln("Hello World");
 }
 ```
It doesn't work, thunderbird just wants to put its own Content-Type in there, and I don't think there's a way to change it. If we had a way to use a custom header to do it, that would work. In fact, I bet it's not too difficult, I might make a PR for it (thanks for identifying the function that can do it) -Steve
May 31
prev sibling parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 13:11, sighoya wrote:
 On Thursday, 27 May 2021 at 09:58:40 UTC, Christian Köstlin wrote:
 I have this small program here

 test.d:
 ```
 import std;
 string doSomething(string[] servers, string user) {
     return user ~ servers[0];
 }
 void main() {
     auto servers = ["s1", "s2", "s3"];
     auto users = ["u1", "u2", "u3"];
     writeln(map!(user => servers.doSomething(user))(users));
     writeln(taskPool.amap!(user => servers.doSomething(user))(users));
 }
 ```
I think it relates to https://issues.dlang.org/show_bug.cgi?id=5710 The reason is that amap requires a this pointer of type TaskPool and a context pointer to the closure which belongs to main, at least because it requires servers. Having both isn't possible due to problems in non DMD compilers. If you rewrite it more statically: ```D string doSomething(string[] servers, string user) {     return user ~ servers[0]; } string closure(string user) {     return servers.doSomething(user); } auto servers = ["s1", "s2", "s3"]; int main() {     auto users = ["u1", "u2", "u3"];     writeln(map!(user => servers.doSomething(user))(users));     writeln(taskPool.amap!(closure)(users));     return 0; } ``` PS: Just enable markdown if you want to highlight D code
On a second not I needed to make server __gshared in my real program, as otherwise its a thread local variable (in the small demo program, this did not occur, I guess because the parallel operations we're too fast). Kind regards, Christian
May 27
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/27/21 2:58 AM, Christian K=C3=B6stlin wrote:

      writeln(taskPool.amap!(user =3D> servers.doSomething(user))(users=
)); Luckily, parallel() is a free-standing function that does not require a=20 "this context". Is the following a workaround for you? auto result =3D new string[users.length]; users.enumerate.parallel.each!(en =3D> result[en.index] =3D=20 servers.doSomething(en.value)); writeln(result); Ali
May 27
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 5/27/21 9:19 AM, Ali =C3=87ehreli wrote:

  =C2=A0 auto result =3D new string[users.length];
  =C2=A0 users.enumerate.parallel.each!(en =3D> result[en.index] =3D=20
 servers.doSomething(en.value));
  =C2=A0 writeln(result);
I still like the foreach version more: auto result =3D new string[users.length]; foreach (i, user; users.parallel) { result[i] =3D servers.doSomething(user); } writeln(result); Ali
May 27
parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 2021-05-27 18:56, Ali Çehreli wrote:
 On 5/27/21 9:19 AM, Ali Çehreli wrote:
 
    auto result = new string[users.length];
    users.enumerate.parallel.each!(en => result[en.index] = 
 servers.doSomething(en.value));
    writeln(result);
I still like the foreach version more:     auto result = new string[users.length];     foreach (i, user; users.parallel) {       result[i] = servers.doSomething(user);     }     writeln(result); Ali
Hi Ali, both of those variants do work for me, thanks a lot! Still not sure which I prefer (almost too many options now :) ). I am so happy that I asked in this forum, help is much appreciated! Christian
May 27