www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how to print progress of a long running parallel() loop?

reply mw <mingwu gmail.com> writes:
https://dlang.org/phobos/std_parallelism.html#.parallel

auto logs = new double[1_000_000];

foreach (i, ref elem; parallel(logs))
{
     elem = log(i + 1.0);
}


In Python, using joblib, I can use `verbose` level like this:

r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in 
range(10))

to print out the progress.

How to do this in D's parallel loop?

thanks.
Dec 07 2020
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:
 r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in 
 range(10))

 to print out the progress.

 How to do this in D's parallel loop?

 thanks.
Allocate a `shared int` before the foreach loop. In the loop when, let's say `!(i & 0xFFF)`, atomically increment the shared variable and print it's number. Disclaimer: no experience about using `shared`, but this is what I'd try.
Dec 07 2020
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/7/20 4:48 AM, Dukc wrote:
 On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:
 r =3D Parallel(n_jobs=3D2, verbose=3D10)(delayed(sleep)(.2) for _ in r=
ange(10))
 to print out the progress.

 How to do this in D's parallel loop?

 thanks.
=20 Allocate a `shared int` before the foreach loop. In the loop when, let'=
s=20
 say `!(i & 0xFFF)`,=C2=A0 atomically increment the shared variable and =
print=20
 it's number.
=20
 Disclaimer: no experience about using `shared`, but this is what I'd tr=
y. Yes, that would be the way but I don't know the effect of using shared=20 or not. But because the progress needs to be somehow reported, an atomic = variable may not be sufficient (unless a separate thread is reading it,=20 etc.) This issue came up during one of my recent presentations at this point: https://www.youtube.com/watch?v=3DdRORNQIB2wA&feature=3Dyoutu.be&t=3D1= 527 I used a synchronized block in that code because it did not affect=20 performance in my use case. Ali
Dec 07 2020