digitalmars.D.learn - how to print progress of a long running parallel() loop?
- mw (12/12) Dec 07 2020 https://dlang.org/phobos/std_parallelism.html#.parallel
- Dukc (6/11) Dec 07 2020 Allocate a `shared int` before the foreach loop. In the loop
- =?UTF-8?Q?Ali_=c3=87ehreli?= (15/29) Dec 07 2020 s=20
https://dlang.org/phobos/std_parallelism.html#.parallel auto logs = new double[1_000_000]; foreach (i, ref elem; parallel(logs)) { elem = log(i + 1.0); } In Python, using joblib, I can use `verbose` level like this: r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in range(10)) to print out the progress. How to do this in D's parallel loop? thanks.
Dec 07 2020
On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:r = Parallel(n_jobs=2, verbose=10)(delayed(sleep)(.2) for _ in range(10)) to print out the progress. How to do this in D's parallel loop? thanks.Allocate a `shared int` before the foreach loop. In the loop when, let's say `!(i & 0xFFF)`, atomically increment the shared variable and print it's number. Disclaimer: no experience about using `shared`, but this is what I'd try.
Dec 07 2020
On 12/7/20 4:48 AM, Dukc wrote:On Monday, 7 December 2020 at 08:16:50 UTC, mw wrote:ange(10))r =3D Parallel(n_jobs=3D2, verbose=3D10)(delayed(sleep)(.2) for _ in r=s=20to print out the progress. How to do this in D's parallel loop? thanks.=20 Allocate a `shared int` before the foreach loop. In the loop when, let'=say `!(i & 0xFFF)`,=C2=A0 atomically increment the shared variable and =print=20it's number. =20 Disclaimer: no experience about using `shared`, but this is what I'd tr=y. Yes, that would be the way but I don't know the effect of using shared=20 or not. But because the progress needs to be somehow reported, an atomic = variable may not be sufficient (unless a separate thread is reading it,=20 etc.) This issue came up during one of my recent presentations at this point: https://www.youtube.com/watch?v=3DdRORNQIB2wA&feature=3Dyoutu.be&t=3D1= 527 I used a synchronized block in that code because it did not affect=20 performance in my use case. Ali
Dec 07 2020