www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Removing some of the elements from vibe.core.concurrency.Future[]

reply kerdemdemir <kerdemdemir hotmail.com> writes:
I am trying to make non blocking web requests to a web service.

vibe.core.concurrency.Future!(UserData)[] futurelist;

// I will make http requests in for loop and push them to 
futureList
foreach( elem; elemList )
{
         // In makeWebRequest I make the httprequest, parse json 
and push to my UserData
         auto future = vibe.core.concurrency.async( 
&elem.makeWebRequest );
         futurelist ~= future;
}


// I want to do call ProcessResponceData for each future which is 
ready.
while ( futurelist.length > 0 )
{
         futurelist.filter!(a => a.ready()).each!(a=> 
ProcessResponceData(a.getResult()));

         //!!!!!  Here is my problem
         //!!!!!  I want to remove the futures which are already 
processed.
         futurelist = futurelist.filter!(a => !a.ready()).array;	
         sleep(10.msecs);

}

I am having troubles while trying to remove the future which I 
already processed.

futurelist = futurelist.filter!(a => !a.ready()).array;	 results 
with:

/usr/local/bin/../import/std/array.d(2716,20): Error: can't have 
array of nothrow  safe void()
/usr/local/bin/../import/std/array.d(2782,46): Error: can't have 
array of inout nothrow  safe void()
/usr/local/bin/../import/std/array.d(3158,1): Error: template 
instance std.array.Appender!(Future!(UserData)[]) error 
instantiating
/usr/local/bin/../import/std/array.d(133,32):        instantiated 
from here: appender!(Future!(UserData)[])

futurelist = futurelist.filter!(a => !a.ready());	 results with:

  Error: cannot implicitly convert expression (filter(futurelist)) 
of type FilterResult!(__lambda5, Future!(UserData)[]) to 
Future!(AnalyzeData)[]

Do you have any suggestion or workaround for my case?
What lesson I should learn from this case? Do you guys have any 
idea why I am getting those compile errors?


Thanks
Erdem
Oct 28
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 28 October 2017 at 13:51:42 UTC, kerdemdemir wrote:
 I am trying to make non blocking web requests to a web service.

 vibe.core.concurrency.Future!(UserData)[] futurelist;

 // I will make http requests in for loop and push them to 
 futureList
 foreach( elem; elemList )
 {
         // In makeWebRequest I make the httprequest, parse json 
 and push to my UserData
         auto future = vibe.core.concurrency.async( 
 &elem.makeWebRequest );
         futurelist ~= future;
 }


 // I want to do call ProcessResponceData for each future which 
 is ready.
 while ( futurelist.length > 0 )
 {
         futurelist.filter!(a => a.ready()).each!(a=> 
 ProcessResponceData(a.getResult()));

         //!!!!!  Here is my problem
         //!!!!!  I want to remove the futures which are already 
 processed.
         futurelist = futurelist.filter!(a => !a.ready()).array;	
         sleep(10.msecs);

 }

 I am having troubles while trying to remove the future which I 
 already processed.

 futurelist = futurelist.filter!(a => !a.ready()).array;	 
 results with:

 /usr/local/bin/../import/std/array.d(2716,20): Error: can't 
 have array of nothrow  safe void()
 /usr/local/bin/../import/std/array.d(2782,46): Error: can't 
 have array of inout nothrow  safe void()
 /usr/local/bin/../import/std/array.d(3158,1): Error: template 
 instance std.array.Appender!(Future!(UserData)[]) error 
 instantiating
 /usr/local/bin/../import/std/array.d(133,32):        
 instantiated from here: appender!(Future!(UserData)[])

 futurelist = futurelist.filter!(a => !a.ready());	 results with:

  Error: cannot implicitly convert expression 
 (filter(futurelist)) of type FilterResult!(__lambda5, 
 Future!(UserData)[]) to Future!(AnalyzeData)[]

 Do you have any suggestion or workaround for my case?
 What lesson I should learn from this case? Do you guys have any 
 idea why I am getting those compile errors?


 Thanks
 Erdem
I'd take a look at why the error message says `Future!(UserData)[]) to Future!(AnalyzeData)[]` is AnalyzeData the type returned by ProcessResponceData? Alternatively you could use a singly linked list and splice out elements that pass the filter predicate. I think you'd have to roll your own though.
Oct 28
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 29 October 2017 at 01:44:37 UTC, Nicholas Wilson wrote:
 Alternatively you could use a singly linked list and splice out 
 elements that pass the filter predicate. I think you'd have to 
 roll your own though.
Something like https://github.com/dlang/phobos/pull/5821
Oct 28
prev sibling parent kerdemdemir <kerdemdemir hotmail.com> writes:
 I'd take a look at why the error message says
 `Future!(UserData)[]) to Future!(AnalyzeData)[]`
 is AnalyzeData  the type returned by ProcessResponceData?

 Alternatively you could use a singly linked list and splice out 
 elements that pass the filter predicate. I think you'd have to 
 roll your own though.
I am sorry Wilson I normally in my code UserData is AnalyzeData = UserData but I replace the name to make it more understandable. For now I solved my problem like vibe.core.concurrency.Future!(UserData)[] futurelist; foreach( elem; elemList ) { auto future = vibe.core.concurrency.async( &elem.makeWebRequest ); futurelist ~= future; } int maximumSleepTime = 0; while(futurelist.any!(a => !a.ready())) //--> Blocking until all of them ready { maximumSleepTime++; sleep(10.msecs); if ( maximumSleepTime > 2000 ) return; } futurelist.each!(a=> Process(a.getResult())); Unfortunately that is really bad when I make 250 web requests and only one of them is bad(I mean not ready). And 249 request have to wait for the bad one. I will take a deeper look to data structure you suggested. Thanks
Oct 31