www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Curl, how to recieve data.

reply holo <holosian gmail.com> writes:
I'm trying to receive data from curl request my sample code looks 
like that:

...
             auto client = HTTP(endpoint ~ "?" ~ 
canonicalQueryString);
             client.method = HTTP.Method.get;
             client.addRequestHeader("x-amz-date", xamztime);
             client.addRequestHeader("Authorization", 
authorizationHeader);
             client.onReceive = (ubyte[] data)
             {
                 recievedData = data;
                 return data.length;
             };
             client.perform();

             return new Result(recievedData);
...

             ubyte[] receivedData;

...

but im getting only last (like from "tail" command in unix 
systems) part of data which im expecting.

How to receive and save whole data which came as feedback for 
request? Or is there some other way to access it after usage of 
client.perform method?
Oct 18 2015
next sibling parent reply sigod <sigod.mail gmail.com> writes:
On Sunday, 18 October 2015 at 18:04:53 UTC, holo wrote:
 I'm trying to receive data from curl request my sample code 
 looks like that:

 ...
             auto client = HTTP(endpoint ~ "?" ~ 
 canonicalQueryString);
             client.method = HTTP.Method.get;
             client.addRequestHeader("x-amz-date", xamztime);
             client.addRequestHeader("Authorization", 
 authorizationHeader);
             client.onReceive = (ubyte[] data)
             {
                 recievedData = data;
                 return data.length;
             };
             client.perform();

             return new Result(recievedData);
 ...

             ubyte[] receivedData;

 ...

 but im getting only last (like from "tail" command in unix 
 systems) part of data which im expecting.

 How to receive and save whole data which came as feedback for 
 request? Or is there some other way to access it after usage of 
 client.perform method?
I believe `onReceive` called multiple times with chunks of received data. So, you need to use `Appender` or `~`. A bit off-topic: Are you trying to download file from S3? It seems I should really start working on my S3 library...
Oct 18 2015
parent reply holo <holosian gmail.com> writes:
 sigod

Actually im working on ec2 requests. Thank you  for help, it is 
working right now. I don't know why i was trying "+=" before 
instead of "~=". Is it good solution to make it such way?

 Suliaman

I need to collect information about my instances and put it to 
DB. I want to present those datas with with vibe.d, but collector 
want to make without too many dependencies (i was thinking to 
make it as separate thread but simplest solution will be just to 
use cron to run collector i think)
Oct 18 2015
parent reply sigod <sigod.mail gmail.com> writes:
On Sunday, 18 October 2015 at 20:05:24 UTC, holo wrote:
  sigod

 Actually im working on ec2 requests. Thank you  for help, it is 
 working right now. I don't know why i was trying "+=" before 
 instead of "~=". Is it good solution to make it such way?
Not really as it will trigger allocation on every call. Better use [`Appender`][0]. [0]: http://dlang.org/phobos/std_array.html#.Appender
Oct 18 2015
parent reply holo <holosian gmail.com> writes:
On Sunday, 18 October 2015 at 20:12:42 UTC, sigod wrote:
 On Sunday, 18 October 2015 at 20:05:24 UTC, holo wrote:
  sigod

 Actually im working on ec2 requests. Thank you  for help, it 
 is working right now. I don't know why i was trying "+=" 
 before instead of "~=". Is it good solution to make it such 
 way?
Not really as it will trigger allocation on every call. Better use [`Appender`][0]. [0]: http://dlang.org/phobos/std_array.html#.Appender
I changed it to such code: ... auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString); client.method = HTTP.Method.get; client.addRequestHeader("x-amz-date", xamztime); client.addRequestHeader("Authorization", authorizationHeader); client.onReceive = (ubyte[] data){receivedData.put(data); return data.length;}; client.perform(); return new Result(receivedData.data); ... auto receivedData = appender!string(); ... and it is really much more faster than it was - hope it is what you had on your mind.
Oct 18 2015
parent reply sigod <sigod.mail gmail.com> writes:
On Sunday, 18 October 2015 at 21:01:05 UTC, holo wrote:
 On Sunday, 18 October 2015 at 20:12:42 UTC, sigod wrote:
 [...]
I changed it to such code: ... auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString); client.method = HTTP.Method.get; client.addRequestHeader("x-amz-date", xamztime); client.addRequestHeader("Authorization", authorizationHeader); client.onReceive = (ubyte[] data){receivedData.put(data); return data.length;}; client.perform(); return new Result(receivedData.data); ... auto receivedData = appender!string(); ... and it is really much more faster than it was - hope it is what you had on your mind.
Yes, this is exactly what I meant.
Oct 18 2015
parent holo <holosian gmail.com> writes:
On Sunday, 18 October 2015 at 21:11:58 UTC, sigod wrote:
 On Sunday, 18 October 2015 at 21:01:05 UTC, holo wrote:
 On Sunday, 18 October 2015 at 20:12:42 UTC, sigod wrote:
 [...]
I changed it to such code: ... auto client = HTTP(endpoint ~ "?" ~ canonicalQueryString); client.method = HTTP.Method.get; client.addRequestHeader("x-amz-date", xamztime); client.addRequestHeader("Authorization", authorizationHeader); client.onReceive = (ubyte[] data){receivedData.put(data); return data.length;}; client.perform(); return new Result(receivedData.data); ... auto receivedData = appender!string(); ... and it is really much more faster than it was - hope it is what you had on your mind.
Yes, this is exactly what I meant.
Thank you for help.
Oct 18 2015
prev sibling parent Suliman <evermind live.ru> writes:
On Sunday, 18 October 2015 at 18:04:53 UTC, holo wrote:
 I'm trying to receive data from curl request my sample code 
 looks like that:

 ...
             auto client = HTTP(endpoint ~ "?" ~ 
 canonicalQueryString);
             client.method = HTTP.Method.get;
             client.addRequestHeader("x-amz-date", xamztime);
             client.addRequestHeader("Authorization", 
 authorizationHeader);
             client.onReceive = (ubyte[] data)
             {
                 recievedData = data;
                 return data.length;
             };
             client.perform();

             return new Result(recievedData);
 ...

             ubyte[] receivedData;

 ...

 but im getting only last (like from "tail" command in unix 
 systems) part of data which im expecting.

 How to receive and save whole data which came as feedback for 
 request? Or is there some other way to access it after usage of 
 client.perform method?
You also may try to look at http://vibed.org/api/vibe.http.client/requestHTTP curl maybe a little bit hard in use.
Oct 18 2015