digitalmars.D.announce - dlang-requests 0.1.7 released
- ikod (12/12) Jun 11 2016 Hello,
- Suliman (1/1) Jun 11 2016 Voting for making requests replacement of curl!
- Alexander Milushev (2/15) Jun 12 2016 Hi, this is great project, but what about vibe.d compatibility?
- ikod (4/24) Jun 12 2016 This project doesn't depend on vibe.d, and I'm not sure if it can
- Alexander Milushev (3/12) Jun 13 2016 Yeah, I understood, but is it possible to add support of vibe.d
- ikod (7/21) Jun 13 2016 If you want to use requests from vibe.d application then this
- Andrei Alexandrescu (3/13) Jun 14 2016 Thanks! Does the project have a dub presence? How does it compare
- ikod (9/20) Jun 14 2016 ...
- Seb (6/27) Jun 14 2016 No worries - `std.net.curl` doesn't expose all the exotic
- ikod (41/50) Jun 17 2016 ...
- ikod (12/64) Jun 20 2016 Finally after all unnecessary memory movements were cleaned up,
- Zekereth (23/27) Jun 21 2016 I have a couple of questions. If a url can't be found and I catch
- ikod (6/34) Jun 22 2016 Hello,
Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance. It provide interfaces for HTTP(S) and FTP requests. You can tune request details, but for most cases you will need single and simple API call. Latest release added streaming content from server and some performance improvements (avoiding some of unnecessary buffer copying). Code and docs available at https://github.com/ikod/dlang-requests or as dub package. Thanks Dlang authors and community for excellent language.
Jun 11 2016
Suliman <evermind live.ru> 
On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance. It provide interfaces for HTTP(S) and FTP requests. You can tune request details, but for most cases you will need single and simple API call. Latest release added streaming content from server and some performance improvements (avoiding some of unnecessary buffer copying). Code and docs available at https://github.com/ikod/dlang-requests or as dub package. Thanks Dlang authors and community for excellent language.Hi, this is great project, but what about vibe.d compatibility?
Jun 12 2016
On Sunday, 12 June 2016 at 22:37:34 UTC, Alexander Milushev wrote:On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:This project doesn't depend on vibe.d, and I'm not sure if it can be directly used from inside vibe.d application, as it use blocked socket io.Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance. It provide interfaces for HTTP(S) and FTP requests. You can tune request details, but for most cases you will need single and simple API call. Latest release added streaming content from server and some performance improvements (avoiding some of unnecessary buffer copying). Code and docs available at https://github.com/ikod/dlang-requests or as dub package. Thanks Dlang authors and community for excellent language.Hi, this is great project, but what about vibe.d compatibility?
Jun 12 2016
On Monday, 13 June 2016 at 00:09:12 UTC, ikod wrote:On Sunday, 12 June 2016 at 22:37:34 UTC, Alexander Milushev wrote:Yeah, I understood, but is it possible to add support of vibe.d sockets too?On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:This project doesn't depend on vibe.d, and I'm not sure if it can be directly used from inside vibe.d application, as it use blocked socket io.[...]Hi, this is great project, but what about vibe.d compatibility?
Jun 13 2016
On Monday, 13 June 2016 at 08:29:28 UTC, Alexander Milushev wrote:On Monday, 13 June 2016 at 00:09:12 UTC, ikod wrote:If you want to use requests from vibe.d application then this probably can be done using https://github.com/rejectedsoftware/vibe.d/wiki#integrating-non-vibed-io But this require writing some wrapper. I can investigate this in the future. Thanks!On Sunday, 12 June 2016 at 22:37:34 UTC, Alexander Milushev wrote:Yeah, I understood, but is it possible to add support of vibe.d sockets too?On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:This project doesn't depend on vibe.d, and I'm not sure if it can be directly used from inside vibe.d application, as it use blocked socket io.[...]Hi, this is great project, but what about vibe.d compatibility?
Jun 13 2016
On 6/11/16 7:03 PM, ikod wrote:Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance. It provide interfaces for HTTP(S) and FTP requests. You can tune request details, but for most cases you will need single and simple API call. Latest release added streaming content from server and some performance improvements (avoiding some of unnecessary buffer copying). Code and docs available at https://github.com/ikod/dlang-requests or as dub package. Thanks Dlang authors and community for excellent language.Thanks! Does the project have a dub presence? How does it compare feature-wise and speed-wise with curl? -- Andrei
Jun 14 2016
On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu wrote:On 6/11/16 7:03 PM, ikod wrote:...Hello, Dlang-requests is library created under influence ofYes, it is in the dub repository under name "requests" As for speed comparison, I'll post numbers in a day or two, when I'll finish some performance improvements. Feature matrix for libcurl(per https://curl.haxx.se/docs/features.html) and dlang-requests: https://github.com/ikod/dlang-requests/blob/master/docs/matrix.txt Please note, that dlang-requests supports only http,https and ftp.Code and docs available at https://github.com/ikod/dlang-requests or as dub package. Thanks Dlang authors and community for excellent language.Thanks! Does the project have a dub presence? How does it compare feature-wise and speed-wise with curl? -- Andrei
Jun 14 2016
On Tuesday, 14 June 2016 at 22:46:00 UTC, ikod wrote:On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu wrote:No worries - `std.net.curl` doesn't expose all the exotic protocols that libcurl supports either ;-) Same goes for the features, it's more interestingly to see what std.net.curl exposes to the user: http://dlang.org/phobos/std_net_curl.htmlOn 6/11/16 7:03 PM, ikod wrote:...Hello, Dlang-requests is library created under influence ofYes, it is in the dub repository under name "requests" As for speed comparison, I'll post numbers in a day or two, when I'll finish some performance improvements. Feature matrix for libcurl(per https://curl.haxx.se/docs/features.html) and dlang-requests: https://github.com/ikod/dlang-requests/blob/master/docs/matrix.txt Please note, that dlang-requests supports only http,https and ftp.Code and docs available at https://github.com/ikod/dlang-requests or as dub package. Thanks Dlang authors and community for excellent language.Thanks! Does the project have a dub presence? How does it compare feature-wise and speed-wise with curl? -- Andrei
Jun 14 2016
On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu wrote:On 6/11/16 7:03 PM, ikod wrote:...Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance.Thanks! Does the project have a dub presence? How does it compare feature-wise and speed-wise with curl? -- AndreiHello, Finally, I made some improvements and run minimal performance tests against command-line curl. I wrote simple code for file download using dlang-requests, run it and curl for the same urls(httpbin server on my notebook) and compare "total", "system", and "user" time for different cases. You can find numbers and code below. So my conclusion is - performances are comparable for these cases, but there is some field for improvement in dlang-requests. Case 1 - 50Mb of random data, no any encoding Case 2 - 50Mb of random data, transfer chunked Case 3 - 50Mb of random data, transfer chunked, content gzip measured times, sec ----------------------------------------- | user | system | total Case|-----------|-----------|----------- | d-r | curl| d-r| curl| d-r | curl -----|-----|-----|-----|-----|-----|----- 1 | 0.17| 0.14| 0.20| 0.32| 51.7| 52.2 2 | 0.19| 0.11| 0.15| 0.21| 51.8| 51.9 3 | 0.21| 0.15| 0.11| 0.15| 51.5| 52.1 import std.stdio; import requests; pragma(lib, "ssl"); pragma(lib, "crypto"); void main() { auto sink = File("/dev/null", "wb"); auto rq = Request(); rq.useStreaming = true; auto rs = rq.get("http://127.0.0.1:8080/stream-bytes/51200000"); auto stream = rs.receiveAsRange(); while(!stream.empty) { sink.rawWrite(stream.front); stream.popFront; } }
Jun 17 2016
On Friday, 17 June 2016 at 17:31:55 UTC, ikod wrote:On Tuesday, 14 June 2016 at 14:59:37 UTC, Andrei Alexandrescu wrote:Finally after all unnecessary memory movements were cleaned up, results for dlang-requests and curl almost equals. measured times, sec ----------------------------------------- | user | system | total Case|-----------|-----------|----------- | d-r | curl| d-r| curl| d-r | curl -----|-----|-----|-----|-----|-----|----- 1 | 0.14| 0.14| 0.15| 0.32| 53.3| 52.2 2 | 0.13| 0.11| 0.11| 0.21| 51.1| 51.9 3 | 0.18| 0.15| 0.08| 0.15| 51.2| 52.1On 6/11/16 7:03 PM, ikod wrote:...Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance.Thanks! Does the project have a dub presence? How does it compare feature-wise and speed-wise with curl? -- AndreiHello, Finally, I made some improvements and run minimal performance tests against command-line curl. I wrote simple code for file download using dlang-requests, run it and curl for the same urls(httpbin server on my notebook) and compare "total", "system", and "user" time for different cases. You can find numbers and code below. So my conclusion is - performances are comparable for these cases, but there is some field for improvement in dlang-requests. Case 1 - 50Mb of random data, no any encoding Case 2 - 50Mb of random data, transfer chunked Case 3 - 50Mb of random data, transfer chunked, content gzip measured times, sec ----------------------------------------- | user | system | total Case|-----------|-----------|----------- | d-r | curl| d-r| curl| d-r | curl -----|-----|-----|-----|-----|-----|----- 1 | 0.17| 0.14| 0.20| 0.32| 51.7| 52.2 2 | 0.19| 0.11| 0.15| 0.21| 51.8| 51.9 3 | 0.21| 0.15| 0.11| 0.15| 51.5| 52.1import std.stdio; import requests; pragma(lib, "ssl"); pragma(lib, "crypto"); void main() { auto sink = File("/dev/null", "wb"); auto rq = Request(); rq.useStreaming = true; auto rs = rq.get("http://127.0.0.1:8080/stream-bytes/51200000"); auto stream = rs.receiveAsRange(); while(!stream.empty) { sink.rawWrite(stream.front); stream.popFront; } }
Jun 20 2016
On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance.I have a couple of questions. If a url can't be found and I catch the ConnectError exception and handle it[2] I still get an additional error[1] printed. For example: try { getContent(url); } catch(ConnectError ex) { debug writeln("Error getting page: ", ex.msg); } Will error with this: [1]2016-06-21T20:59:15.073:streams.d:connect:750 Failed to connect: can't resolve www.systemcontrolpanel.com - getaddrinfo error: Name or service not known [2]Error getting page: Can't connect to www.systemcontrolpanel.com:80: getaddrinfo error: Name or service not known The additional error[1] will still print in release build. The only workaround I could find was to check getAddress myself and ensure that the page exists before calling getContent. Thanks!
Jun 21 2016
On Wednesday, 22 June 2016 at 02:04:25 UTC, Zekereth wrote:On Saturday, 11 June 2016 at 23:03:52 UTC, ikod wrote:Hello, It makes no sense to log error message and throw exception in next line, so I'll remove this error messages from the code. Thanks for report(but better place it on the github project page so that I can link code changes with bug reports)!Hello, Dlang-requests is library created under influence of Python-requests, with primary goal of easy to use and performance.I have a couple of questions. If a url can't be found and I catch the ConnectError exception and handle it[2] I still get an additional error[1] printed. For example: try { getContent(url); } catch(ConnectError ex) { debug writeln("Error getting page: ", ex.msg); } Will error with this: [1]2016-06-21T20:59:15.073:streams.d:connect:750 Failed to connect: can't resolve www.systemcontrolpanel.com - getaddrinfo error: Name or service not known [2]Error getting page: Can't connect to www.systemcontrolpanel.com:80: getaddrinfo error: Name or service not known The additional error[1] will still print in release build. The only workaround I could find was to check getAddress myself and ensure that the page exists before calling getContent. Thanks!
Jun 22 2016