www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - new XML and JSON libs and replacement of std.net.curl

reply Oleg B <code.viator gmail.com> writes:
Hello.
In std.xml docs I read that is deprecated, and std.net.curl can 
be deprecated too (not remember here I read about std.net.curl). 
About std.json I read what it's has slow (de)serialization.

What I must use at this time?
vibe.data.json has evolution 
https://github.com/s-ludwig/std_data_json, but, if I correctly 
understood, they are not compatible and I can't use 
stdx.data.json in vibe.

For XML I found this project 
https://github.com/lodo1995/experimental.xml. Is this really 
candidate to std, or author just called it as he want?

As replacement of std.net.curl I found 
https://github.com/ikod/dlang-requests. Who know's about this 
lib? Is this good replacement of std.net.curl?

Maybe if I need json and http requests I must fully use vibe for 
these things? Vibe not resolve xml question =(
I want to use minimal count of dependencies...
Aug 15 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/08/2016 3:01 AM, Oleg B wrote:

 Hello.
 In std.xml docs I read that is deprecated, and std.net.curl can be
 deprecated too (not remember here I read about std.net.curl). About
 std.json I read what it's has slow (de)serialization.

 What I must use at this time?
 vibe.data.json has evolution https://github.com/s-ludwig/std_data_json,
 but, if I correctly understood, they are not compatible and I can't use
 stdx.data.json in vibe.

 For XML I found this project
 https://github.com/lodo1995/experimental.xml. Is this really candidate
 to std, or author just called it as he want?


Official GSOC project so most likely will be accepted.


 As replacement of std.net.curl I found
 https://github.com/ikod/dlang-requests. Who know's about this lib? Is
 this good replacement of std.net.curl?


Nope, not a replacement.


 Maybe if I need json and http requests I must fully use vibe for these
 things? Vibe not resolve xml question =(
 I want to use minimal count of dependencies...


Basically they have been there for a while, expect them be there for a 
long time. So unless they don't work for you, may as well use them.
Aug 15 2016
parent reply Seb <seb wilzba.ch> writes:
On Monday, 15 August 2016 at 15:04:29 UTC, rikki cattermole wrote:

 On 16/08/2016 3:01 AM, Oleg B wrote:

 As replacement of std.net.curl I found
 https://github.com/ikod/dlang-requests. Who know's about this 
 lib? Is
 this good replacement of std.net.curl?


 Nope, not a replacement.


Why? I like it a lot more than std.net.curl (try to get the error 
code in std.net.curl) & the decision that etc (which wrap curl & 
zlib) needs to go has been made at Dconf, so I would love to see 
dlang-requests as std.experimental.http ;-)
Aug 15 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/08/2016 3:20 AM, Seb wrote:

 On Monday, 15 August 2016 at 15:04:29 UTC, rikki cattermole wrote:

 On 16/08/2016 3:01 AM, Oleg B wrote:

 As replacement of std.net.curl I found
 https://github.com/ikod/dlang-requests. Who know's about this lib? Is
 this good replacement of std.net.curl?


 Nope, not a replacement.


 Why? I like it a lot more than std.net.curl (try to get the error code
 in std.net.curl) & the decision that etc (which wrap curl & zlib) needs
 to go has been made at Dconf, so I would love to see dlang-requests as
 std.experimental.http ;-)


No matter how much I would like curl to go, it still requires somebody 
willing to do it. Keep in mind that it won't be able to support https 
without adding a dependency such as Botan into the mix.
A heck a lot of work.
Aug 15 2016
parent reply Seb <seb wilzba.ch> writes:
On Monday, 15 August 2016 at 15:25:22 UTC, rikki cattermole wrote:

 On 16/08/2016 3:20 AM, Seb wrote:

 On Monday, 15 August 2016 at 15:04:29 UTC, rikki cattermole 
 wrote:

 On 16/08/2016 3:01 AM, Oleg B wrote:

 As replacement of std.net.curl I found
 https://github.com/ikod/dlang-requests. Who know's about 
 this lib? Is
 this good replacement of std.net.curl?


 Nope, not a replacement.


 Why? I like it a lot more than std.net.curl (try to get the 
 error code
 in std.net.curl) & the decision that etc (which wrap curl & 
 zlib) needs
 to go has been made at Dconf, so I would love to see 
 dlang-requests as
 std.experimental.http ;-)


 No matter how much I would like curl to go, it still requires 
 somebody willing to do it. Keep in mind that it won't be able 
 to support https without adding a dependency such as Botan into 
 the mix.
 A heck a lot of work.


Oh I thought the author was just asking whether it is a good 
replacement for him personally.
Well before we worry about Botan, I think libasync needs to 
arrive first ;-)

https://github.com/etcimon/libasync

(it's from the same author as Botan)
Aug 15 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/08/2016 3:35 AM, Seb wrote:

 On Monday, 15 August 2016 at 15:25:22 UTC, rikki cattermole wrote:

 On 16/08/2016 3:20 AM, Seb wrote:

 On Monday, 15 August 2016 at 15:04:29 UTC, rikki cattermole wrote:

 On 16/08/2016 3:01 AM, Oleg B wrote:

 As replacement of std.net.curl I found
 https://github.com/ikod/dlang-requests. Who know's about this lib? Is
 this good replacement of std.net.curl?


 Nope, not a replacement.


 Why? I like it a lot more than std.net.curl (try to get the error code
 in std.net.curl) & the decision that etc (which wrap curl & zlib) needs
 to go has been made at Dconf, so I would love to see dlang-requests as
 std.experimental.http ;-)


 No matter how much I would like curl to go, it still requires somebody
 willing to do it. Keep in mind that it won't be able to support https
 without adding a dependency such as Botan into the mix.
 A heck a lot of work.


 Oh I thought the author was just asking whether it is a good replacement
 for him personally.
 Well before we worry about Botan, I think libasync needs to arrive first
 ;-)

 https://github.com/etcimon/libasync

 (it's from the same author as Botan)


My personal opinion of libasync is that while it is a great idea and is 
needed, the coding is pretty horrible.
So I started SPEW[0] which should handle windowing primarily (but also 
sockets, timers ext.).

[0] http://spew.cf/
Aug 15 2016
parent reply yawniek <dlang srtnwz.com> writes:
imo things should be modularized.

so there should be (fast) protocol parsers first, something like

https://github.com/h2o/picohttpparser or
https://github.com/seanmonstar/httparse

then a very simple eventloop that has abstractions and range 
based interfaces for reading/writing data into different types of 
sockets.

and at this point we need to start talking about Fibers and 
if/how they work with the eventloop.
and how to make async code nice in D.
Rust has a new approach that looks very promising: 
http://aturon.github.io/blog/2016/08/11/futures/

only then we can think about making http calls, using TLS and 
having a webserver.
as for TLS a dual approach might be needed anyway, because there 
people would probably want to be able to dynamically link to a 
system provided library such as OpenSSL.

=> in my opinion the groundwork (parsers, eventloop) etc should 
be DESIGNED thoroughly and we should not just take the best 
available library and stuff it into phobos.
and there is no shame to look at Go and rust, i still think D 
would have a good future as Language to implement Services.
Aug 16 2016
parent reply ikod <geller.garry gmail.com> writes:
On Tuesday, 16 August 2016 at 09:16:40 UTC, yawniek wrote:

 imo things should be modularized.

 so there should be (fast) protocol parsers first, something like

 https://github.com/h2o/picohttpparser or
 https://github.com/seanmonstar/httparse

 then a very simple eventloop that has abstractions and range 
 based interfaces for reading/writing data into different types 
 of sockets.


There is common http message parser that used in nginx and 
nodejs. I think it can be ported from C to D. Both sync and async 
code can be built around this parser. But async library accepted 
as standard need before this.
Aug 16 2016
parent yawniek <dlang srtnwz.com> writes:
On Tuesday, 16 August 2016 at 10:06:05 UTC, ikod wrote:

 On Tuesday, 16 August 2016 at 09:16:40 UTC, yawniek wrote:

 There is common http message parser that used in nginx and 
 nodejs. I think it can be ported from C to D.


that is pico, see:
https://github.com/nodejs/http-parser/pull/200
https://github.com/h2o/picohttpparser/issues/7


  But async library accepted as standard need before this.


i don't think there is any temporal dependency between an 
eventloop and a http parser making it into phobos.
only later then a simple http server or client needs both (+ TLS 
).
Aug 16 2016
prev sibling next sibling parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Monday, 15 August 2016 at 15:01:13 UTC, Oleg B wrote:

 Hello.
 In std.xml docs I read that is deprecated, [...]

 For XML I found this project 
 https://github.com/lodo1995/experimental.xml. Is this really 
 candidate to std, or author just called it as he want?


Hi!
I'm the developer of that XML library. It is still under heavy 
development, so if you decide to use it and find any problem, 
feel free to contact me on GitHub. Feedback from actual usage is 
very important to understand what needs more work. Thank you.

As a side note, the GSoC is almost over, so the iter for 
inclusion in Phobos should start soon.
Aug 15 2016
prev sibling next sibling parent ikod <geller.garry gmail.com> writes:
On Monday, 15 August 2016 at 15:01:13 UTC, Oleg B wrote:

 Hello.

 As replacement of std.net.curl I found 
 https://github.com/ikod/dlang-requests. Who know's about this 
 lib? Is this good replacement of std.net.curl?

 Maybe if I need json and http requests I must fully use vibe 
 for these things? Vibe not resolve xml question =(
 I want to use minimal count of dependencies...


Hello,

Can you use dlang-requests instead of curl, or not, depends on 
your requirements. If you find any bugs, you can post bug report 
to project, usually I react quickly. dlang-requests depend only 
on libssl/libcrypt.

Regards,

Igor
Aug 15 2016
prev sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Monday, August 15, 2016 15:01:13 Oleg B via Digitalmars-d-learn wrote:

 Hello.
 In std.xml docs I read that is deprecated, and std.net.curl can
 be deprecated too (not remember here I read about std.net.curl).
 About std.json I read what it's has slow (de)serialization.


std.json and std.xml are definitely going to get the axe at some point. It's
just a question of when we're going to have the replacements ready, and
they've been slow in coming - primarily because one or more people have to
actually do them, and they're a big undertaking. You can certainly use
std.json and std.xml in the interim, but be aware that at some point, they
will be deprecated, and you'll be forced to change your code to use
something else.


 What I must use at this time?
 vibe.data.json has evolution
 https://github.com/s-ludwig/std_data_json, but, if I correctly
 understood, they are not compatible and I can't use
 stdx.data.json in vibe.


std_data_json is the most likely candidate for replacing std.json. It did go
through the Phobos review process on some level, but it was decided at the
time that it wasn't ready (unfortunately, I wasn't pay a lot of attention to
it at the time, so I don't know the details). So, it will probably end up
replacing std.json eventually, but I don't know when, and I don't know how
much it will change before then. I have been using it for my projects, and
it works reasonably well. It's what I'd suggest that you use, but you can
choose to use std.json, and there are at least a couple of other projects on
code.dlang.org that handle JSON.

As to whether std_data_json is compatible with vibe.d, I don't know. It
looks like it uses the same package, so I would have thought that vibe.d
would just rely on it, but I don't know what Sonke is up to in that regard.


 For XML I found this project
 https://github.com/lodo1995/experimental.xml. Is this really
 candidate to std, or author just called it as he want?


That's an official GSOC project that is attempting to create a replacement
for std.xml. It's very much a work in progress, and even if it is accepted,
it's likely to go through a lot of changes before then. But as long as it's
versioned well enough to use without having to deal with constant breakage,
it would be valuable for you to use it and give feedback. If you don't want
to deal with that, you can still use std.xml for now (as horrible as it is),
and I think that there are at least a couple of other XML-related projects
on code.dlang.org.


 As replacement of std.net.curl I found
 https://github.com/ikod/dlang-requests. Who know's about this
 lib? Is this good replacement of std.net.curl?


I fully expect that std.json and std.xml will get replaced at some point.
std.net.curl is a bit more questionable. The main reason that a number of
folks want it gone is because it requires that you have an external, C
library (curl) installed, and it's caused issues as a result, though those
have more or less been resolved, and while Walter wants to get rid of it,
Andrei doesn't seem to agree, so it may be here to stay. And since curl does
stuff with SSL, and it's highly unlikely that we'd implement all of the
crypto stuff in D without relying on an external C library, if we tried to
get rid of std.net.curl, we'd either be trading one external library for
another, or we'd lose functionality in Phobos. But regardless, if it did get
removed, it would end up on code.dlang.org. While it does have various and
cons, it's not like it's fundamentally bad. It's just questionable as to
whether we want it in the standard library. So, if std.net.curl does what
you want, then just use it. Worst case, at some point you _might_ have to
change it so that your project pulls in std.net.curl as dub package instead
of just getting it from Phobos. But there are alternatives on code.dlang.org
that you can use if you'd prefer.

- Jonathan M Davis
Aug 15 2016