www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why std_data_json is not default in phobos

reply aberba <karabutaworld gmail.com> writes:
I've taken a close look at the Api and its looks good. Its been 
around for a while. Why is it not yet in phobos?

https://code.dlang.org/packages/std_data_json
Feb 20 2018
next sibling parent aberba <karabutaworld gmail.com> writes:
On Tuesday, 20 February 2018 at 09:20:15 UTC, aberba wrote:

 I've taken a close look at the Api and its looks good. Its been 
 around for a while. Why is it not yet in phobos?

 https://code.dlang.org/packages/std_data_json


To add some context,  it was proposed in 2015 for review [1] and 
some insightful feedbacks were given in the thread. Along 
discussion. RC allocator seem to be in good shape in the 2.79 dmd 
release. Some naming conventions too.



[1] 
https://forum.dlang.org/thread/nbuhouhimowvcqssvfnr forum.dlang.org?page=1
Feb 20 2018
prev sibling next sibling parent reply bauss <jj_1337 live.dk> writes:
On Tuesday, 20 February 2018 at 09:20:15 UTC, aberba wrote:

 I've taken a close look at the Api and its looks good. Its been 
 around for a while. Why is it not yet in phobos?

 https://code.dlang.org/packages/std_data_json


As far as I remember it's because Andrei isn't satisfied with the 
implementation, leading to disagreements in the design of the 
module.

-- Someone correct me if I'm wrong.
Feb 20 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, February 20, 2018 11:55:54 bauss via Digitalmars-d wrote:

 On Tuesday, 20 February 2018 at 09:20:15 UTC, aberba wrote:

 I've taken a close look at the Api and its looks good. Its been
 around for a while. Why is it not yet in phobos?

 https://code.dlang.org/packages/std_data_json


 As far as I remember it's because Andrei isn't satisfied with the
 implementation, leading to disagreements in the design of the
 module.

 -- Someone correct me if I'm wrong.


It is my understanding that there were disagreements of what the design
should look like, so you're probably right, but regardless of the details,
it didn't get far enough in the Phobos review process to come to a vote, and
Sonke has not attempted to get it through the review process again. IIRC, I
talked to Sonke about it at last year's dconf and came away with the
impression that he'd given up on trying to get it into Phobos, but I don't
know. I get the impression that a lot of folks who do anything that might be
worth putting into Phobos don't want to go through the review process, and
as dub and code.dlang.org have become more prevalent, I think that the
motivation for folks to get any major piece of functionality into Phobos has
decreased. So, while we've gotten new functions here and there, there hasn't
been much in the way of new modules or seriouse rewrites of existing modules
being proposed for official review for a while now.

- Jonathan M Davis
Feb 20 2018
parent reply aberba <karabutaworld gmail.com> writes:
On Tuesday, 20 February 2018 at 12:10:35 UTC, Jonathan M Davis 
wrote:

 [...]


Hmm. I'm glad its available in dub. Not helpful to those not 
using dub though. Beginners too.
Feb 20 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, February 20, 2018 15:08:04 aberba via Digitalmars-d wrote:

 On Tuesday, 20 February 2018 at 12:10:35 UTC, Jonathan M Davis

 wrote:

 [...]


 Hmm. I'm glad its available in dub. Not helpful to those not
 using dub though. Beginners too.


Well, ideally, std.json would be replaced with something better, but it does
work, and there's always going to be stuff outside of the standard library.
Pretty much no matter what you're doing, at some point, you're either going
to have to use a third party library for something that you need or write it
yourself (usually, some combination of both). And there are plenty of
standard libraries that don't have any kind of JSON support.

However, as for avoiding dub, some folks do, but really, the D ecosystem as
a whole is going in the direction of using dub such that dub is _the_ way
that folks make libraries available. So, anyone refusing to use dub is going
to be hamstringing themselves in terms of having access to third party
libraries, and even if one thing that they need is in the standard library,
something else isn't going to be. It _is_ possible to use such libraries
without dub; it's just more of a pain, and whether it's worth the effort
depends on what you're doing. Hopefully, dub is improved so that those who
have issues with it will be able to use it, but as it stands, anyone who
wants to use 3rd party libraries in D without jumping through a bunch of
hoops is going to have to use to dub with all of its pros and cons.

And from the standpoint of a library developer, with dub and code.dlang.org,
they can just make their libraries available for others to use without
jumping through any of the hoops necessary to get it into Phobos. So, unless
they really want something in Phobos, it's just easier to make it available
on code.dlang.org and not bother with trying to get it into Phobos.

- Jonathan M Davis
Feb 20 2018
next sibling parent Chris <wendlec tcd.ie> writes:
On Tuesday, 20 February 2018 at 21:53:59 UTC, Jonathan M Davis 
wrote:

 On Tuesday, February 20, 2018 15:08:04 aberba via Digitalmars-d 
 wrote:



 And from the standpoint of a library developer, with dub and 
 code.dlang.org, they can just make their libraries available 
 for others to use without jumping through any of the hoops 
 necessary to get it into Phobos. So, unless they really want 
 something in Phobos, it's just easier to make it available on 
 code.dlang.org and not bother with trying to get it into Phobos.

 - Jonathan M Davis


I've noticed that things are going in that direction. Whether 
it's a good thing or a bad thing I can't tell, but I guess it's a 
natural development once a language is managed in a stricter and 
more consistent way. Maybe we should highlight the 3rd party libs 
a bit more so people get used to looking up things in 1. Phobos > 
2. code.dlang.org.
Feb 21 2018
prev sibling parent reply aberba <karabutaworld gmail.com> writes:
On Tuesday, 20 February 2018 at 21:53:59 UTC, Jonathan M Davis 
wrote:

 On Tuesday, February 20, 2018 15:08:04 aberba via Digitalmars-d 
 wrote:

 [...]


 Well, ideally, std.json would be replaced with something 
 better, but it does work, and there's always going to be stuff 
 outside of the standard library. Pretty much no matter what 
 you're doing, at some point, you're either going to have to use 
 a third party library for something that you need or write it 
 yourself (usually, some combination of both). And there are 
 plenty of standard libraries that don't have any kind of JSON 
 support.

 [...]


Out of curiosity, do you work on serverside related stuff at work 
or something. I noticed you were using Java. Why you developed 
dxml
Feb 21 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, February 22, 2018 07:24:30 aberba via Digitalmars-d wrote:

 On Tuesday, 20 February 2018 at 21:53:59 UTC, Jonathan M Davis

 wrote:

 On Tuesday, February 20, 2018 15:08:04 aberba via Digitalmars-d

 wrote:

 [...]


 Well, ideally, std.json would be replaced with something
 better, but it does work, and there's always going to be stuff
 outside of the standard library. Pretty much no matter what
 you're doing, at some point, you're either going to have to use
 a third party library for something that you need or write it
 yourself (usually, some combination of both). And there are
 plenty of standard libraries that don't have any kind of JSON
 support.

 [...]


 Out of curiosity, do you work on serverside related stuff at work
 or something. I noticed you were using Java.


I did use Java in previous jobs, but I don't curently use it. In recent
years, I've done far more with C++ and D, and my current job mostly involves
D. And I would almost certainly turn down a job that involved much in the
way of Java unless I were desperate. It's just too primitive to be a
pleasant language to program in.

Most of the code that I've done in my career has been server side, assuming
that there is any client or server involved at all. In particular, I have
done _very_ little with browser-based technologies like javascript. The
closest that I've really done to something that would be client-side would
be a client that viewed video that it pulled from an NVR over the LAN in a
video security system, and that was a desktop application written in C++. In
any project that involved a browser, I've always been working on the server
side of things.


 Why you developed dxml


XML makes a lot of sense for several projects that I have. I wanted to have
a good XML parser for those projects, and it was clear that that wasn't
going to materialize on its own.

For instance, one project that I have that I'd like to get back to is
creating wrappers for Qt5 so that it can be used from D. That would be far
too error-prone and time-consuming to do by hand, so it involves code
generation, and my solution for that was to create description files for the
Qt headers so that the necessary C/C++ and D code could be generated from
them. Any improvements to the code would therefore involve improving the
code generation rather than having to edit a ton of code by hand (Daniel
Murphy also suggested that I parse the C++ code like he did when porting dmd
from C++ to D, so I'll probably automate the generation of the description
files later, but I wouldn't want parsing C++ code to be part of building the
library, and at least some manual tweaks of the description files may be
required).

I had originally thought to do those description files in JSON, but they
would have been incredibly verbose and ugly that way in comparison to what I
could do in XML, because XML is far more free-form, whereas everything in
JSON pretty much has to be a value, an array, or an associative array.
Initially, I'd used std_experimental_xml, and it worked, but I had to work
around bugs in it and make changes to my local copy that never made it into
the upstream repo, because it wasn't maintained. So, there was no way that I
could release anything that way, and I needed an XML parser that was
actually maintained and worked without local fixes. And for better or worse,
that meant writing my own XML parser, which also delays any projects
requiring an XML parser, but sometimes, you have to spend time on one
project in order to have the tools you need to complete another. Heck, even
the Qt5 wrappers are in that boat. I'm doing them, because there are a
couple of applications that I'd like to write that need Qt, not just because
I like the idea of being able to use Qt from D.

Also, while I would have preferred to spend my time on projects other than
an XML parser, I do actually like writing parsers.

- Jonathan M Davis
Feb 21 2018
parent aberba <karabutaworld gmail.com> writes:
On Thursday, 22 February 2018 at 07:56:12 UTC, Jonathan M Davis 
wrote:

 On Thursday, February 22, 2018 07:24:30 aberba via 
 Digitalmars-d wrote:

 [...]


 I did use Java in previous jobs, but I don't curently use it. 
 In recent years, I've done far more with C++ and D, and my 
 current job mostly involves D. And I would almost certainly 
 turn down a job that involved much in the way of Java unless I 
 were desperate. It's just too primitive to be a pleasant 
 language to program in.

 [...]


Nice.
Feb 22 2018
prev sibling parent Seb <seb wilzba.ch> writes:
On Tuesday, 20 February 2018 at 15:08:04 UTC, aberba wrote:

 On Tuesday, 20 February 2018 at 12:10:35 UTC, Jonathan M Davis 
 wrote:

 [...]


 Hmm. I'm glad its available in dub. Not helpful to those not 
 using dub though. Beginners too.


Well, you could vote for this PR:

https://github.com/dlang/phobos/pull/6111

The idea is to improve the status quo by add least noting that 
the std.json is old and that there are better alternatives.

(and also e.g. https://github.com/dlang/phobos/pull/6116 for 
std.net.curl)
Feb 21 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 20 February 2018 at 09:20:15 UTC, aberba wrote:

 I've taken a close look at the Api and its looks good. Its been 
 around for a while. Why is it not yet in phobos?

 https://code.dlang.org/packages/std_data_json


A good reason is that a dmd regression broke it:

https://issues.dlang.org/show_bug.cgi?id=18267

I haven't been able to update from dmd 2.077.1 at work because of 
this.

Atila
Feb 22 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, February 22, 2018 09:35:06 Atila Neves via Digitalmars-d wrote:

 On Tuesday, 20 February 2018 at 09:20:15 UTC, aberba wrote:

 I've taken a close look at the Api and its looks good. Its been
 around for a while. Why is it not yet in phobos?

 https://code.dlang.org/packages/std_data_json


 A good reason is that a dmd regression broke it:

 https://issues.dlang.org/show_bug.cgi?id=18267

 I haven't been able to update from dmd 2.077.1 at work because of
 this.


Well, while that's definitely a problem, it really has nothing to do with
why it's not in Phobos (no attempt to get it into Phobos has occurred since
well before 2.077.1 was released), and if it _were_ in Phobos, then the
regression would have been caught by the autotester, and the code that
caused it would never have been merged - which is one advantage to having
something be in Phobos rather than on code.dlang.org.

- Jonathan M Davis
Feb 22 2018
parent reply JN <666total wp.pl> writes:
On Thursday, 22 February 2018 at 09:45:25 UTC, Jonathan M Davis 
wrote:

 Well, while that's definitely a problem, it really has nothing 
 to do with why it's not in Phobos (no attempt to get it into 
 Phobos has occurred since well before 2.077.1 was released), 
 and if it _were_ in Phobos, then the regression would have been 
 caught by the autotester, and the code that caused it would 
 never have been merged - which is one advantage to having 
 something be in Phobos rather than on code.dlang.org.

 - Jonathan M Davis


I believe it's a common practice in other languages to do 
something like testing the compiler against top 20 (or more) most 
popular packages, while I doubt this package would make it to top 
20 (do we even have such metrics from dub?), do you know if 
something like this is being done for D too?
Feb 22 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, February 22, 2018 10:32:34 JN via Digitalmars-d wrote:

 On Thursday, 22 February 2018 at 09:45:25 UTC, Jonathan M Davis

 wrote:

 Well, while that's definitely a problem, it really has nothing
 to do with why it's not in Phobos (no attempt to get it into
 Phobos has occurred since well before 2.077.1 was released),
 and if it _were_ in Phobos, then the regression would have been
 caught by the autotester, and the code that caused it would
 never have been merged - which is one advantage to having
 something be in Phobos rather than on code.dlang.org.

 - Jonathan M Davis


 I believe it's a common practice in other languages to do
 something like testing the compiler against top 20 (or more) most
 popular packages, while I doubt this package would make it to top
 20 (do we even have such metrics from dub?), do you know if
 something like this is being done for D too?


There is some testing that is done against a set of projects on
code.dlang.org, though I get the impression that it's not very reliable (it
frequently seems to fail due to reasons unrelated to the PRs being tested),
and for better or worse, it doesn't have to pass for something to be merged.
So, even if std_data_json is on the list of projects being tested, I'm not
sure that it would have helped much. I don't know if std_data_json is on the
list though.

CI stuff like that has been improving, but there's still work to be done on
that front.

- Jonathan M Davis
Feb 22 2018
parent Seb <seb wilzba.ch> writes:
On Thursday, 22 February 2018 at 10:40:11 UTC, Jonathan M Davis 
wrote:

 There is some testing that is done against a set of projects on 
 code.dlang.org, though I get the impression that it's not very 
 reliable (it frequently seems to fail due to reasons unrelated 
 to the PRs being tested), and for better or worse, it doesn't 
 have to pass for something to be merged.


It has to pass since a few weeks.
Feb 22 2018
prev sibling next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 22/02/2018 11:32 PM, JN wrote:

 On Thursday, 22 February 2018 at 09:45:25 UTC, Jonathan M Davis wrote:

 Well, while that's definitely a problem, it really has nothing to do 
 with why it's not in Phobos (no attempt to get it into Phobos has 
 occurred since well before 2.077.1 was released), and if it _were_ in 
 Phobos, then the regression would have been caught by the autotester, 
 and the code that caused it would never have been merged - which is 
 one advantage to having something be in Phobos rather than on 
 code.dlang.org.

 - Jonathan M Davis


 
 I believe it's a common practice in other languages to do something like 
 testing the compiler against top 20 (or more) most popular packages, 
 while I doubt this package would make it to top 20 (do we even have such 
 metrics from dub?), do you know if something like this is being done for 
 D too?


Yes http://code.dlang.org/api/packages/dump

	"stats": {
		"updatedAt": "2018-02-22T10:21:07.711Z",
		"downloads": {
			"total": 226,
			"monthly": 0,
			"weekly": 0,
			"daily": 0
		},
		"repo": {
			"stars": 7,
			"watchers": 2,
			"forks": 6,
			"issues": 1
		},
		"rating": 1.62909460067749
	},
Feb 22 2018
prev sibling parent Seb <seb wilzba.ch> writes:
On Thursday, 22 February 2018 at 10:32:34 UTC, JN wrote:

 I believe it's a common practice in other languages to do 
 something like testing the compiler against top 20 (or more) 
 most popular packages, while I doubt this package would make it 
 to top 20 (do we even have such metrics from dub?), do you know 
 if something like this is being done for D too?


We do even better - an example from a recent PR:

https://ci.dlang.io/blue/organizations/jenkins/dlang-org%2Fphobos/detail/PR-6207/1

It's ~40 projects.

(though it's really a PITA to test all these projects because (1) 
Jenkins is a horrible piece of software and (2) some of these 
projects have randomness (and thus random failures) in their 
testsuites.)
Feb 22 2018