www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - parseJSON bug

reply "khurshid" <khurshid.normuradov gmail.com> writes:
std.json  parseJSON has a bug,

//                         leading whitespaces
auto json = parseJSON("1                                  .000");

-- should throws, but it's success parsed as `1.000` .


Khurshid...
Aug 08 2013
next sibling parent reply "khurshid" <khurshid.normuradov gmail.com> writes:
On Thursday, 8 August 2013 at 12:08:00 UTC, khurshid wrote:
 std.json  parseJSON has a bug,

 //                         leading whitespaces
 auto json = parseJSON("1                                  
 .000");

 -- should throws, but it's success parsed as `1.000` .


 Khurshid...

Or more... auto json = parseJSON(`{ "one": 1 .24E +1 }`);//should throws. writeln(toJSON(&json)); // printed `{"one":12.4}`
Aug 08 2013
parent David <d dav1d.de> writes:
Am 08.08.2013 14:15, schrieb khurshid:
 On Thursday, 8 August 2013 at 12:08:00 UTC, khurshid wrote:
 std.json  parseJSON has a bug,

 //                         leading whitespaces
 auto json = parseJSON("1                                  .000");

 -- should throws, but it's success parsed as `1.000` .


 Khurshid...

Or more... auto json = parseJSON(`{ "one": 1 .24E +1 }`);//should throws. writeln(toJSON(&json)); // printed `{"one":12.4}`

Why is this a bug? JSON ignores whitespaces outside strings.
Aug 08 2013
prev sibling next sibling parent "SteveGuo" <steveguo outlook.com> writes:
On Thursday, 8 August 2013 at 12:08:00 UTC, khurshid wrote:
 std.json  parseJSON has a bug,

 //                         leading whitespaces
 auto json = parseJSON("1                                  
 .000");

 -- should throws, but it's success parsed as `1.000` .


 Khurshid...

I think bug reports should be here http://forum.dlang.org/group/digitalmars.D.bugs
Aug 08 2013
prev sibling next sibling parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Thursday, 8 August 2013 at 12:08:00 UTC, khurshid wrote:
 std.json  parseJSON has a bug,

 //                         leading whitespaces
 auto json = parseJSON("1                                  
 .000");

 -- should throws, but it's success parsed as `1.000` .


 Khurshid...

I don't think this is a bug, the json spec seems to indicate that this is valid.
Aug 08 2013
next sibling parent reply David <d dav1d.de> writes:
Am 08.08.2013 14:54, schrieb bearophile:
 Tofu Ninja:
 
 I don't think this is a bug, the json spec seems to indicate that this
 is valid.

The JSON decode of the Python2.6.6 standard library seems to refuse it:
 import json
 json.loads("1.000")



 json.loads("1                                  .000")



File "<stdin>", line 1, in <module> File "...\Python26\lib\json\__init__.py", line 307, in loads return _default_decoder.decode(s) File "...\Python26\lib\json\decoder.py", line 322, in decode raise ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 1 column 35 - line 1 column 39 (char 35 - 39) Bye, bearophile

JSON.parse("1 .000") SyntaxError: Unexpected token . If we follow an existing implementation, we should follow JavaScript
Aug 08 2013
next sibling parent reply David <d dav1d.de> writes:
 The real question is, is this worth fixing before std.serialize makes it
 in? There will likely be other bugs once/if that's accepted. I tried
 extending std.json in the past with static reflection, but that didn't
 make it in for this very reason.

Yep no one seems to care about std.json.
Aug 08 2013
parent reply David <d dav1d.de> writes:
Am 08.08.2013 18:26, schrieb Lemonfiend:
 On Thursday, 8 August 2013 at 15:15:44 UTC, David wrote:
 The real question is, is this worth fixing before std.serialize makes it
 in? There will likely be other bugs once/if that's accepted. I tried
 extending std.json in the past with static reflection, but that didn't
 make it in for this very reason.

Yep no one seems to care about std.json.

I heard about std.xml being crap and was going to use std.json, then I ran into this thread. So what should one use to parse json, or xml for that matter?

std.json is Ok, the API is a real pain. I made a pull request improving the API a few weeks ago, no one seems to really care. Better use a 3rd party implementation or roll your own (what seems to be Ds mentallity)
Aug 08 2013
parent reply David <d dav1d.de> writes:
Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...
Aug 08 2013
next sibling parent David <d dav1d.de> writes:
Am 08.08.2013 22:15, schrieb Tofu Ninja:
 On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
 Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...

It is really bad that people are actually talking about starting there own standard lib, I wasn't around for the whole phobos vs tango thing but from what I hear, it wasn't pretty. If there is problems with phobos or the way its managed, I feel like we should try and fix them and not try to replace it.

phobos is actually useable, still I don't like many of its design choices. I feel the same with D. D itself is better than most languages, I still think there is a long way to go, but "official" D people seem like it's enough and block progress (e.g. property/-property the whole situation is a mess, but instead of properly reimplementing it, they try to fix an already broken system and break code with it, why not break code once, but replace it with a proper system)
Aug 08 2013
prev sibling parent David <d dav1d.de> writes:
Am 09.08.2013 00:19, schrieb Johannes Pfau:
 Am Thu, 08 Aug 2013 22:15:28 +0200
 schrieb "Tofu Ninja" <emmons0 purdue.edu>:
 
 On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
 Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...

It is really bad that people are actually talking about starting there own standard lib, I wasn't around for the whole phobos vs tango thing but from what I hear, it wasn't pretty. If there is problems with phobos or the way its managed, I feel like we should try and fix them and not try to replace it.

I'm sorry that you've had such bad experiences when contributing to phobos. I think this is less of a phobos than a std.json problem. I'd like to explain the special issue we have with std.json - as far as I understand it:

I think the problem is, people are afraid that changes will hurt D. The mentallity to improve D and make it awsome is lost (at least if feels like it), now all what is left "WE NEED TO MAKE D PRODUCTION READY!!!1111111 COMPANIES USE D!!!!1111" which makes sense, but one shouldn't forget why D was "invented", to improve the programming situation and craft a language which is superior to all others and which allows to write simple clean but yet powerful code!
Aug 08 2013
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Aug 08, 2013 at 10:15:28PM +0200, Tofu Ninja wrote:
 On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
I made a pull request improving the API a few weeks ago,
no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...

It is really bad that people are actually talking about starting there own standard lib, I wasn't around for the whole phobos vs tango thing but from what I hear, it wasn't pretty. If there is problems with phobos or the way its managed, I feel like we should try and fix them and not try to replace it.

Yeah, another Phobos/Tango (or whatever it may be) split is probably going to be the thing that finally kills off D before it ever reaches widespread adoption. Please, let's not go there. Here's another approach. If you feel something is wrong with Phobos and your pulls aren't getting accepted for whatever reason, why not package it up nicely in a form that's easily installable by users, and publish it? Say you have a replacement for std.json that you feel is far superior, well, then make it installable as a standalone library (say alt.json), and promote it here and get many people to start to use it. Once it's widely-used, I suspect it will be far easier to convince people to put it into Phobos. :-) T -- "I'm running Windows '98." "Yes." "My computer isn't working now." "Yes, you already said that." -- User-Friendly
Aug 08 2013
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Aug 08, 2013 at 08:24:19PM +0200, Adam D. Ruppe wrote:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
I made a pull request improving the API a few weeks ago,
no one seems to really care.

Phobos needs a new dictator.

Or rather, Phobos needs more reviewers/committers. In the past 2 weeks or so, pull requests have been piling up, and lots of them have a green on the autotester, yet very few requests are actually being pulled. Sounds like the usually-active Phobos devs are busy with other things. T -- In a world without fences, who needs Windows and Gates? -- Christian Surchi
Aug 08 2013
prev sibling next sibling parent "anonymous" <anonymous example.com> writes:
On Thursday, 8 August 2013 at 12:38:39 UTC, SteveGuo wrote:
 I think bug reports should be here 
 http://forum.dlang.org/group/digitalmars.D.bugs

Actually, that only shows the activity on the bug tracker. New issues should be reported here: http://d.puremagic.com/issues
Aug 08 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Tofu Ninja:

 I don't think this is a bug, the json spec seems to indicate 
 that this is valid.

The JSON decode of the Python2.6.6 standard library seems to refuse it:
 import json
 json.loads("1.000")



 json.loads("1                                  .000")



File "<stdin>", line 1, in <module> File "...\Python26\lib\json\__init__.py", line 307, in loads return _default_decoder.decode(s) File "...\Python26\lib\json\decoder.py", line 322, in decode raise ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 1 column 35 - line 1 column 39 (char 35 - 39) Bye, bearophile
Aug 08 2013
prev sibling next sibling parent "khurshid" <khurshid.normuradov gmail.com> writes:
On Thursday, 8 August 2013 at 12:38:39 UTC, SteveGuo wrote:
 On Thursday, 8 August 2013 at 12:08:00 UTC, khurshid wrote:
 std.json  parseJSON has a bug,

 //                         leading whitespaces
 auto json = parseJSON("1                                  
 .000");

 -- should throws, but it's success parsed as `1.000` .


 Khurshid...

I think bug reports should be here http://forum.dlang.org/group/digitalmars.D.bugs

I reported. http://d.puremagic.com/issues/show_bug.cgi?id=10776
Aug 08 2013
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
David:

 If we follow an existing implementation, we should follow 
 JavaScript

In my opinion we should follow the formal JSON grammar. Bye, bearophile
Aug 08 2013
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Aug 08, 2013 at 11:40:45PM +0200, David wrote:
 Am 08.08.2013 22:15, schrieb Tofu Ninja:

 It is really bad that people are actually talking about starting
 there own standard lib, I wasn't around for the whole phobos vs
 tango thing but from what I hear, it wasn't pretty. If there is
 problems with phobos or the way its managed, I feel like we should
 try and fix them and not try to replace it.

phobos is actually useable, still I don't like many of its design choices. I feel the same with D. D itself is better than most languages, I still think there is a long way to go, but "official" D people seem like it's enough and block progress

Well, I don't speak for them, whoever they are, but in my understanding one of the main reasons there's such a high bar to changes is because we're trying to stabilize the language so that adopters can rely on it not having drastic changes every other release, that will require massive code rewrites. No language will ever be perfect, and if we keep on breaking people's existing code in the endless quest to improve the language, D will have no users left.
 (e.g.  property/-property the whole situation is a mess, but instead
 of properly reimplementing it, they try to fix an already broken
 system and break code with it, why not break code once, but replace it
 with a proper system)

The problem is that nobody can agree on what the "proper" system should be. We've had that property discussion before. Nobody is happy with the current state of affairs, but nobody can agree with what the solution should be, either. We just have long neverending debates about it but no conclusion is ever reached. Perhaps what's needed, is somebody who's headstrong enough, and persistent enough, to just pick one of the solutions -- any one of them -- take the time to implement it in DMD, and show it to the rest of us to prove how superior it is. Then continue to pester Walter ceaselessly until he agrees to merge it. In the current state of stalemate, an actual, working implementation of a solution (even if it's not quite the one that one may have in mind) should be pretty convincing. Complaining about it yet being unwilling to do anything about it, OTOH, is very likely to just fall on deaf ears, judging from what I've seen around these parts. That's just the way things work, unfortunately. *shrug* T -- Some days you win; most days you lose.
Aug 08 2013
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
On Saturday, 10 August 2013 at 18:28:32 UTC, H. S. Teoh wrote:
 ...
 T

Perhaps there's a forum bug here. I was not in this broken piece of therad. Bye, bearophile
Aug 10 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.
Aug 08 2013
prev sibling next sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

Then my reading says this really _is_ a bug. At least, according to http://www.ietf.org/rfc/rfc4627.txt, which states and shows only that: "Insignificant whitespace is allowed before or after any of the six structural characters." ..unless there's a different version that supersedes that one? -Wyatt
Aug 08 2013
prev sibling next sibling parent reply "Tyler Jameson Little" <beatgammit gmail.com> writes:
On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

Exactly. Here's the official web page complete with nice graphics detailing the grammar: http://json.org/. I've read the JSON RFC before, but I can't remember what it says about whitespace within a basic value, but the graphics here make it very clear that whitespace does *not* belong inside a value. The real question is, is this worth fixing before std.serialize makes it in? There will likely be other bugs once/if that's accepted. I tried extending std.json in the past with static reflection, but that didn't make it in for this very reason.
Aug 08 2013
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Aug 10, 2013 at 09:09:17PM +0200, bearophile wrote:
 On Saturday, 10 August 2013 at 18:28:32 UTC, H. S. Teoh wrote:
...
T

Perhaps there's a forum bug here. I was not in this broken piece of therad.

You snipped too much of the context, so I've no idea what you're talking about. But this is a known problem with the mailman/NNTP interface: somewhere along the line message IDs get rewritten when they shouldn't be, causing threads to break. Somebody seriously needs to look into this, as it's an annoying problem that keeps cropping up. T -- He who laughs last thinks slowest.
Aug 10 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

A formal grammar can be found here starting on page 202 but I don't know enough about grammars to be able to interpret it. http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf Im starting to become less sure if its a bug or not...
Aug 08 2013
prev sibling next sibling parent "Borislav Kosharov" <boby_dsm abv.bg> writes:
On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

I agree with Dicebot. I posted on the other thread for JSON real numbers and explained why it isn't a bug.
Aug 08 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 08 Aug 2013 17:17:38 +0200
"Tofu Ninja" <emmons0 purdue.edu> wrote:

 On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

A formal grammar can be found here starting on page 202 but I don't know enough about grammars to be able to interpret it. http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf Im starting to become less sure if its a bug or not...

I think that makes it clear that whitespace is NOT allowed within numeric or fractional literals. Note that the JSONNumber and JSONFraction tokens are defined in the lexical section (15.12.1.1). In a lexer, whitespace is something that is always handled explicitly (as they do, in both the JSONWhiteSpace and JSONStringCharacter tokens). Down in the "Syntactic Grammar" section (15.12.1.2), they don't explicitly mention anything about whitespace (for example, in the JSONElementList token, where we already *know* whitespace is allowed). This is undoubtedly because (aside from lexing) whitespace is generally NOT explicit, but rather implied to optionally exist before/after each token. It sounds like the bug is that std.json is currently treating JSONNumber (and maybe JSONFraction too) as a parser-handled nonterminal, instead of a lexer-handled terminal as the spec indicates.
Aug 08 2013
prev sibling next sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Thursday, 8 August 2013 at 15:17:40 UTC, Tofu Ninja wrote:
 A formal grammar can be found here starting on page 202 but I 
 don't know enough about grammars to be able to interpret it.
 http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf

 Im starting to become less sure if its a bug or not...

I don't recogise that exact BNF notation (single colon? New to me.), but unless it's fairly...unique, it's still a bug according to a strict reading. (I feel the RFC version is somewhat more clear, honestly.) -Wyatt
Aug 08 2013
prev sibling next sibling parent "Lemonfiend" <lemon fie.nd> writes:
On Thursday, 8 August 2013 at 15:15:44 UTC, David wrote:
 The real question is, is this worth fixing before 
 std.serialize makes it
 in? There will likely be other bugs once/if that's accepted. I 
 tried
 extending std.json in the past with static reflection, but 
 that didn't
 make it in for this very reason.

Yep no one seems to care about std.json.

I heard about std.xml being crap and was going to use std.json, then I ran into this thread. So what should one use to parse json, or xml for that matter?
Aug 08 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 8 August 2013 at 16:26:40 UTC, Lemonfiend wrote:
 On Thursday, 8 August 2013 at 15:15:44 UTC, David wrote:
 The real question is, is this worth fixing before 
 std.serialize makes it
 in? There will likely be other bugs once/if that's accepted. 
 I tried
 extending std.json in the past with static reflection, but 
 that didn't
 make it in for this very reason.

Yep no one seems to care about std.json.

I heard about std.xml being crap and was going to use std.json, then I ran into this thread. So what should one use to parse json, or xml for that matter?

There are several JSON implementations floating around, I personally prefer vibe.d one - https://github.com/Dicebot/vibe.d/blob/master/source/vibe/data/json.d For XML Tango module is often mentioned as most awesome one - https://github.com/SiegeLord/Tango-D2/tree/d2port/tango/text/xml
Aug 08 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Thursday, 8 August 2013 at 15:45:22 UTC, Nick Sabalausky wrote:
 On Thu, 08 Aug 2013 17:17:38 +0200
 "Tofu Ninja" <emmons0 purdue.edu> wrote:

 On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

A formal grammar can be found here starting on page 202 but I don't know enough about grammars to be able to interpret it. http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf Im starting to become less sure if its a bug or not...

I think that makes it clear that whitespace is NOT allowed within numeric or fractional literals. Note that the JSONNumber and JSONFraction tokens are defined in the lexical section (15.12.1.1). In a lexer, whitespace is something that is always handled explicitly (as they do, in both the JSONWhiteSpace and JSONStringCharacter tokens). Down in the "Syntactic Grammar" section (15.12.1.2), they don't explicitly mention anything about whitespace (for example, in the JSONElementList token, where we already *know* whitespace is allowed). This is undoubtedly because (aside from lexing) whitespace is generally NOT explicit, but rather implied to optionally exist before/after each token. It sounds like the bug is that std.json is currently treating JSONNumber (and maybe JSONFraction too) as a parser-handled nonterminal, instead of a lexer-handled terminal as the spec indicates.

Just out of curiosity what is this sort of grammar format called or what would one want to try to look up to understand it? I just have seen it around a few times and never really knew what it was about other than some how it describes a language.
Aug 08 2013
prev sibling next sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Thursday, 8 August 2013 at 17:37:37 UTC, Tofu Ninja wrote:
 Just out of curiosity what is this sort of grammar format 
 called or what would one want to try to look up to understand 
 it? I just have seen it around a few times and never really 
 knew what it was about other than some how it describes a 
 language.

That'd be some flavour of BNF: https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form Very handy thing to know how to read (though I prefer PEGs). -Wyatt
Aug 08 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.
Aug 08 2013
prev sibling next sibling parent "Craig Dillabaugh" <cdillaba cg.scs.carleton.ca> writes:
On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
 Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...

Maybe the Tango developers would want to join in :o)
Aug 08 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
 Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...

It is really bad that people are actually talking about starting there own standard lib, I wasn't around for the whole phobos vs tango thing but from what I hear, it wasn't pretty. If there is problems with phobos or the way its managed, I feel like we should try and fix them and not try to replace it.
Aug 08 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Thu, 08 Aug 2013 22:15:28 +0200
schrieb "Tofu Ninja" <emmons0 purdue.edu>:

 On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
 Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop=20 caring about phobos. I think, I even know a few who would help me...

It is really bad that people are actually talking about starting=20 there own standard lib, I wasn't around for the whole phobos vs=20 tango thing but from what I hear, it wasn't pretty. If there is=20 problems with phobos or the way its managed, I feel like we=20 should try and fix them and not try to replace it.

I'm sorry that you've had such bad experiences when contributing to phobos. I think this is less of a phobos than a std.json problem. I'd like to explain the special issue we have with std.json - as far as I understand it: std.json is a very old module:=20 * It's API is not up to current phobos standards * there are some - probably serious - bugs * no interaction with ranges * doesn't use safe const, nothrow IIRC * inherently unsafe API (accessing the union part) * could be (should be) much faster * the orignal author is no longer around. AFAICS there's nobody feeling responsible for this module. The last point is probably the biggest problem. Most people here would probably like to see a complete replacement - std.json2. Some of us have argued that we should remove std.json (and std.xml) from phobos ASAP even if there's no replacement as this code is really not what we want in the standard lib. Others are strongly against any API breakage (especially without replacement) and therefore std.json is still there. But this makes things even more difficult: Improving std.json means we'd have both API breakage and a sub-optimal std.json. I guess nobody wants to merge changes to std.json because 1) they feel std.json has to be replaced anyway and improving the old design is wasting time 2) they don't want to be blamed for any possible API breakage. There were some discussions how a std.json replacement should look like, I'll try to reiterate the main points: =3D=3D=3D Input API: =3D=3D=3D 1) A pull-parser/tokenizer/lexer * Should implement a InputRange API with ElementType JSONToken. * Should be as fast as possible * shouldn't allocate memory * shouldn't even convert strings to numbers. Instead JSONToken should have a 'type' field (ObjectStart, ArrayStart, String, Number, ...) and a value field which is always a (raw!) w/d/string. A templated T get!(JSONType) helper which could also verify the type would be useful. Any decoding of JSON strings should be done in the get function. Get may optionally use some caching. * It should be specialized for input arrays (const/immutable/mutable) of char/wchar/dchar which are completely in-memory and use slicing in these cases. * It should work with arbitrary ranges of w/d/char. To avoid memory allocation a fixed-length internal buffer should be used and token values should be slices to that buffer. (It would be good if the user can query whether values are slices of the original input which can be reused, or sliced of the iternal buffer which have to be .dup ed to keep them) 2) A DOM-API * Should be based on 1) * Same principle as current std.json, but in modern D * API similar to std.variant, DYAML optional, nice-to have: 3) A Sax-style API (based on 1) 4) A simple deserialization API * basically T fromJSON(T, CHAR)(CHAR input) (supporting all inputs supported by 1) * Shouldn't allocate any memory * Optionally skipping fields which are in the JSON data but not part of T and the other way round. * Usage: auto artist =3D fromJSON(Artist, `{"name": "", "songs":42}=C2= =B4); =3D=3D=3D Output API: =3D=3D=3D 1) A OutputRange of JSONTokens * Shouldn't allocate * Should work on top of other OutputRanges * should accept the tokens from the pull-parser, so we can do copy(JSONTokenizer(json), JSONWriter(stdout)); * Additionally provide an easier user API: JSONWriter.startObject(), JSONWriter.startArray(), JSONWriter.writeField(name, T),... 2) DOM API * It should of course be possible to manipulate the DOM, then write it back * Should be the same as the input DOM API * Should be based on 2) optional, nice-to have: 3) A simple serialization API * counterpart to the deserialization API * Shouldn't allocate any memory * Based on output-ranges and 1) * Usage: myvariable.toJSON(stdout); * Helper function using Appender: string json =3D myvariable.toJSON(); Another questions is if and how this should interact with streams, if we ever get them. This list is quite long. As you see the demands are quite high and therefore it will be a difficult and time-consuming task to write a std.json2 which pleases everyone. Of course it's possible we're overthinking this. I don't want to make any particular point here whether std.json should be removed, partially updated & fixed or simply made deprecated and left alone. I'm just trying to give some background information on this issue as far as I perceive it. BTW: Because of these issues there already are alternative libraries implementing JSON stuff. Apart from vibe.d there's also ae.util.json and some more. Fortunately as long as nobody decides to fork druntime there's no real problem. Other languages (C/C++) don't have JSON in the standard library either. But still please don't let this turn you away from phobos in general - improvements to other modules are usually handled in a much better way.
Aug 08 2013
prev sibling next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Thursday, 8 August 2013 at 21:40:46 UTC, David wrote:
 Am 08.08.2013 22:15, schrieb Tofu Ninja:
 On Thursday, 8 August 2013 at 18:31:52 UTC, David wrote:
 Am 08.08.2013 20:24, schrieb Adam D. Ruppe:
 On Thursday, 8 August 2013 at 17:33:38 UTC, David wrote:
 I made a pull request improving the API a few weeks ago,
 no one seems to really care.

Phobos needs a new dictator.

Either that, or I will soon start my own standard lib and stop caring about phobos. I think, I even know a few who would help me...

It is really bad that people are actually talking about starting there own standard lib, I wasn't around for the whole phobos vs tango thing but from what I hear, it wasn't pretty. If there is problems with phobos or the way its managed, I feel like we should try and fix them and not try to replace it.

phobos is actually useable, still I don't like many of its design choices. I feel the same with D. D itself is better than most languages, I still think there is a long way to go, but "official" D people seem like it's enough and block progress (e.g. property/-property the whole situation is a mess, but instead of properly reimplementing it, they try to fix an already broken system and break code with it, why not break code once, but replace it with a proper system)

I can understand how you feel, I do sort of feel like people are way to scared of breaking changes and it causing things that badly need to be fix to not be... but still, I don't think a whole new standard lib would do much good
Aug 08 2013
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Thursday, 8 August 2013 at 22:19:28 UTC, Johannes Pfau wrote:
 I'd like to explain the special issue we have with std.json -
 as far as I understand it:
 *snip*

This actually brings up the main beefs I have with the phobos dev process: 1) your requirements list should be prominently documented, so people considering writing something for phobos know what is needed up front 2) what's considered for phobos and what is just outside its scope? 3) these processes should be more authoritative than "as far as I understand it" - then new people could get involved with reviewing too, since they have an objective list of stuff to be on the lookout for and don't have to wait for someone else to come along and say something 4) phobos lets the perfect be the enemy of the good. std.json really isn't that bad, and ~50 lines of prettier add-on API could make it nicer, but instead of doing that we wait years for something that doesn't seem to be happening at all.
 * the orignal author is no longer around. AFAICS there's nobody 
 feeling responsible for this module.

This is a problem too: a module shouldn't be in the hands of one person. If any random contributor follows the documented rules, they should get their code pulled in. Since phobos is a community project, I think we should all be equally responsible for every part of it. One person might take the lead and do most the work.... but it shouldn't be *exclusive*, so if that one person can't or won't do something, someone else can just do it.
Aug 08 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Fri, 09 Aug 2013 00:39:50 +0200
schrieb "Adam D. Ruppe" <destructionator gmail.com>:

 On Thursday, 8 August 2013 at 22:19:28 UTC, Johannes Pfau wrote:
 I'd like to explain the special issue we have with std.json -
 as far as I understand it:
 *snip*

This actually brings up the main beefs I have with the phobos dev process: 1) your requirements list should be prominently documented, so people considering writing something for phobos know what is needed up front

I posted this to the wiki: http://wiki.dlang.org/Wish_list/std.json and linked it from http://wiki.dlang.org/Wish_list If you know some better place for that page, feel free to move it.
 
 2) what's considered for phobos and what is just outside its 
 scope?

Sometimes I wonder about that as well. We usually follow the 'batteries included' approach and all kind of modules are accepted into phobos. However Andrei often dislikes 'trivial to implement' functions and simple helper functions or helper aliases.
 
 3) these processes should be more authoritative than "as far as I 
 understand it" - then new people could get involved with 
 reviewing too, since they have an objective list of stuff to be 
 on the lookout for and don't have to wait for someone else to 
 come along and say something

There was a discussion about std.json some time ago and the list I posted is what I remember from that discussion. But we should really document such 'requirements'. Though std.json is an exception here, we usually don't have a list of 'requirements' before the module is written.
 
 4) phobos lets the perfect be the enemy of the good. std.json 
 really isn't that bad, and ~50 lines of prettier add-on API could 
 make it nicer, but instead of doing that we wait years for 
 something that doesn't seem to be happening at all.
 

when phobos is not ready for that yet. Although if it's really add-on only this shouldn't be a problem.
 
 * the orignal author is no longer around. AFAICS there's nobody 
 feeling responsible for this module.

This is a problem too: a module shouldn't be in the hands of one person. If any random contributor follows the documented rules, they should get their code pulled in. Since phobos is a community project, I think we should all be equally responsible for every part of it. One person might take the lead and do most the work.... but it shouldn't be *exclusive*, so if that one person can't or won't do something, someone else can just do it.

This is working for some modules(std.algorithm, std.range). I don't know why it's not working for modules like std.json or std.xml.
Aug 09 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Aug 08, 2013 at 03:49:15PM +0200, bearophile wrote:
 David:
 
If we follow an existing implementation, we should follow
JavaScript

In my opinion we should follow the formal JSON grammar.

+1. T -- Recently, our IT department hired a bug-fix engineer. He used to work for Volkswagen.
Aug 08 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, August 08, 2013 14:38:34 SteveGuo wrote:
 I think bug reports should be here
 http://forum.dlang.org/group/digitalmars.D.bugs

Please, never, never post to that list. It shouldn't even accept posts to it (and it may not anymore, since I haven't seen anyone post to it for a while). The whole purpose of that list is to sign up for notifications from bugzilla, and bugzilla will post to the list whenever a bug report is created or changed. All bug reports should go to http://d.puremagic.com/issues - Jonathan M Davis
Aug 08 2013
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Aug 08, 2013 at 11:45:19AM -0400, Nick Sabalausky wrote:
 On Thu, 08 Aug 2013 17:17:38 +0200
 "Tofu Ninja" <emmons0 purdue.edu> wrote:
 
 On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
 On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
 In my opinion we should follow the formal JSON grammar.

This. Anyone who wants JavaScript behavior can use own third-party library bust standard library must behave according to published standards and specifications.

A formal grammar can be found here starting on page 202 but I don't know enough about grammars to be able to interpret it. http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf Im starting to become less sure if its a bug or not...


I find that the spec is not very clear as to what is/isn't allowed. RFC-4627 is more unambiguous: it explicitly states that insignificant whitespace is allowed before or after any of the six structural characters, which are defined (in section 2) to be: [ { ] } : , Section 2.4 in RFC-4627 defines the number format very clearly. There is no mention of insignificant whitespace. This, plus the statement above in section 2 about structural characters, makes it clear that whitespace is NOT allowed inside a number literal. So please file a bug against std.json. T -- If you compete with slaves, you become a slave. -- Norbert Wiener
Aug 08 2013