www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to parse JSON in D?

reply tastyminerals <tastyminerals gmail.com> writes:
We need to parse a bunch of JSON files. What does D have? It has 
std.json which is strangely slower than Python json :(
Ok, we go to dub repository and search for something that is 
faster. There are basically two implementations fast.json and 
stdx.data.json. The first one refuses to work on Mac M1 chip so 
we take the stdx.data.json, go to the documentation page and see 
two three snippets on how to use the library. The code examples 
are so scarce that I feel like a person who did it was forced to 
do it out of his own will.

Now, to the problem. I have the following JSON structure.

"items": {
"elements": [
    {
      "name" : "",
      "type" : ""
    },
    {
      "name": "",
      "type": ""
    }
   ]
}

auto elements = parseJSONValue("file.json")["items]["elements"];

I want to access each "name" member of "elements". How do I do it 
in stdx.data.json?
Mar 29 2021
next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 29 March 2021 at 17:21:25 UTC, tastyminerals wrote:

 We need to parse a bunch of JSON files. What does D have? It 
 has std.json which is strangely slower than Python json :(
 Ok, we go to dub repository and search for something that is 
 faster. There are basically two implementations fast.json and 
 stdx.data.json. The first one refuses to work on Mac M1 chip so 
 we take the stdx.data.json, go to the documentation page and 
 see two three snippets on how to use the library. The code 
 examples are so scarce that I feel like a person who did it was 
 forced to do it out of his own will.

 [...]


https://code.dlang.org/search?q=json
Mar 29 2021
prev sibling next sibling parent reply drug <drug2004 bk.ru> writes:
I use asdf https://code.dlang.org/packages/asdf
Also vibe-d https://code.dlang.org/packages/vibe-d has
vibe-d:data subpackage
Mar 29 2021
parent James Blachly <james.blachly gmail.com> writes:
On 3/29/21 1:44 PM, drug wrote:

 I use asdf https://code.dlang.org/packages/asdf
 Also vibe-d https://code.dlang.org/packages/vibe-d has
 vibe-d:data subpackage
 
 


Also a happy `asdf` user
Mar 29 2021
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 29 March 2021 at 17:21:25 UTC, tastyminerals wrote:

 We need to parse a bunch of JSON files. What does D have? It 
 has std.json which is strangely slower than Python json :(
 Ok, we go to dub repository and search for something that is 
 faster. There are basically two implementations fast.json and 
 stdx.data.json. The first one refuses to work on Mac M1 chip so 
 we take the stdx.data.json, go to the documentation page and 
 see two three snippets on how to use the library. The code 
 examples are so scarce that I feel like a person who did it was 
 forced to do it out of his own will.

 [...]


https://youtu.be/un-bZdyumog?t=27m25s
Mar 29 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/29/21 1:48 PM, Imperatorn wrote:

 On Monday, 29 March 2021 at 17:21:25 UTC, tastyminerals wrote:

 We need to parse a bunch of JSON files. What does D have? It has 
 std.json which is strangely slower than Python json :(
 Ok, we go to dub repository and search for something that is faster. 
 There are basically two implementations fast.json and stdx.data.json. 
 The first one refuses to work on Mac M1 chip so we take the 
 stdx.data.json, go to the documentation page and see two three 
 snippets on how to use the library. The code examples are so scarce 
 that I feel like a person who did it was forced to do it out of his 
 own will.

 [...]


 
 https://youtu.be/un-bZdyumog?t=27m25s


FYI that package is here: https://code.dlang.org/packages/jsoniopipe

I really want to start using it, but priorities.

It should be pretty functional, but it's a bit hard to use at the moment 
(unless you just want serialization). I need to work on the parsing API. 
The benefit is it allows parsing the JSON without constructing some 
tree-like representation of the data. The serializer uses this directly, 
which makes it fast. Probably not as fast as asdf, but I could maybe get 
it there with some tricks.

-Steve
Mar 29 2021
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Monday, 29 March 2021 at 18:19:35 UTC, Steven Schveighoffer 
wrote:

 On 3/29/21 1:48 PM, Imperatorn wrote:

 On Monday, 29 March 2021 at 17:21:25 UTC, tastyminerals wrote:

 [...]


 
 https://youtu.be/un-bZdyumog?t=27m25s


 FYI that package is here: 
 https://code.dlang.org/packages/jsoniopipe

 I really want to start using it, but priorities.

 It should be pretty functional, but it's a bit hard to use at 
 the moment (unless you just want serialization). I need to work 
 on the parsing API. The benefit is it allows parsing the JSON 
 without constructing some tree-like representation of the data. 
 The serializer uses this directly, which makes it fast. 
 Probably not as fast as asdf, but I could maybe get it there 
 with some tricks.

 -Steve


Omg asdf gets ~300 MB/s, I thought iopipe's 200 MB/s was fast 😅
Mar 30 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/30/21 3:05 AM, Imperatorn wrote:

 On Monday, 29 March 2021 at 18:19:35 UTC, Steven Schveighoffer wrote:

 On 3/29/21 1:48 PM, Imperatorn wrote:

 On Monday, 29 March 2021 at 17:21:25 UTC, tastyminerals wrote:

 [...]


 https://youtu.be/un-bZdyumog?t=27m25s


 FYI that package is here: https://code.dlang.org/packages/jsoniopipe

 I really want to start using it, but priorities.

 It should be pretty functional, but it's a bit hard to use at the 
 moment (unless you just want serialization). I need to work on the 
 parsing API. The benefit is it allows parsing the JSON without 
 constructing some tree-like representation of the data. The serializer 
 uses this directly, which makes it fast. Probably not as fast as asdf, 
 but I could maybe get it there with some tricks.


 
 Omg asdf gets ~300 MB/s, I thought iopipe's 200 MB/s was fast 😅


It's important to compare apples to apples (or in this case, my macbook 
to my macbook, and doing similar functions) when showing raw numbers, 
but I haven't thought about performance for jsoniopipe since that talk 
really ;)

I assume it would not be much more difficult to match or possibly exceed 
asdf's speed, it seems we have similar design goals. I'll have to at 
some point build asdf and run some benchmarks to see what my targets 
should be. At the time I used rapidjson, I got jsoniopipe to be 
comparable with not much effort.

-Steve
Mar 30 2021