www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - painlessjson released, looking for improvement advice

reply "Pierre Krafft" <kpierre+dlang outlook.com> writes:
Hi,
DUB lacked a package for going between D objects and JSON.
painlessjson is a library just released on DUB looking to be the 
easy solution for converting D data to and from JSON.

painlessjson uses templates and traits to generate code that 
converts to and from std.json.

The goals of painlessjson are:
- Easy to use and set up
- Highly configurable with sensible defaults
- Keep the library code simple by not reinventing what others 
provide for us
- Support as many D types as possible

Current features are:
- Convert to and from std.json, structs and classes with default 
constructor, arrays, associative arrays, and any type 
implementing _toJSON + _fromJSON.
- Includes  property functions
- Ignore a field using  SerializeIgnore
- Rename a field using  SerializedName("Name") or 
 SerializedToName('To')  SerializedFromName('From')

This project gained momentum just recently so there are several 
things to improve. We want input and help to make this as great 
as possible.
The features that are being designed right now are quite tricky 
to get right so any help would be greatly appreciated.

The project is on github: 
https://github.com/BlackEdder/painlessjson
And on DUB: http://code.dlang.org/packages/painlessjson

Our current issues are:
- How do we support subclasses? 
https://github.com/BlackEdder/painlessjson/issues/8
- How should constructors be handled? Don't forget alias this and 
preferably find the best constructor to use without relying on 
annotations. https://github.com/BlackEdder/painlessjson/issues/12
Jan 28 2015
parent reply "rlonstein" <rlonstein+dlang gmail.com> writes:
On Wednesday, 28 January 2015 at 21:54:06 UTC, Pierre Krafft
wrote:

 Hi,
 DUB lacked a package for going between D objects and JSON.
 painlessjson is a library just released on DUB looking to be 
 the easy solution for converting D data to and from JSON.

 painlessjson uses templates and traits to generate code that 
 converts to and from std.json.

 The goals of painlessjson are:
 - Easy to use and set up
 - Highly configurable with sensible defaults
 - Keep the library code simple by not reinventing what others 
 provide for us
 - Support as many D types as possible


Not quite the same, but I've been using dson
(https://github.com/w0rp/dson).

Have you looked it over?
Jan 28 2015
parent reply "BlackEdder" <edder tkwsping.nl> writes:
On Thursday, 29 January 2015 at 00:24:44 UTC, rlonstein wrote:

 Not quite the same, but I've been using dson
 (https://github.com/w0rp/dson).

 Have you looked it over?


Did not know about that one. From looking through the source it 
seems to have a different goal though. dson looks like it is an 
alternative to std.json. Painlessjson is meant to make it easier 
to use json, by automatically (de)serializing objects/structs and 
converting them to (and from) json.

For example:
class Point
{
   double x;
   double y;
}

void main()
{
   auto pnt = new Point( 1,2 );
   auto jsonPnt = toJSON( pnt ); // Returns a JSONValue( "{ "x": 
1.0, "y": 2.0 }" )
   auto nPnt = fromJSON!Point( jsonPnt );
   assert( nPnt.x == pnt.x && nPnt.y == pnt.y );
}

I guess it would be interesting to use dson instead of std.json 
as a basis for Painlessjson.
Jan 29 2015
parent reply "Pierre Krafft" <kpierre+dlang outlook.com> writes:
On Thursday, 29 January 2015 at 13:16:48 UTC, BlackEdder wrote:

 On Thursday, 29 January 2015 at 00:24:44 UTC, rlonstein wrote:

 Not quite the same, but I've been using dson
 (https://github.com/w0rp/dson).

 Have you looked it over?


 Did not know about that one. From looking through the source it 
 seems to have a different goal though. dson looks like it is an 
 alternative to std.json. Painlessjson is meant to make it 
 easier to use json, by automatically (de)serializing 
 objects/structs and converting them to (and from) json.

 For example:
 class Point
 {
   double x;
   double y;
 }

 void main()
 {
   auto pnt = new Point( 1,2 );
   auto jsonPnt = toJSON( pnt ); // Returns a JSONValue( "{ "x": 
 1.0, "y": 2.0 }" )
   auto nPnt = fromJSON!Point( jsonPnt );
   assert( nPnt.x == pnt.x && nPnt.y == pnt.y );
 }

 I guess it would be interesting to use dson instead of std.json 
 as a basis for Painlessjson.


I believe we should continue using std.json, and instead push for 
taking dson (or its lessons learned) into phobos if it's a better 
implementation. The best general solution should be the one that 
is available out of the box.
Jan 29 2015
parent reply "Chris" <wendlec tcd.ie> writes:
On Thursday, 29 January 2015 at 13:44:04 UTC, Pierre Krafft wrote:

 On Thursday, 29 January 2015 at 13:16:48 UTC, BlackEdder wrote:

 On Thursday, 29 January 2015 at 00:24:44 UTC, rlonstein wrote:

 Not quite the same, but I've been using dson
 (https://github.com/w0rp/dson).

 Have you looked it over?


 Did not know about that one. From looking through the source 
 it seems to have a different goal though. dson looks like it 
 is an alternative to std.json. Painlessjson is meant to make 
 it easier to use json, by automatically (de)serializing 
 objects/structs and converting them to (and from) json.

 For example:
 class Point
 {
  double x;
  double y;
 }

 void main()
 {
  auto pnt = new Point( 1,2 );
  auto jsonPnt = toJSON( pnt ); // Returns a JSONValue( "{ "x": 
 1.0, "y": 2.0 }" )
  auto nPnt = fromJSON!Point( jsonPnt );
  assert( nPnt.x == pnt.x && nPnt.y == pnt.y );
 }

 I guess it would be interesting to use dson instead of 
 std.json as a basis for Painlessjson.


 I believe we should continue using std.json, and instead push 
 for taking dson (or its lessons learned) into phobos if it's a 
 better implementation. The best general solution should be the 
 one that is available out of the box.


Yeah, I was wondering, if you have to import std.json and use it 
as a basis for painlessjson, is it really so big an improvement? 
Especially since std.json might be replaced (sooner or later). 
I'd prefer an "easy to use" implementation that replaces std.json 
completely.

(I don't want to slight the work done on painlessjson and it 
might come in handy here and there. Maybe it could be added to 
the std.json module?)
Jan 29 2015
parent reply "BlackEdder" <edder tkwsping.nl> writes:
On Thursday, 29 January 2015 at 14:05:25 UTC, Chris wrote:

 Yeah, I was wondering, if you have to import std.json and use 
 it as a basis for painlessjson, is it really so big an 
 improvement? Especially since std.json might be replaced 
 (sooner or later). I'd prefer an "easy to use" implementation 
 that replaces std.json completely.


Painlessjson in many ways abstracts away the std.json 
implementation, so using it you should rarely/never have to use 
std.json directly. It basically abstracts away most of the 
complexity and you should only ever have to use toJSON and 
fromJSON. This works with built-in types (double/int/string etc.) 
and also with ranges and associative arrays.

On top of that Painlessjson also makes it easy to convert your 
own types/struct/classes to and from JSON by (de)serializing them 
automatically (as far as possible).

If a better simpler std.json gets developed then we can rebase 
painlessjson on that and it might become a thinner wrapper, but I 
would expect the (de)serialization still to be useful in many 
cases.
Jan 29 2015
parent reply Rory McGuire via Digitalmars-d-announce writes:
:) I use jsvar for any JSON work in D. Javascript is the only thing I've
used that is possibly easier to work with JSON values.

https://github.com/adamdruppe/arsd/blob/master/jsvar.d


On Thu, Jan 29, 2015 at 5:18 PM, BlackEdder via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:


 On Thursday, 29 January 2015 at 14:05:25 UTC, Chris wrote:


 Yeah, I was wondering, if you have to import std.json and use it as a
 basis for painlessjson, is it really so big an improvement? Especially
 since std.json might be replaced (sooner or later). I'd prefer an "easy to
 use" implementation that replaces std.json completely.


 Painlessjson in many ways abstracts away the std.json implementation, so
 using it you should rarely/never have to use std.json directly. It
 basically abstracts away most of the complexity and you should only ever
 have to use toJSON and fromJSON. This works with built-in types
 (double/int/string etc.) and also with ranges and associative arrays.

 On top of that Painlessjson also makes it easy to convert your own
 types/struct/classes to and from JSON by (de)serializing them automatically
 (as far as possible).

 If a better simpler std.json gets developed then we can rebase
 painlessjson on that and it might become a thinner wrapper, but I would
 expect the (de)serialization still to be useful in many cases.
Jan 29 2015
parent reply "Pierre Krafft" <kpierre+dlang outlook.com> writes:
On Thursday, 29 January 2015 at 18:47:37 UTC, Rory McGuire wrote:

 :) I use jsvar for any JSON work in D. Javascript is the only 
 thing I've
 used that is possibly easier to work with JSON values.

 https://github.com/adamdruppe/arsd/blob/master/jsvar.d


 On Thu, Jan 29, 2015 at 5:18 PM, BlackEdder via 
 Digitalmars-d-announce <
 digitalmars-d-announce puremagic.com> wrote:


 On Thursday, 29 January 2015 at 14:05:25 UTC, Chris wrote:


 Yeah, I was wondering, if you have to import std.json and use 
 it as a
 basis for painlessjson, is it really so big an improvement? 
 Especially
 since std.json might be replaced (sooner or later). I'd 
 prefer an "easy to
 use" implementation that replaces std.json completely.


 Painlessjson in many ways abstracts away the std.json 
 implementation, so
 using it you should rarely/never have to use std.json 
 directly. It
 basically abstracts away most of the complexity and you should 
 only ever
 have to use toJSON and fromJSON. This works with built-in types
 (double/int/string etc.) and also with ranges and associative 
 arrays.

 On top of that Painlessjson also makes it easy to convert your 
 own
 types/struct/classes to and from JSON by (de)serializing them 
 automatically
 (as far as possible).

 If a better simpler std.json gets developed then we can rebase
 painlessjson on that and it might become a thinner wrapper, 
 but I would
 expect the (de)serialization still to be useful in many cases.




It's fun to see that there are so many different solutions to 
working with JSON in D. jsvar seems to be for keeping your 
variables in JavaScript-land, something I think is a bad idea in 
most cases. The idea of painlessjson is to convert to native D as 
fast as possible. The goal is to get something like 
https://github.com/FasterXML/jackson for D.
We are still looking for input on how inheritance and 
constructors should work.
Jan 29 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 29 January 2015 at 20:04:51 UTC, Pierre Krafft wrote:

 It's fun to see that there are so many different solutions to 
 working with JSON in D. jsvar seems to be for keeping your 
 variables in JavaScript-land, something I think is a bad idea 
 in most cases. The idea of painlessjson is to convert to native 
 D as fast as possible. The goal is to get something like 
 https://github.com/FasterXML/jackson for D.
 We are still looking for input on how inheritance and 
 constructors should work.


This is nothing new, I believe Vibe.d has a similar JSON 
(de)serializer.

Here's my own: 
https://github.com/CyberShadow/ae/blob/master/utils/json.d

Too slow? Try this one instead:

https://github.com/CyberShadow/ae/blob/master/utils/serialization/json.d

(Caveat: last one needs 5-ish compiler patches to work.)
Jan 31 2015
next sibling parent reply "data man" <datamanrb gmail.com> writes:
On Sunday, 1 February 2015 at 06:08:35 UTC, Vladimir Panteleev 
wrote:

 (Caveat: last one needs 5-ish compiler patches to work.)


Which, please specify?
Jan 31 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 07:56:53 UTC, data man wrote:

 On Sunday, 1 February 2015 at 06:08:35 UTC, Vladimir Panteleev 
 wrote:

 (Caveat: last one needs 5-ish compiler patches to work.)


 Which, please specify?


It should tell you nicely when you try to compile without them:

https://github.com/CyberShadow/ae/blob/master/utils/meta/caps.d
Feb 01 2015
parent reply "data man" <datamanrb gmail.com> writes:
On Sunday, 1 February 2015 at 10:07:42 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 1 February 2015 at 07:56:53 UTC, data man wrote:

 On Sunday, 1 February 2015 at 06:08:35 UTC, Vladimir Panteleev 
 wrote:

 (Caveat: last one needs 5-ish compiler patches to work.)


 Which, please specify?


 It should tell you nicely when you try to compile without them:

 https://github.com/CyberShadow/ae/blob/master/utils/meta/caps.d




Alas, but...

void main()
{
     pragma(msg, is(typeof({ struct S { int i; } S s; 
__traits(child, s, S.i) = 0; })));
}

output "false"
Feb 01 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 11:12:11 UTC, data man wrote:

 Alas, but...

 void main()
 {
     pragma(msg, is(typeof({ struct S { int i; } S s; 
 __traits(child, s, S.i) = 0; })));
 }

 output "false"


By "5-ish compiler patches" I meant patches that are not in the 
official compiler. I would be very, very surprised if it worked 
out of the box for you.
Feb 01 2015
parent reply "data man" <datamanrb gmail.com> writes:
On Sunday, 1 February 2015 at 13:43:19 UTC, Vladimir Panteleev 
wrote:

 By "5-ish compiler patches" I meant patches that are not in the 
 official compiler. I would be very, very surprised if it worked 
 out of the box for you.


Yes, I understand.
You do not plan to modify their for current state of the D 
repository?
Feb 01 2015
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 1 February 2015 at 14:10:58 UTC, data man wrote:

 On Sunday, 1 February 2015 at 13:43:19 UTC, Vladimir Panteleev 
 wrote:

 By "5-ish compiler patches" I meant patches that are not in 
 the official compiler. I would be very, very surprised if it 
 worked out of the box for you.


 Yes, I understand.
 You do not plan to modify their for current state of the D 
 repository?


It would be a massive rewrite, and since I won't have the 
compiler features I want, the code will be twice as bigger/uglier.

I'd rather wait until I have time to properly formulate and 
propose my changes as language improvements. It's a big project 
and an uphill battle, since it's fairly niche.
Feb 01 2015
prev sibling parent "Pierre Krafft" <kpierre+dlang outlook.com> writes:
On Sunday, 1 February 2015 at 06:08:35 UTC, Vladimir Panteleev 
wrote:

 On Thursday, 29 January 2015 at 20:04:51 UTC, Pierre Krafft 
 wrote:

 It's fun to see that there are so many different solutions to 
 working with JSON in D. jsvar seems to be for keeping your 
 variables in JavaScript-land, something I think is a bad idea 
 in most cases. The idea of painlessjson is to convert to 
 native D as fast as possible. The goal is to get something 
 like https://github.com/FasterXML/jackson for D.
 We are still looking for input on how inheritance and 
 constructors should work.


 This is nothing new, I believe Vibe.d has a similar JSON 
 (de)serializer.

 Here's my own: 
 https://github.com/CyberShadow/ae/blob/master/utils/json.d

 Too slow? Try this one instead:

 https://github.com/CyberShadow/ae/blob/master/utils/serialization/json.d

 (Caveat: last one needs 5-ish compiler patches to work.)


You are right. My intention is not to make it seem like this is 
new or revolutionary. The intention is to get some input on the 
work and hopefully provide a solution that in the end will be 
well documented, easy to find, and easy to integrate with. I 
don't know why I didn't find the one in vibe.d but it's still 
bundled with a lot else and uses it's own JSON implementation.

Painlessjon was the only library I could find on Github when 
searching for JSON that I could get working. Almost every library 
provided their own instead of using std.json.If std.json is bad 
it should be improved to help everyone using it.

The D community need to be better at putting their work on DUB 
and to separate their libraries more. I know that there are many 
tools out there that is not fully utilized because it's so hard 
to find them.

I will continue working on painlessjson and will take a look on 
the implementations I have now found and try to take the best 
from them. I have already seen some things in your code that I 
can make use of.
Feb 01 2015