www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Any (working) JSON library for D2?

reply Johannes Pfau <spam example.com> writes:

the problematic part from std.json and got it to compile (by disabling
support for floating point numbers...) but using it correctly creates
very verbose code:
----------------------------------------------------------------------
JSONValue a;
if(a.type == JSONTYPE.OBJECT)
    if("member" in a)
        if(a["member"].type == TYPE.NUMBER)
            uint count = a["member"].number;
----------------------------------------------------------------------
(pseudo-code, but that's the basic workflow when using std.json)

I know there also was a std.json replacement proposed by Robert Jacques
but it requires significant patches to phobos, a std.variant
replacement and the patches are against phobos 2.050 or something like
that, so those could be out of date.

To cut a long story short: does anyone know of another JSON library
for D2?
-- 
Johannes Pfau
Jun 19 2011
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
I use std.json with a couple helper function to make it shorter.

To use:

writeln(toJson(whatever));

or

JSONValue val = toJsonValue(whatever);

Works on most basic data types: int, string, array, assoc, struct,
etc.

=====

import std.json;
import std.traits;
import std.conv;

string toJson(T)(T a) {
	auto v = toJsonValue(a);
	return toJSON(&v);
}

JSONValue toJsonValue(T)(T a) {
	JSONValue val;
	static if(is(T == JSONValue)) {
		val = a;
	} else static if(__traits(compiles, val = a.makeJsonValue())) {
		val = a.makeJsonValue();
	} else static if(isIntegral!(T)) {
		val.type = JSON_TYPE.INTEGER;
		val.integer = to!long(a);
	} else static if(isFloatingPoint!(T)) {
		val.type = JSON_TYPE.FLOAT;
		val.floating = to!real(a);
		static assert(0);
	} else static if(is(T == void*)) {
		val.type = JSON_TYPE.NULL;
	} else static if(is(T == bool)) {
		if(a == true)
			val.type = JSON_TYPE.TRUE;
		if(a == false)
			val.type = JSON_TYPE.FALSE;
	} else static if(isSomeString!(T)) {
		val.type = JSON_TYPE.STRING;
		val.str = to!string(a);
	} else static if(isAssociativeArray!(T)) {
		val.type = JSON_TYPE.OBJECT;
		foreach(k, v; a) {
			val.object[to!string(k)] = toJsonValue(v);
		}
	} else static if(isArray!(T)) {
		val.type = JSON_TYPE.ARRAY;
		val.array.length = a.length;
		foreach(i, v; a) {
			val.array[i] = toJsonValue(v);
		}
	} else static if(is(T == struct)) {
		val.type = JSON_TYPE.OBJECT;

		foreach(i, member; a.tupleof) {
			string name = a.tupleof[i].stringof[2..$];
			static if(a.tupleof[i].stringof[2] != '_')
				val.object[name] = toJsonValue(member);
		}
	} else { /* our catch all is to just do strings */
		val.type = JSON_TYPE.STRING;
		val.str = to!string(a);
	}

	return val;
}

====
Jun 19 2011
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
Oh, wait a minute, you were doing from json, not to json.

Try this on for size. It converts from a std.json.JSONValue
to a std.variant.Varaint, which is quite a bit simpler to use.

======

import std.variant;
import std.json;

Variant jsonToVariant(string json) {
	auto decoded = parseJSON(json);
	return jsonValueToVariant(decoded);
}

Variant jsonValueToVariant(JSONValue v) {
	Variant ret;

	final switch(v.type) {
		case JSON_TYPE.STRING:
			ret = v.str;
		break;
		case JSON_TYPE.INTEGER:
			ret = v.integer;
		break;
		case JSON_TYPE.FLOAT:
			ret = v.floating;
		break;
		case JSON_TYPE.OBJECT:
			Variant[string] obj;
			foreach(k, val; v.object) {
				obj[k] = jsonValueToVariant(val);
			}

			ret = obj;
		break;
		case JSON_TYPE.ARRAY:
			Variant[] arr;
			foreach(i; v.array) {
				arr ~= jsonValueToVariant(i);
			}

			ret = arr;
		break;
		case JSON_TYPE.TRUE:
			ret = true;
		break;
		case JSON_TYPE.FALSE:
			ret = false;
		break;
		case JSON_TYPE.NULL:
			ret = null;
		break;
	}

	return ret;
}

======
Jun 19 2011
parent Johannes Pfau <spam example.com> writes:
Adam D. Ruppe wrote:

Oh, wait a minute, you were doing from json, not to json.

Try this on for size. It converts from a std.json.JSONValue
to a std.variant.Varaint, which is quite a bit simpler to use.

======

import std.variant;
import std.json;

Variant jsonToVariant(string json) {
	auto decoded = parseJSON(json);
	return jsonValueToVariant(decoded);
}

Variant jsonValueToVariant(JSONValue v) {
	Variant ret;

	final switch(v.type) {
		case JSON_TYPE.STRING:
			ret = v.str;
		break;
		case JSON_TYPE.INTEGER:
			ret = v.integer;
		break;
		case JSON_TYPE.FLOAT:
			ret = v.floating;
		break;
		case JSON_TYPE.OBJECT:
			Variant[string] obj;
			foreach(k, val; v.object) {
				obj[k] = jsonValueToVariant(val);
			}

			ret = obj;
		break;
		case JSON_TYPE.ARRAY:
			Variant[] arr;
			foreach(i; v.array) {
				arr ~= jsonValueToVariant(i);
			}

			ret = arr;
		break;
		case JSON_TYPE.TRUE:
			ret = true;
		break;
		case JSON_TYPE.FALSE:
			ret = false;
		break;
		case JSON_TYPE.NULL:
			ret = null;
		break;
	}

	return ret;
}

======


Thanks, that looks great!
Can't test it right now, but it seems this even works recursively?
Awesome!

-- 
Johannes Pfau
Jun 19 2011
prev sibling parent reply Johannes Pfau <spam example.com> writes:
Adam D. Ruppe wrote:

I use std.json with a couple helper function to make it shorter.

To use:

writeln(toJson(whatever));

or

JSONValue val = toJsonValue(whatever);

Works on most basic data types: int, string, array, assoc, struct,
etc.

=====

import std.json;
import std.traits;
import std.conv;

string toJson(T)(T a) {
	auto v = toJsonValue(a);
	return toJSON(&v);
}

JSONValue toJsonValue(T)(T a) {
	JSONValue val;
	static if(is(T == JSONValue)) {
		val = a;
	} else static if(__traits(compiles, val = a.makeJsonValue())) {
		val = a.makeJsonValue();
	} else static if(isIntegral!(T)) {
		val.type = JSON_TYPE.INTEGER;
		val.integer = to!long(a);
	} else static if(isFloatingPoint!(T)) {
		val.type = JSON_TYPE.FLOAT;
		val.floating = to!real(a);
		static assert(0);
	} else static if(is(T == void*)) {
		val.type = JSON_TYPE.NULL;
	} else static if(is(T == bool)) {
		if(a == true)
			val.type = JSON_TYPE.TRUE;
		if(a == false)
			val.type = JSON_TYPE.FALSE;
	} else static if(isSomeString!(T)) {
		val.type = JSON_TYPE.STRING;
		val.str = to!string(a);
	} else static if(isAssociativeArray!(T)) {
		val.type = JSON_TYPE.OBJECT;
		foreach(k, v; a) {
			val.object[to!string(k)] = toJsonValue(v);
		}
	} else static if(isArray!(T)) {
		val.type = JSON_TYPE.ARRAY;
		val.array.length = a.length;
		foreach(i, v; a) {
			val.array[i] = toJsonValue(v);
		}
	} else static if(is(T == struct)) {
		val.type = JSON_TYPE.OBJECT;

		foreach(i, member; a.tupleof) {
			string name = a.tupleof[i].stringof[2..$];
			static if(a.tupleof[i].stringof[2] != '_')
				val.object[name] = toJsonValue(member);
		}
	} else { /* our catch all is to just do strings */
		val.type = JSON_TYPE.STRING;
		val.str = to!string(a);
	}

	return val;
}

====


I guess you do not have similar helper functions to parse JSON?

as only formatting functionality is used, but any call to parseJSON

std.conv.parse!real(string) but I'm not sure how that could be worked
around. Maybe I'll have to remove floating point support and write some
wrappers as you suggested, that could work.

-- 
Johannes Pfau
Jun 19 2011
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
Johannes Pfau wrote:

 I guess you do not have similar helper functions to parse JSON?


My other message has some. It isn't quite as nice to use though -
getting structs and such takes a little bit of work.

For example, I use it to get stuff from Facebook, and it
looks kinda like this:

===
	auto request = parseSignedRequest(signed_request).get!(Variant[string]);

	if("page" in request) {
		auto page = request["page"].get!(Variant[string]);
		pageId = page["id"].coerce!string;
		likes = page["liked"].get!bool;
	}
===

It's not as beautiful as it could be, but it works reasonably
well anyway, which is why I'm happy enough with it as it is.






I don't know anymore! For a while, I used a private fork of
std.json with the floating point functionality removed and
a utf related bug worked around, but now that fork is completely
commented out and I just use the stock std.json.

Problem is I don't remember if it's because the bugs got fixed
upstream, or if they just didn't bother me anymore...

Regardless though, it works in a test on my box at least.
Paste that code into a fresh file.


void main() {
	auto v = jsonToVariant("4.2");
	writeln(v.get!real);
}

compiles and runs correctly.
Jun 19 2011
parent reply Johannes Pfau <spam example.com> writes:
Adam D. Ruppe wrote:

Johannes Pfau wrote:

 I guess you do not have similar helper functions to parse JSON?


My other message has some. It isn't quite as nice to use though -
getting structs and such takes a little bit of work.

For example, I use it to get stuff from Facebook, and it
looks kinda like this:

===
	auto request =
	parseSignedRequest(signed_request).get!(Variant[string]);

	if("page" in request) {
		auto page = request["page"].get!(Variant[string]);
		pageId = page["id"].coerce!string;
		likes = page["liked"].get!bool;
	}
===

It's not as beautiful as it could be, but it works reasonably
well anyway, which is why I'm happy enough with it as it is.






I don't know anymore! For a while, I used a private fork of
std.json with the floating point functionality removed and
a utf related bug worked around, but now that fork is completely
commented out and I just use the stock std.json.

Problem is I don't remember if it's because the bugs got fixed
upstream, or if they just didn't bother me anymore...

Regardless though, it works in a test on my box at least.
Paste that code into a fresh file.


void main() {
	auto v = jsonToVariant("4.2");
	writeln(v.get!real);
}

compiles and runs correctly.


That's interesting, that code works indeed. Even more interesting:

dmd src/etc/curl.d src/vevo/cli/main.d src/vevo/api.d -ofvevo 
/usr/include/d/dmd/phobos/std/conv.d(1301): Error: function
std.conv.parse!(real,string).parse compiler error, parameter 'p',
bugzilla 2962? dmd: glue.c:744: virtual void
FuncDeclaration::toObjFile(int): Assertion `0' failed.

But this works:
dmd src/vevo/api.d src/etc/curl.d src/vevo/cli/main.d -ofvevo

The only difference is the argument order for dmd!
Thinking of it I think I saw a similar bug when compiling dustmite. It
consists of only two files, but it compiles only one way.

-- 
Johannes Pfau
Jun 19 2011
parent Adam D. Ruppe <destructionator gmail.com> writes:
Johannes Pfau :

 The only difference is the argument order for dmd!


Aye, I saw this one when I updated to 2.053.... now I
remember wasting an hour on that bug!

Bugzilla suggests for the workaround to just put a dummy module
in there as the first argument:

icehack.d
====
module icehack;
import std.json;
static if(__traits(compiles, parseJSON("hello"))) {}
=====

Compile:

dmd icehack.d [the rest of your arguments]



It has to do with something in dmd not being initialized
in the proper order... or something. But it's a fairly
recent regression and pretty easily worked around if it
comes up.

My work project incorporated this into it's makefile
and that's what made the pain stop.
Jun 19 2011
prev sibling parent "Lloyd Dupont" <ld galador.net> writes:
Doost : http://www.dsource.org/projects/doost

Has a serializer that can read value to and from JSon! ;)

"Johannes Pfau"  wrote in message 
news:20110619193834.2c6afdf7 jpf-Satellite-A100...


the problematic part from std.json and got it to compile (by disabling
support for floating point numbers...) but using it correctly creates
very verbose code:
----------------------------------------------------------------------
JSONValue a;
if(a.type == JSONTYPE.OBJECT)
    if("member" in a)
        if(a["member"].type == TYPE.NUMBER)
            uint count = a["member"].number;
----------------------------------------------------------------------
(pseudo-code, but that's the basic workflow when using std.json)

I know there also was a std.json replacement proposed by Robert Jacques
but it requires significant patches to phobos, a std.variant
replacement and the patches are against phobos 2.050 or something like
that, so those could be out of date.

To cut a long story short: does anyone know of another JSON library
for D2?
-- 
Johannes Pfau
Jun 19 2011