www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - [Library Release] dproto

reply Matt Soucy <msoucy csh.rit.edu> writes:
Hello, all-

I'd like to present a library that I've been working on on-and-off for
the last couple of months.

Protocol Buffers are a format produced by Google that acts as an
Interface Description Language for serializable data. Basically, given a
file containing lines like:

message Point {
	optional int32 x =3D 1 [default=3D166];
	required int32 y =3D 2;
	optional string label =3D 3;
	message Coord {
		required int32 a =3D 1;
		required int32 b =3D 2;
	}
}

You can get a structure that behaves as:

struct Point {
	int x=3D166;
	int y;
	string label;
	struct Coord {
		int a,b;
	}
}

What's the benefit? This structure also has some useful methods for
serialization and deserialization to a well-documented format - in
dproto's case, the resulting data can be stored in any ubyte[].

What makes this library special? Unlike the C++/Java/Python compilers
for .proto files, this library does the conversion at compile time using
mixins and string manipulation. All that's needed is to add the file
directories to the string imports, and then:

	mixin ProtocolBuffer!"point.proto";

Where can you get it?
It's available as a Dub package ("dproto"), or on Github at
http://github.com/msoucy/dproto

dproto is licensed under the BSD 3-clause license, and I'm definitely
open to suggestions for improvement - currently, a lot of the code was
translated from Java and so there are some things that are quite
unidiomatic.

-Matt Soucy
Oct 05 2013
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
This is more of an FYI. I've been using/updating
https://github.com/opticron/ProtocolBuffer

Boost License
And while it doesn't have any helper functions, it can generate 
source at compile time.
Generates D1 code if requested

Been using it to walk OSM data for no particular reason
https://gist.github.com/JesseKPhillips/6051600
Oct 05 2013
parent reply Matt Soucy <msoucy csh.rit.edu> writes:
On 10/05/2013 06:20 PM, Jesse Phillips wrote:

 This is more of an FYI. I've been using/updating
 https://github.com/opticron/ProtocolBuffer
=20
 Boost License
 And while it doesn't have any helper functions, it can generate source
 at compile time.
 Generates D1 code if requested
=20
 Been using it to walk OSM data for no particular reason
 https://gist.github.com/JesseKPhillips/6051600


Thanks for the tip - I actually did find this one when I started using
this, however I found it on another page that hadn't been updated (that
I ironically can't find now), so I wrongfully assumed that it was
abandoned. I did find the github page, but I wasn't thrilled by it.

Pros for opticron's:
	Much more idiomatic code
	Supports D1
	Much better unit tests
Pros for mine (highly based on opinion):
	Exposes slightly more convenient helpers
	(In my opinion) slightly easier to read
	Generated code looks like a POD struct, though with some "hidden"
attributes (optional variables support .clean(), .exists(), instead of
looking like myStruct.foo_exists())

Basically, mine is much more simplistic, which is both good and bad. I'd
like to improve mine a bit more, and the one you pointed me at is pretty
good. It gave me some things to improve on.
Oct 08 2013
parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 8 October 2013 at 20:50:22 UTC, Matt Soucy wrote:

 Thanks for the tip - I actually did find this one when I 
 started using
 this, however I found it on another page that hadn't been 
 updated (that
 I ironically can't find now), so I wrongfully assumed that it 
 was
 abandoned. I did find the github page, but I wasn't thrilled by 
 it.


You're thinking: 
https://256.makerslocal.org/wiki/index.php/ProtocolBuffer
I've been updating it from that


 Pros for opticron's:
 	Much more idiomatic code
 	Supports D1
 	Much better unit tests
 Pros for mine (highly based on opinion):
 	Exposes slightly more convenient helpers
 	(In my opinion) slightly easier to read
 	Generated code looks like a POD struct, though with some 
 "hidden"
 attributes (optional variables support .clean(), .exists(), 
 instead of
 looking like myStruct.foo_exists())


For the D2 output I've got it really close to POD. It declares 
fields as Nullable!T meaning myStruct.foo.isNull would check 
availability. I was intending to distinguish required/optional 
with this, but that would remove error checking (which I don't 
think is there yet).


 Basically, mine is much more simplistic, which is both good and 
 bad. I'd
 like to improve mine a bit more, and the one you pointed me at 
 is pretty
 good. It gave me some things to improve on.


Go for it, have fun.
Oct 09 2013
parent Matt Soucy <msoucy csh.rit.edu> writes:
On 10/09/2013 10:38 AM, Jesse Phillips wrote:

 On Tuesday, 8 October 2013 at 20:50:22 UTC, Matt Soucy wrote:

 Thanks for the tip - I actually did find this one when I started using=





 this, however I found it on another page that hadn't been updated (tha=




t

 I ironically can't find now), so I wrongfully assumed that it was
 abandoned. I did find the github page, but I wasn't thrilled by it.


=20
 You're thinking: https://256.makerslocal.org/wiki/index.php/ProtocolBuf=


fer

 I've been updating it from that
=20


Exactly, that's the page! All I remembered about the location was that
it was a wiki with a number in the URL...

 Pros for opticron's:
     Much more idiomatic code
     Supports D1
     Much better unit tests
 Pros for mine (highly based on opinion):
     Exposes slightly more convenient helpers
     (In my opinion) slightly easier to read
     Generated code looks like a POD struct, though with some "hidden"
 attributes (optional variables support .clean(), .exists(), instead of=





 looking like myStruct.foo_exists())


=20
 For the D2 output I've got it really close to POD. It declares fields a=


s

 Nullable!T meaning myStruct.foo.isNull would check availability. I was
 intending to distinguish required/optional with this, but that would
 remove error checking (which I don't think is there yet).
=20

 Basically, mine is much more simplistic, which is both good and bad. I=




'd

 like to improve mine a bit more, and the one you pointed me at is pret=




ty

 good. It gave me some things to improve on.


=20
 Go for it, have fun.


--=20
Matt Soucy
http://msoucy.me/
Oct 09 2013
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 5 October 2013 at 20:56:21 UTC, Matt Soucy wrote:

 message Point {
 	optional int32 x = 1 [default=166];
 	required int32 y = 2;
 	optional string label = 3;
 	message Coord {
 		required int32 a = 1;
 		required int32 b = 2;
 	}
 }

 You can get a structure that behaves as:

 struct Point {
 	int x=166;
 	int y;
 	string label;
 	struct Coord {
 		int a,b;
 	}
 }


Should it be really like that? If you just declare Coord struct, 
it doesn't place a Coord instance in Point.
Oct 08 2013
next sibling parent Matt Soucy <msoucy csh.rit.edu> writes:
On 10/08/2013 04:11 AM, Kagamin wrote:

 On Saturday, 5 October 2013 at 20:56:21 UTC, Matt Soucy wrote:

 message Point {
     optional int32 x =3D 1 [default=3D166];
     required int32 y =3D 2;
     optional string label =3D 3;
     message Coord {
         required int32 a =3D 1;
         required int32 b =3D 2;
     }
 }

 You can get a structure that behaves as:

 struct Point {
     int x=3D166;
     int y;
     string label;
     struct Coord {
         int a,b;
     }
 }


=20
 Should it be really like that? If you just declare Coord struct, it
 doesn't place a Coord instance in Point.


No, that's intentional. The idea was to show the possibility of nested
structs. You can create a Coord with:

auto c =3D Point.Coord();

I admit the example is kind of poor, though.
-Matt Soucy
Oct 08 2013
prev sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 8 October 2013 at 08:11:34 UTC, Kagamin wrote:

 On Saturday, 5 October 2013 at 20:56:21 UTC, Matt Soucy wrote:

 message Point {
 	optional int32 x = 1 [default=166];
 	required int32 y = 2;
 	optional string label = 3;
 	message Coord {
 		required int32 a = 1;
 		required int32 b = 2;
 	}
 }

 You can get a structure that behaves as:

 struct Point {
 	int x=166;
 	int y;
 	string label;
 	struct Coord {
 		int a,b;
 	}
 }


 Should it be really like that? If you just declare Coord 
 struct, it doesn't place a Coord instance in Point.


It is correct. A message defined in another is just namespacing 
as it is in D. If you want to include the type then the message 
needs a field of that type. Groups are a way to combine this, but 
they are deprecated so who'd want to support that.
Oct 08 2013
parent Matt Soucy <msoucy csh.rit.edu> writes:
On 10/09/2013 01:55 AM, Jesse Phillips wrote:

 Groups are a way to combine this, but they are deprecated so
 who'd want to support that.


Exactly, I didn't even bother planning out support for them.

--=20
Matt Soucy
http://msoucy.me/
Oct 09 2013