digitalmars.D - [OT] Do you see a problem in this serialization format ?
- Basile B. (91/91) Jun 06 2016 I had to make a custom format because
- WebFreak001 (7/8) Jun 07 2016 Why do you want to make a custom format? There is a SDL library
I had to make a custom format because
- XML is not readable (think by hand edition; setting file)
- JSON is not good to represent D data types (an object must be
created for each data, currently i have a json writer I could
post a sample if you don't get what i mean)
I will use it as for Object Pascal streaming, to store the
properties of the visual objects. The serializer uses a custom
RTTI system instead of traits code. A factory could be used to
create sub objects.
============================= SAMPLE
===============================
Object root = "TextEn"
{
Alignment alignment = "none"
float width = "200"
float height = "75"
float left = "10"
float top = "180"
bool visible = "true"
bool focusable = "true"
bool wantMouse = "true"
string caption = "Lorem ipsum"
Brush fill = "Brush"
{
FillKind fillKind = "uniform"
uint color = "4294965965"
GradientKind gradientKind = "horizontal"
FillAdjustment fillAdjustment = "none"
Gradient gradient = "Gradient"
Bitmap bitmap = "Bitmap"
{
Stream data = "[]"
}
}
Brush stroke = "Brush"
{
FillKind fillKind = "uniform"
uint color = "4287245282"
GradientKind gradientKind = "horizontal"
FillAdjustment fillAdjustment = "none"
Gradient gradient = "Gradient"
Bitmap bitmap = "Bitmap"
{
Stream data = "[]"
}
}
Pen pen = "Pen"
{
float width = "1"
Dash dash = "none"
Join join = "miter"
LineCap cap = "butt"
}
Font font = "Font"
{
float size = "32"
string name = "/usr/share/fonts/truetype/FreeMono.ttf"
}
Justify justify = "left"
}
====================================================================
The structure is simple, I'm not a language technician but the
grammar would be:
({ } = " are unamed literals)
escapes
\" \n
Type
[\S]+
name
[\S]+
value
"[escapes ^"]+" opt ObjectEntry
Entry
Type name = value
Entries
Entry0
Entry1
....
EntryN
ObjectEntry
{ Entries }
sub Object declarations may look strange, eg
Stuff stuff = "Stuff" {}
instead of
Stuff stuff = {}
but actually it was easier to parse whithout special case. For
example once I reach the terminal double quote I just have to
read while white and to check the character. If it's an opened
brace I know that I'll have to read a ObjectEntry, otherwise it's
Entry.
So, what do you think, is there any design flaw ?
Jun 06 2016
On Tuesday, 7 June 2016 at 02:11:55 UTC, Basile B. wrote:I had to make a custom format becauseWhy do you want to make a custom format? There is a SDL library for D (dub uses it) and SDL looks pretty nice if you absolutely don't want to use JSON. This is the SDL page which s`ludwig made with some implementation links: http://sdlang.org/ Still for your example I think JSON would have been fine. JSON is also a widely supported standard, so I would always try to use it.
Jun 07 2016
On Tuesday, 7 June 2016 at 10:24:56 UTC, WebFreak001 wrote:On Tuesday, 7 June 2016 at 02:11:55 UTC, Basile B. wrote:SDL would have the same problem as JSON. I need to store the type, so key = value does not work and an object (that contains name = "thename", type = "thetype", value = "value") must be created for each property. The serialisation data become huge and is painfull to read. The serializer needs the type to be stored so that it can retrieve the RTTI while deserializing (or to create objects from a factory, or to retrieve a delegate in a reference storage). It's because the serialization system is type safe. A property are only restored if the name and the RTTI match.I had to make a custom format becauseWhy do you want to make a custom format? There is a SDL library for D (dub uses it) and SDL looks pretty nice if you absolutely don't want to use JSON. This is the SDL page which s`ludwig made with some implementation links: http://sdlang.org/ Still for your example I think JSON would have been fine. JSON is also a widely supported standard, so I would always try to use it.
Jun 07 2016
On Tuesday, 7 June 2016 at 11:22:55 UTC, Basile B. wrote:On Tuesday, 7 June 2016 at 10:24:56 UTC, WebFreak001 wrote:not necessarily and i don't completely understand why you want to include the schema in each message. this is a lot of bloat. and once you need more complicated datastructures it will be hard anyway and you need manual glue code. e.g. `Dlist!string[] mylist = <here be dragons>` i would go for protobufs, thrift or if you want something easy to parse then json maybe with json schema. plug: try https://github.com/tamediadigital/asdf it was more or leass created for what you want. - no object instantiation overhead - instead of "factory" methods you just use the deserializer methodsOn Tuesday, 7 June 2016 at 02:11:55 UTC, Basile B. wrote:SDL would have the same problem as JSON. I need to store the type, so key = value does not work and an object (that contains name = "thename", type = "thetype", value = "value") must be created for each property. The serialisation data become huge and is painfull to read.I had to make a custom format because
Jun 07 2016
Basile B. <b2.temp gmx.com> On Tuesday, 7 June 2016 at 11:52:55 UTC, yawniek wrote:On Tuesday, 7 June 2016 at 11:22:55 UTC, Basile B. wrote:The whole schema (so adding the type) is necessary for example to store the events targets: - in a object: Get bool delegate(string) propertyA(); - in another object bool target(string); - the target is stored with its type: storeRef!(typeof(propertyA))("obj.target", &obj.target) - propertyA = &obj.target; The serializer is able to detect that propertyA is assigned and that the target is a reference that's stored so it writes bool-delegate(string) propertyA = "obj.target" When deserializing, the serializer will try to find "obj.target" in the reference storage and is able to assign the event. If the type is not stored then - target of events can be serialized - class instances can't be created with a factory - target of properties that returns an object can't be serialized - objects that are serialized can't involve: 1 march 2016: class Segment {} class Foo { SetGet Segment segment; } written Segment segment = "" {...} 12 august 2016, for a reason or another the declarations change class Foo { SetGet float[] segment; } When reloading what's written with the 1/03/16 version in the 12/08/16 version the serializer detects the type change and an event can be used to handle the reloading of the old data. This is why the type is included. Actually the serializer already exists: https://github.com/BBasile/iz/blob/master/import/iz/serializer.d But as it will be used in a gui library: https://github.com/BBasile/kheops I want to be sure that the format is coherant. Those who know Delphi for example knows that object streaming is a master piece. Without object streaming nothing works and all the GUI construction must be hardcoded. In addition this library will use this system to apply the style of a control (for example the code posted this morning is obtained by serializing one of the primitive used to stylize a control). If I want to store the type, in JSON I obtain: { "name": "root", "type": "Object", "value": [ { "name": "alignment", "type": "Alignment", "value": "none" }, { "name": "width", "type": "float", "value": "200" }, { "name": "height", "type": "float", "value": "75" }, { "name": "left", "type": "float", "value": "10" }, { "name": "top", "type": "float", "value": "180" }, { "name": "visible", "type": "bool", "value": "true" }, { "name": "focusable", "type": "bool", "value": "true" }, { "name": "wantMouse", "type": "bool", "value": "true" }, { "name": "caption", "type": "string", "value": "Reset pos using iz.serializer" }, { "name": "fill", "type": "Brush", "value": [ { "name": "fillKind", "type": "FillKind", "value": "uniform" }, { "name": "color", "type": "uint", "value": "4294965965" }, { "name": "gradientKind", "type": "GradientKind", "value": "horizontal" }, { "name": "fillAdjustment", "type": "FillAdjustment", "value": "none" }, { "name": "gradient", "type": "Gradient", "value": "Gradient" }, { "name": "bitmap", "type": "Bitmap", "value": [ { "name": "data", "type": "Stream", "value": "[]" } ] } ] }, { "name": "stroke", "type": "Brush", "value": [ { "name": "fillKind", "type": "FillKind", "value": "uniform" }, { "name": "color", "type": "uint", "value": "4287245282" }, { "name": "gradientKind", "type": "GradientKind", "value": "horizontal" }, { "name": "fillAdjustment", "type": "FillAdjustment", "value": "none" }, { "name": "gradient", "type": "Gradient", "value": "Gradient" }, { "name": "bitmap", "type": "Bitmap", "value": [ { "name": "data", "type": "Stream", "value": "[]" } ] } ] }, { "name": "pen", "type": "Pen", "value": [ { "name": "width", "type": "float", "value": "1" }, { "name": "dash", "type": "Dash", "value": "none" }, { "name": "join", "type": "Join", "value": "miter" }, { "name": "cap", "type": "LineCap", "value": "butt" } ] }, { "name": "font", "type": "Font", "value": [ { "name": "size", "type": "float", "value": "32" }, { "name": "name", "type": "string", "value": "/usr/share/fonts/truetype/FreeMono.ttf" } ] }, { "name": "justify", "type": "Justify", "value": "left" } ] } that's insane.On Tuesday, 7 June 2016 at 10:24:56 UTC, WebFreak001 wrote:not necessarily and i don't completely understand why you want to include the schema in each message. this is a lot of bloat. and once you need more complicated datastructures it will be hard anyway and you need manual glue code. e.g. `Dlist!string[] mylist = <here be dragons>` i would go for protobufs, thrift or if you want something easy to parse then json maybe with json schema. plug: try https://github.com/tamediadigital/asdf it was more or leass created for what you want. - no object instantiation overhead - instead of "factory" methods you just use the deserializer methodsOn Tuesday, 7 June 2016 at 02:11:55 UTC, Basile B. wrote:SDL would have the same problem as JSON. I need to store the type, so key = value does not work and an object (that contains name = "thename", type = "thetype", value = "value") must be created for each property. The serialisation data become huge and is painfull to read.I had to make a custom format because
Jun 07 2016