www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Orange - Free from D1/Tango

reply Jacob Carlborg <doob me.com> writes:
I just stripped out all D1 and Tango related code from Orange. D1/Tango 
is still supported in the d1 branch. Hopefully this will make it easier 
to integrate into Phobos.

It also now supports UDA's for indicating a field/class/struct shouldn't 
be serialized:

class Foo
{
      nonSerialized int a;
}

 nonSerialized class Bar { }

For those unfamiliar with Orange, it's a serialization library. More 
information available at github:

https://github.com/jacob-carlborg/orange

-- 
/Jacob Carlborg
Feb 17 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange. D1/Tango is
still
 supported in the d1 branch. Hopefully this will make it easier to integrate
into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct shouldn't be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?
Feb 17 2013
next sibling parent reply "Adam Wilson" <flyboynw gmail.com> writes:
On Sun, 17 Feb 2013 13:18:05 -0800, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange. D1/Tango  
 is still
 supported in the d1 branch. Hopefully this will make it easier to  
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct  
 shouldn't be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

Every serialization library i've used is opt-out. -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Feb 17 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-17 22:52, Adam Wilson wrote:

 Every serialization library i've used is opt-out.

Java - Requires annotation Ruby - Doesn't require anything -- /Jacob Carlborg
Feb 17 2013
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Feb 2013 16:18:05 -0500, Walter Bright  
<newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange. D1/Tango  
 is still
 supported in the d1 branch. Hopefully this will make it easier to  
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct  
 shouldn't be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

I would think D's type system would be capable enough where a serialization library can tell whether a type can be serialized or not. I also think it would be a tremendous burden to put serialized on every type that could be serialized. When I used C#'s serializer, it was definitely annoying to have to annotate every type I wanted to serialize, and every type that was used within that type. What is the concern with an opt-out approach? I would think the default would be the most common option -- save everything. -Steve
Feb 17 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
 I would think D's type system would be capable enough where a
 serialization library can tell whether a type can be serialized or not.

Sometimes you don't want to serialize a field, regardless if the serialization library can serialize it or not. This will require a manual annotation.
 I also think it would be a tremendous burden to put  serialized on every
 type that could be serialized.

 When I used C#'s serializer, it was definitely annoying to have to
 annotate every type I wanted to serialize, and every type that was used
 within that type.

 What is the concern with an opt-out approach?  I would think the default
 would be the most common option -- save everything.

I agree. -- /Jacob Carlborg
Feb 17 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 04:22, Steven Schveighoffer wrote:

 Right, but in that case, you may have a common reference serialized
 several times.  Not necessarily the end of the world.

No reference type should be serialized more than once. If it is, it's bug.
 Perhaps what we need is two things:

 1. A UDA that indicates "this type is certified correct for serialization"
 2. A flag to serializer somehow that indicates "only serialize types
 that are certified correct for serialization, and pure value types"

Hmm, I'm wondering if that's getting too complex.
 I would hate to (and have hated to) mark serializable types when it is a
 trivial type, just because it's used inside a serializable type.

Me too. -- /Jacob Carlborg
Feb 17 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 15:32, Steven Schveighoffer wrote:

 Then maybe I didn't understand deadalnix's point, or maybe you have
 solved his problem?  I don't know enough about your library.

I didn't really understand deadalnix's point. But I answered your comment. My answer to your comment is true but I don't know if that has anything to do with what deadalnix said.
 If there isn't a problem, then we don't need a solution.  But if there
 is a problem, we may need something like this.

I don't know, is there a problem? -- /Jacob Carlborg
Feb 18 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 18:46, Steven Schveighoffer wrote:

 Hm... maybe something like this:

 class Widget
 {
    Display d;
 }

 If you serialize Widget, then it will try to serialize d, but d is not
 owned by Widget, it's just referenced by Widget.

 How does Orange deal with this?

The first time Orange sees "d" it will just serialize it. If there's any other reference to "d", somewhere in the data structure that is being serialized, it will serialize a reference. If "d" is not owned by Widget it will serialize it anyway. It doesn't care where it comes from. This code: http://pastebin.com/n1jwG9vE Will serialize to this XML: http://pastebin.com/tHbzVRqZ
 What if whoever wrote Widget never
 intended it to be serialized, but wasn't aware of the serialization
 library, so didn't know to mark it as  nonSerialized

I don't know if there is solution for that if the serialization library is using an opt-out approach. The one serializing the data structure is responsible. -- /Jacob Carlborg
Feb 18 2013
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 17 February 2013 at 21:18:12 UTC, Walter Bright wrote:
 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange. 
 D1/Tango is still
 supported in the d1 branch. Hopefully this will make it easier 
 to integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct 
 shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

Surely not... I shouldn't have to explicitly mark every type as allowed-to-be-serialised, it makes much more sense to opt-out.
Feb 17 2013
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky  
<SeeWebsiteToContactMe semitwist.com> wrote:

 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange.
 D1/Tango is still supported in the d1 branch. Hopefully this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a field/class/struct
 shouldn't be serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

You're already opting-in to serialization anyway when you say "serialize object foobar". serializable would just be redundant.

I think Walter's point is that the author of foobar may not have opted in. My response to that is, so? If someone is trying to serialize your class, and you didn't test for that, too bad for that person if it doesn't work. The reality is, an author may write foobar without intending it to be serializable, but it actually is. In that case, it is still on the user, but if it works, great! The user is taking the risk that later the serialization breaks, and could always submit a patch to the author of foobar if it somehow becomes not-working in a future version. To have to say: struct S { int x; int y; } is serializable seems like super-redundant info. The D type system is one of the most advanced I've ever seen. Let's try and use it! -Steve
Feb 17 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 02:27, Steven Schveighoffer wrote:

 I think Walter's point is that the author of foobar may not have opted in.

 My response to that is, so?  If someone is trying to serialize your
 class, and you didn't test for that, too bad for that person if it
 doesn't work.

 The reality is, an author may write foobar without intending it to be
 serializable, but it actually is.  In that case, it is still on the
 user, but if it works, great!  The user is taking the risk that later
 the serialization breaks, and could always submit a patch to the author
 of foobar if it somehow becomes not-working in a future version.

 To have to say:

 struct S
 {
    int x;
    int y;
 }

 is serializable seems like super-redundant info.  The D type system is
 one of the most advanced I've ever seen.  Let's try and use it!

I agree. -- /Jacob Carlborg
Feb 17 2013
prev sibling next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
18-Feb-2013 05:27, Steven Schveighoffer пишет:
 On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky
 <SeeWebsiteToContactMe semitwist.com> wrote:

 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange.
 D1/Tango is still supported in the d1 branch. Hopefully this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a field/class/struct
 shouldn't be serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

You're already opting-in to serialization anyway when you say "serialize object foobar". serializable would just be redundant.

I think Walter's point is that the author of foobar may not have opted in. My response to that is, so? If someone is trying to serialize your class, and you didn't test for that, too bad for that person if it doesn't work. The reality is, an author may write foobar without intending it to be serializable, but it actually is. In that case, it is still on the user, but if it works, great! The user is taking the risk that later the serialization breaks, and could always submit a patch to the author of foobar if it somehow becomes not-working in a future version. To have to say: struct S { int x; int y; } is serializable seems like super-redundant info. The D type system is one of the most advanced I've ever seen. Let's try and use it!

Exactly. Would be nice to be able to mark the whole structs as non-serializable though. And of course, to not have to import serialization library just to get the annotations.
 -Steve

-- Dmitry Olshansky
Feb 18 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 10:43, Dmitry Olshansky wrote:

 Exactly.

 Would be nice to be able to mark the whole structs as non-serializable
 though. And of course, to not have to import serialization library just
 to get the annotations.

import orange.serialization.Serializable; nonSerialized class Bar { } -- /Jacob Carlborg
Feb 18 2013
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
18-Feb-2013 14:28, Jacob Carlborg пишет:
 On 2013-02-18 10:43, Dmitry Olshansky wrote:

 Exactly.

 Would be nice to be able to mark the whole structs as non-serializable
 though. And of course, to not have to import serialization library just
 to get the annotations.

import orange.serialization.Serializable; nonSerialized class Bar { }

It's just would be really nice to have nonSerialized and simillar somewhere in object.di with prescribed semantics. Then e.g. any serialization libraries including some 3rd party ones will play by the same basic rules as far as user is concerned. -- Dmitry Olshansky
Feb 18 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-18 11:56, Dmitry Olshansky wrote:

 Good.

 It's just would be really nice to have  nonSerialized and simillar
 somewhere in object.di with prescribed semantics. Then e.g. any
 serialization libraries including some 3rd party ones will play
 by the same basic rules as far as user is concerned.

Yeah, when/if in Phobos that would be a good idea. -- /Jacob Carlborg
Feb 18 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 12:51, Johannes Pfau wrote:

 Does this really belong into object.di? It adds more stuff to the
 global namespace (and druntime). Couldn't we just place "standard,
 library attributes" into std.attributes or something similar?

That's true. I don't know what's best. -- /Jacob Carlborg
Feb 18 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
18-Feb-2013 15:51, Johannes Pfau пишет:
 Am Mon, 18 Feb 2013 12:03:29 +0100
 schrieb Jacob Carlborg <doob me.com>:

 On 2013-02-18 11:56, Dmitry Olshansky wrote:

 Good.

 It's just would be really nice to have  nonSerialized and simillar
 somewhere in object.di with prescribed semantics. Then e.g. any
 serialization libraries including some 3rd party ones will play
 by the same basic rules as far as user is concerned.

Yeah, when/if in Phobos that would be a good idea.

Does this really belong into object.di? It adds more stuff to the global namespace (and druntime). Couldn't we just place "standard, library attributes" into std.attributes or something similar?

I think it doesn't matter and shouldn't become a roadblock to its inclusion someplace standard. But if it's in object.d or core.attributes then druntime too can make use of it. It's a recurring problem that druntime has no access to certain useful things like std.traits simply because of their placement in fs ;) and I'd hate to see this problem repeat again. I also suspect that it'd be around 10-20 attributes tops. -- Dmitry Olshansky
Feb 18 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-18 13:27, Dmitry Olshansky wrote:

 I think it doesn't matter and shouldn't become a roadblock to its
 inclusion someplace standard. But if it's in object.d or core.attributes
 then druntime too can make use of it. It's a recurring problem that
 druntime has no access to certain useful things like std.traits simply
 because of their placement in fs ;) and I'd hate to see this problem
 repeat again.

Yeah, druntime you use this attribute if it was added. We don't want to serialize threads and IO objects. -- /Jacob Carlborg
Feb 18 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-19 00:43, qznc wrote:

 I agree that for lots of code it is redundant (data structures in
 general). It might even be desirable for library code, whose author did
 not consider Orange. However, it might lead into subtle bugs in some cases.

 For example, what happens with a mutex? Would Orange serialize it with
 all its state (locked or unlocked)?

It would just serialize it. It serialize all internal sate, except for void pointers.
 What happens with data structures, which are used in parallel. For a
 consistent serialization it might be necessary to take a lock first. In
 this case, the data structure must be able to provide its own
 serialization method.

Yes, or perhaps the user can use a lock or similar when perform the serialization?
 I have not seen a "skip on serialize, but re-initialize on deserialize"
 annotation, which would be the correct behavior for uncontested
 thread-safe data structures using locks.

The library supports custom serialization, both for your own types and for third party types. https://github.com/jacob-carlborg/orange/blob/master/tests/Custom.d https://github.com/jacob-carlborg/orange/blob/master/tests/NonIntrusive.d
 The most complex case would probably be something like a GUI toolkit. If
 I serialize a GtkD app on Linux and deserialize it on Windows, will it
 be able to produce a working GUI? It must provide a custom deserialize
 method to do that.

Yes, it will most likely need a custom (de)serialization method. If we take DWT as an example. It could automatically serialize all internal state except for the native widget that is stored. So in this case it would probably be best to use custom serialization and only use the public API to set the properties. This would make sure that the underlying native widget is updated as well. Theoretically it would be possible to serialize the native widget as well, at least on Mac OS X. That's how Xcode/InterfaceBuilder is working.
 Looking at all those edge cases, an opt-in approach seems not that
 stupid to me. It might be tedious to add all those  serializeable
 annotations, but at least you do not run into weird deadlocks.

I don't agree. I think that most data types _can_ be serialized. -- /Jacob Carlborg
Feb 18 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-19 04:26, Steven Schveighoffer wrote:

 Most likely a mutex is a core library feature, and we can mark it as not
 serializeable.  If (when?) phobos gets serialization, I would expect the
 core types to be marked as such.

Exactly. IO objects, threads are also included in this category.
 I would expect that something that complex has to be specifically
 written to deal with serialization.

Yes, exactly, see my other reply: http://forum.dlang.org/thread/kfrfs4$2h6f$1 digitalmars.com?page=4#post-kfvard:242nfi:241:40digitalmars.com
 Just because someone is stupid and tries to serialize a non-serializable
 construct such as a gui toolkit, it's not the serializer's fault.

 I don't think opt-in is stupid, it's just a more conservative set of
 rules.  I think the number of constructs that WON'T be serializable will
 be far less than the ones that will be.

I agree.
 And multi-thread access items could be made opt-in.  I'm not sure what
 Orange does now.

Orange just serialize everything. I haven't actually tried with anything marked as "shared". Perhaps I should do that. -- /Jacob Carlborg
Feb 18 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 17 February 2013 at 22:13:21 UTC, Steven Schveighoffer 
wrote:
 On Sun, 17 Feb 2013 16:18:05 -0500, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from 
 Orange. D1/Tango is still
 supported in the d1 branch. Hopefully this will make it 
 easier to integrate into
 Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

I would think D's type system would be capable enough where a serialization library can tell whether a type can be serialized or not.

It isn't as it don't convey ownership. And hopefully as getting rid of ownership is important for idioms involving immutability.
Feb 17 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 17 February 2013 at 21:52:53 UTC, Adam Wilson wrote:
 On Sun, 17 Feb 2013 13:18:05 -0800, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from 
 Orange. D1/Tango is still
 supported in the d1 branch. Hopefully this will make it 
 easier to integrate into
 Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

Every serialization library i've used is opt-out.

That doesn't mean it is the way. I have to say I don't know if it is better to opt in or out, simply that most serialization framewrok I have used as of now weren't that great, and I'm sure we can do better with both compile time reflection and annotations.
Feb 17 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 17 Feb 2013 22:08:26 -0500, deadalnix <deadalnix gmail.com> wrote:

 On Sunday, 17 February 2013 at 22:13:21 UTC, Steven Schveighoffer wrote:
 On Sun, 17 Feb 2013 16:18:05 -0500, Walter Bright  
 <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange.  
 D1/Tango is still
 supported in the d1 branch. Hopefully this will make it easier to  
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct  
 shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

I would think D's type system would be capable enough where a serialization library can tell whether a type can be serialized or not.

It isn't as it don't convey ownership. And hopefully as getting rid of ownership is important for idioms involving immutability.

Right, but in that case, you may have a common reference serialized several times. Not necessarily the end of the world. Perhaps what we need is two things: 1. A UDA that indicates "this type is certified correct for serialization" 2. A flag to serializer somehow that indicates "only serialize types that are certified correct for serialization, and pure value types" I would hate to (and have hated to) mark serializable types when it is a trivial type, just because it's used inside a serializable type. -Steve
Feb 17 2013
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/17/2013 1:43 PM, H. S. Teoh wrote:
 I believe the purpose of the library is to provide convenient saving /
 loading of objects (among other things), so you want most of your data
 serialized, and non-serialized fields are the exception rather than the
 norm.  So it makes sense to serialize by default and mark out fields
 that shouldn't be serialized.

I've done some work with serialization, and it was a minority of the data types that needed serialization.
Feb 17 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Feb 17, 2013 at 07:22:03PM -0800, Walter Bright wrote:
 On 2/17/2013 1:43 PM, H. S. Teoh wrote:
I believe the purpose of the library is to provide convenient saving /
loading of objects (among other things), so you want most of your data
serialized, and non-serialized fields are the exception rather than the
norm.  So it makes sense to serialize by default and mark out fields
that shouldn't be serialized.

I've done some work with serialization, and it was a minority of the data types that needed serialization.

Wait, I thought the whole point was that the user code specifically asks for the serialization of a particular data structure, so presumeably most of the objects recursively referenced by that object will tend to be data objects where the vast majority of fields should be serialized. I don't think the point was to blindly serialize every object in the program? T -- If Java had true garbage collection, most programs would delete themselves upon execution. -- Robert Sewell
Feb 17 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 18 February 2013 at 03:22:07 UTC, Steven Schveighoffer 
wrote:
 Right, but in that case, you may have a common reference 
 serialized several times.  Not necessarily the end of the world.

This is a problem as D promote the concept of identity in addition to value.
Feb 17 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-02-17 22:43, H. S. Teoh wrote:

 I believe the purpose of the library is to provide convenient saving /
 loading of objects (among other things), so you want most of your data
 serialized, and non-serialized fields are the exception rather than the
 norm.  So it makes sense to serialize by default and mark out fields
 that shouldn't be serialized.

Exactly, Orange has had this behavior from day one. -- /Jacob Carlborg
Feb 17 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 18 February 2013 at 12:28:01 UTC, Dmitry Olshansky 
wrote:
 18-Feb-2013 15:51, Johannes Pfau пишет:
 Am Mon, 18 Feb 2013 12:03:29 +0100
 schrieb Jacob Carlborg <doob me.com>:

 On 2013-02-18 11:56, Dmitry Olshansky wrote:

 Good.

 It's just would be really nice to have  nonSerialized and 
 simillar
 somewhere in object.di with prescribed semantics. Then e.g. 
 any
 serialization libraries including some 3rd party ones will 
 play
 by the same basic rules as far as user is concerned.

Yeah, when/if in Phobos that would be a good idea.

Does this really belong into object.di? It adds more stuff to the global namespace (and druntime). Couldn't we just place "standard, library attributes" into std.attributes or something similar?

I think it doesn't matter and shouldn't become a roadblock to its inclusion someplace standard. But if it's in object.d or core.attributes then druntime too can make use of it. It's a recurring problem that druntime has no access to certain useful things like std.traits simply because of their placement in fs ;) and I'd hate to see this problem repeat again. I also suspect that it'd be around 10-20 attributes tops.

This has no meaning without the lib. This belong with the lib or in a separated module that the lib uses as well.
Feb 18 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Feb 2013 02:42:50 -0500, Jacob Carlborg <doob me.com> wrote:

 On 2013-02-18 04:22, Steven Schveighoffer wrote:

 Right, but in that case, you may have a common reference serialized
 several times.  Not necessarily the end of the world.

No reference type should be serialized more than once. If it is, it's bug.

Then maybe I didn't understand deadalnix's point, or maybe you have solved his problem? I don't know enough about your library.
 Perhaps what we need is two things:

 1. A UDA that indicates "this type is certified correct for  
 serialization"
 2. A flag to serializer somehow that indicates "only serialize types
 that are certified correct for serialization, and pure value types"

Hmm, I'm wondering if that's getting too complex.

If there isn't a problem, then we don't need a solution. But if there is a problem, we may need something like this. -Steve
Feb 18 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Feb 2013 10:09:30 -0500, Jacob Carlborg <doob me.com> wrote:

 On 2013-02-18 15:32, Steven Schveighoffer wrote:

 Then maybe I didn't understand deadalnix's point, or maybe you have
 solved his problem?  I don't know enough about your library.

I didn't really understand deadalnix's point. But I answered your comment. My answer to your comment is true but I don't know if that has anything to do with what deadalnix said.

Hm... maybe something like this: class Widget { Display d; } If you serialize Widget, then it will try to serialize d, but d is not owned by Widget, it's just referenced by Widget. How does Orange deal with this? What if whoever wrote Widget never intended it to be serialized, but wasn't aware of the serialization library, so didn't know to mark it as nonSerialized -Steve
Feb 18 2013
prev sibling next sibling parent "qznc" <qznc go.to> writes:
On Monday, 18 February 2013 at 01:27:17 UTC, Steven Schveighoffer 
wrote:
 On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky 
 <SeeWebsiteToContactMe semitwist.com> wrote:

 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from 
 Orange.
 D1/Tango is still supported in the d1 branch. Hopefully 
 this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct
 shouldn't be serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

You're already opting-in to serialization anyway when you say "serialize object foobar". serializable would just be redundant.

I think Walter's point is that the author of foobar may not have opted in. My response to that is, so? If someone is trying to serialize your class, and you didn't test for that, too bad for that person if it doesn't work. The reality is, an author may write foobar without intending it to be serializable, but it actually is. In that case, it is still on the user, but if it works, great! The user is taking the risk that later the serialization breaks, and could always submit a patch to the author of foobar if it somehow becomes not-working in a future version. To have to say: struct S { int x; int y; } is serializable seems like super-redundant info. The D type system is one of the most advanced I've ever seen. Let's try and use it!

I agree that for lots of code it is redundant (data structures in general). It might even be desirable for library code, whose author did not consider Orange. However, it might lead into subtle bugs in some cases. For example, what happens with a mutex? Would Orange serialize it with all its state (locked or unlocked)? What happens with data structures, which are used in parallel. For a consistent serialization it might be necessary to take a lock first. In this case, the data structure must be able to provide its own serialization method. I have not seen a "skip on serialize, but re-initialize on deserialize" annotation, which would be the correct behavior for uncontested thread-safe data structures using locks. The most complex case would probably be something like a GUI toolkit. If I serialize a GtkD app on Linux and deserialize it on Windows, will it be able to produce a working GUI? It must provide a custom deserialize method to do that. Looking at all those edge cases, an opt-in approach seems not that stupid to me. It might be tedious to add all those serializeable annotations, but at least you do not run into weird deadlocks.
Feb 18 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 18 Feb 2013 18:43:16 -0500, qznc <qznc go.to> wrote:

 On Monday, 18 February 2013 at 01:27:17 UTC, Steven Schveighoffer wrote:
 On Sun, 17 Feb 2013 19:58:06 -0500, Nick Sabalausky  
 <SeeWebsiteToContactMe semitwist.com> wrote:

 On Sun, 17 Feb 2013 13:18:05 -0800
 Walter Bright <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from > Orange.
 D1/Tango is still supported in the d1 branch. Hopefully > this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a > field/class/struct
 shouldn't be serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

You're already opting-in to serialization anyway when you say "serialize object foobar". serializable would just be redundant.

I think Walter's point is that the author of foobar may not have opted in. My response to that is, so? If someone is trying to serialize your class, and you didn't test for that, too bad for that person if it doesn't work. The reality is, an author may write foobar without intending it to be serializable, but it actually is. In that case, it is still on the user, but if it works, great! The user is taking the risk that later the serialization breaks, and could always submit a patch to the author of foobar if it somehow becomes not-working in a future version. To have to say: struct S { int x; int y; } is serializable seems like super-redundant info. The D type system is one of the most advanced I've ever seen. Let's try and use it!

I agree that for lots of code it is redundant (data structures in general). It might even be desirable for library code, whose author did not consider Orange. However, it might lead into subtle bugs in some cases. For example, what happens with a mutex? Would Orange serialize it with all its state (locked or unlocked)?

Most likely a mutex is a core library feature, and we can mark it as not serializeable. If (when?) phobos gets serialization, I would expect the core types to be marked as such.
 What happens with data structures, which are used in parallel. For a  
 consistent serialization it might be necessary to take a lock first. In  
 this case, the data structure must be able to provide its own  
 serialization method.

For D, we have shared which indicates it may be viewed by more than one thread. Shared data could be opt-in for serialization.
 I have not seen a "skip on serialize, but re-initialize on deserialize"  
 annotation, which would be the correct behavior for uncontested  
 thread-safe data structures using locks.

 The most complex case would probably be something like a GUI toolkit. If  
 I serialize a GtkD app on Linux and deserialize it on Windows, will it  
 be able to produce a working GUI? It must provide a custom deserialize  
 method to do that.

I would expect that something that complex has to be specifically written to deal with serialization.
 Looking at all those edge cases, an opt-in approach seems not that  
 stupid to me. It might be tedious to add all those  serializeable  
 annotations, but at least you do not run into weird deadlocks.

Just because someone is stupid and tries to serialize a non-serializable construct such as a gui toolkit, it's not the serializer's fault. I don't think opt-in is stupid, it's just a more conservative set of rules. I think the number of constructs that WON'T be serializable will be far less than the ones that will be. And multi-thread access items could be made opt-in. I'm not sure what Orange does now. -Steve
Feb 18 2013
prev sibling next sibling parent "Kapps" <opantm2+spam gmail.com> writes:
On Sunday, 17 February 2013 at 21:18:12 UTC, Walter Bright wrote:
 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange. 
 D1/Tango is still
 supported in the d1 branch. Hopefully this will make it easier 
 to integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct 
 shouldn't be
 serialized:

 class Foo
 {
      nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

The vast majority of objects can be serialized without any problems. Those that can't usually aren't referenced in objects that are being serialized and you know what they are ahead of time. Forcing opt-in serialization would make every single library out there at the moment unusable with std.serialize. In a Component system for example, you won't know ahead of time what needs to be serialized, and a single object not being marked Serialized will completely prevent serialization. That being said, personally I think serialization in Phobos has been delayed far too long for something that's so important to many programs. I'd be happy with either opt-in or opt-out, however I believe that there should be a way to customize how an object gets serialized, such as an onSerialize method or interface.
Feb 18 2013
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Monday, 18 February 2013 at 07:36:07 UTC, Jacob Carlborg wrote:
 On 2013-02-17 22:52, Adam Wilson wrote:

 Every serialization library i've used is opt-out.

Java - Requires annotation Ruby - Doesn't require anything

.NET languages as well.
Feb 20 2013
prev sibling parent reply "js.mdnq" <js_adddot+mdng gmail.com> writes:
On Sunday, 17 February 2013 at 21:31:31 UTC, Manu wrote:
 On 18 February 2013 07:18, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from 
 Orange. D1/Tango
 is still
 supported in the d1 branch. Hopefully this will make it 
 easier to
 integrate into
 Phobos.

 It also now supports UDA's for indicating a 
 field/class/struct shouldn't
 be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

I think both are useful, but I would expect opt-in rather than opt out as Walter asserts. For instance: serialise class Foo // everything in the class { int a; noSerialise int cacheValue; // except this }

yes, I think this is most useful, and: noSerialise class Foo // nothing in the class { int a; Serialise int cacheValue; // except this }
Feb 20 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-20 14:08, js.mdnq wrote:

 yes, I think this is most useful, and:

  noSerialise class Foo // nothing in the class
 {
       int a;
        Serialise int cacheValue; // except this
 }

I don't see any point in having both. -- /Jacob Carlborg
Feb 20 2013
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--f46d0444808352e4f804d5f25475
Content-Type: text/plain; charset=UTF-8

On 18 February 2013 07:18, Walter Bright <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:

 I just stripped out all D1 and Tango related code from Orange. D1/Tango
 is still
 supported in the d1 branch. Hopefully this will make it easier to
 integrate into
 Phobos.

 It also now supports UDA's for indicating a field/class/struct shouldn't
 be
 serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

I think both are useful, but I would expect opt-in rather than opt out as Walter asserts. For instance: serialise class Foo // everything in the class { int a; noSerialise int cacheValue; // except this } --f46d0444808352e4f804d5f25475 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">On 18 February 2013 07:18, Walter Bright <span dir=3D"ltr"=
&lt;<a href=3D"mailto:newshound2 digitalmars.com" target=3D"_blank">newsho=

iv class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"><div class=3D"im">On 2/17/2013 12:51 PM, Jacob Carlborg wr= ote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-= left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p= adding-left:1ex"> I just stripped out all D1 and Tango related code from Orange. D1/Tango is = still<br> supported in the d1 branch. Hopefully this will make it easier to integrate= into<br> Phobos.<br> <br> It also now supports UDA&#39;s for indicating a field/class/struct shouldn&= #39;t be<br> serialized:<br> <br> class Foo<br> {<br> =C2=A0 =C2=A0 =C2=A0 nonSerialized int a;<br> }<br> <br> nonSerialized class Bar { }<br> </blockquote> <br></div> Hmm, shouldn&#39;t it be the other way around - marking the ones to be seri= alized?<br></blockquote><div><br></div><div style>I think both are useful, = but I would expect opt-in rather than opt out as Walter asserts. For instan= ce:</div> <div style><span style=3D"color:rgb(80,0,80)"><br></span></div><div style><= span style=3D"color:rgb(80,0,80)"> serialise class Foo // everything in the= class</span><br style=3D"color:rgb(80,0,80)"><span style=3D"color:rgb(80,0= ,80)">{</span><br style=3D"color:rgb(80,0,80)"> <span style=3D"color:rgb(80,0,80)">=C2=A0 =C2=A0 =C2=A0int a;</span></div><= div style>=C2=A0 =C2=A0 =C2=A0 noSerialise int cacheValue; // except this<b= r style=3D"color:rgb(80,0,80)"><span style=3D"color:rgb(80,0,80)">}</span><= br></div><div style><span style=3D"color:rgb(80,0,80)"><br> </span></div></div></div></div> --f46d0444808352e4f804d5f25475--
Feb 17 2013
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Feb 17, 2013 at 01:18:05PM -0800, Walter Bright wrote:
 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
I just stripped out all D1 and Tango related code from Orange.
D1/Tango is still supported in the d1 branch. Hopefully this will
make it easier to integrate into Phobos.

It also now supports UDA's for indicating a field/class/struct
shouldn't be serialized:

class Foo
{
      nonSerialized int a;
}

 nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

I believe the purpose of the library is to provide convenient saving / loading of objects (among other things), so you want most of your data serialized, and non-serialized fields are the exception rather than the norm. So it makes sense to serialize by default and mark out fields that shouldn't be serialized. T -- "640K ought to be enough" -- Bill G., 1984. "The Internet is not a primary goal for PC usage" -- Bill G., 1995. "Linux has no impact on Microsoft's strategy" -- Bill G., 1999.
Feb 17 2013
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sun, 17 Feb 2013 13:18:05 -0800
Walter Bright <newshound2 digitalmars.com> wrote:

 On 2/17/2013 12:51 PM, Jacob Carlborg wrote:
 I just stripped out all D1 and Tango related code from Orange.
 D1/Tango is still supported in the d1 branch. Hopefully this will
 make it easier to integrate into Phobos.

 It also now supports UDA's for indicating a field/class/struct
 shouldn't be serialized:

 class Foo
 {
       nonSerialized int a;
 }

  nonSerialized class Bar { }

Hmm, shouldn't it be the other way around - marking the ones to be serialized?

You're already opting-in to serialization anyway when you say "serialize object foobar". serializable would just be redundant.
Feb 17 2013
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Mon, 18 Feb 2013 12:03:29 +0100
schrieb Jacob Carlborg <doob me.com>:

 On 2013-02-18 11:56, Dmitry Olshansky wrote:
 
 Good.

 It's just would be really nice to have  nonSerialized and simillar
 somewhere in object.di with prescribed semantics. Then e.g. any
 serialization libraries including some 3rd party ones will play
 by the same basic rules as far as user is concerned.

Yeah, when/if in Phobos that would be a good idea.

Does this really belong into object.di? It adds more stuff to the global namespace (and druntime). Couldn't we just place "standard, library attributes" into std.attributes or something similar?
Feb 18 2013