www.digitalmars.com         C & C++   DMDScript  

D - Looking: if somone coded a serialization lib

reply School <school users.sf.net> writes:
I am just asking that. Although serialization is not a common technique,
I am interested in having one in D.
:-D
-- 
School, yet another nickname for anonymous.
:D ;-D
Apr 11 2004
next sibling parent reply "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Do you mean serialization of classes, or serialization of threads? If the
former, then I may be able to help you out. If you meant the latter, then
read on:

It appears as though Java 1.5 is adopting the full embrace of Doug Lea's
asynchronous package. Well, at least it looks like Doug's stuff. His home
page is over here:  http://gee.cs.oswego.edu/dl/

It would be uber-cool if someone were to port even part of Doug's impressive
synchronization and serialization library over to D ...

- Kris

"School" <school users.sf.net> wrote in message
news:c5b9tf$s3t$1 digitaldaemon.com...
 I am just asking that. Although serialization is not a common technique,
 I am interested in having one in D.
 :-D
 --
 School, yet another nickname for anonymous.
 :D ;-D

Apr 11 2004
next sibling parent reply School <school users.sf.net> writes:
Kris 提到:
 Do you mean serialization of classes, or serialization of threads? If the
 former, then I may be able to help you out. If you meant the latter, then
 read on:
 
 It appears as though Java 1.5 is adopting the full embrace of Doug Lea's
 asynchronous package. Well, at least it looks like Doug's stuff. His home
 page is over here:  http://gee.cs.oswego.edu/dl/
 
 It would be uber-cool if someone were to port even part of Doug's impressive
 synchronization and serialization library over to D ...
 
 - Kris
 
 "School" <school users.sf.net> wrote in message
 news:c5b9tf$s3t$1 digitaldaemon.com...
 
I am just asking that. Although serialization is not a common technique,
I am interested in having one in D.
:-D
--
School, yet another nickname for anonymous.
:D ;-D


Thank you. -- School, yet another nickname for anonymous. :D ;-D
Apr 11 2004
parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Dsc.io can handle that kind of thing quite well, with help from the
developer. For each "serializable" class you implement the ISerializable
interface (two method: read & write), and then use one of the Reader/Writer
derivatives to thaw/freeze the contents of said class (CompositeReader and
CompositeWriter are a good choice for this kind of thing, possibly in
conjunction with the Endian classes).

That is, Dsc eschews the Java approach of "serialization through
introspection" in favour of a framework to get the job done under the
watchful eye of a developer. There's no rocket science involved here; in
fact, you could do this kind of thing with printf and scanf if you really
wanted to <g>

Naturally, this approach only serializes the 'data' portion of a class; not
the code portion! Dsc.server will eventually use this stuff internally to
maintain (optimized) state across a load-balanced cluster.

- Kris


"School" <school users.sf.net> wrote in message
news:c5d8on$pfv$1 digitaldaemon.com...
 Kris 提到:
 Do you mean serialization of classes, or serialization of threads? If


 former, then I may be able to help you out. If you meant the latter,


 read on:

 It appears as though Java 1.5 is adopting the full embrace of Doug Lea's
 asynchronous package. Well, at least it looks like Doug's stuff. His


 page is over here:  http://gee.cs.oswego.edu/dl/

 It would be uber-cool if someone were to port even part of Doug's


 synchronization and serialization library over to D ...

 - Kris

 "School" <school users.sf.net> wrote in message
 news:c5b9tf$s3t$1 digitaldaemon.com...

I am just asking that. Although serialization is not a common technique,
I am interested in having one in D.
:-D
--
School, yet another nickname for anonymous.
:D ;-D


Thank you. -- School, yet another nickname for anonymous. :D ;-D

Apr 12 2004
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Kris" <someidiot earthlink.dot.dot.dot.net> wrote in message
news:c5c2fd$20p7$1 digitaldaemon.com...
 Do you mean serialization of classes, or serialization of threads? If the
 former, then I may be able to help you out. If you meant the latter, then
 read on:

 It appears as though Java 1.5 is adopting the full embrace of Doug Lea's
 asynchronous package. Well, at least it looks like Doug's stuff. His home
 page is over here:  http://gee.cs.oswego.edu/dl/

 It would be uber-cool if someone were to port even part of Doug's

 synchronization and serialization library over to D ...

 - Kris

It does look like a nice package, and best of all, it's explicitly public domain. This means it can be transferred to D without worries (except for the two classes that carry a Sun copyright). Anyone want to take the lead on this?
Apr 12 2004
parent reply Ben Hinkle <bhinkle4 juno.com> writes:
Walter wrote:
 "Kris" <someidiot earthlink.dot.dot.dot.net> wrote in message
 news:c5c2fd$20p7$1 digitaldaemon.com...
 
Do you mean serialization of classes, or serialization of threads? If the
former, then I may be able to help you out. If you meant the latter, then
read on:

It appears as though Java 1.5 is adopting the full embrace of Doug Lea's
asynchronous package. Well, at least it looks like Doug's stuff. His home
page is over here:  http://gee.cs.oswego.edu/dl/

It would be uber-cool if someone were to port even part of Doug's

impressive
synchronization and serialization library over to D ...

- Kris

It does look like a nice package, and best of all, it's explicitly public domain. This means it can be transferred to D without worries (except for the two classes that carry a Sun copyright). Anyone want to take the lead on this?

Hmm. I'm surprised no-one seems interested. Are there any HOWTOs or scripts that can help with Java -> D conversions? Not in terms of library calls - I just mean syntax. It shouldn't be too hard to convert this code.
Apr 18 2004
parent reply "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
"Ben Hinkle" <bhinkle4 juno.com> wrote in message
news:c5v9nt$rjv$1 digitaldaemon.com...
 Hmm. I'm surprised no-one seems interested. Are there any HOWTOs or
 scripts that can help with Java -> D conversions? Not in terms of
 library calls - I just mean syntax. It shouldn't be too hard to convert
 this code.

As far as porting Java goes, my meager experience identifies these candidates: 1) replace boolean with bool 2) replace implements with ':' or ',' 3) remove "throws X" syntax 4) replace String with char[] 5) replace package with module 6) replace "(x== null)" with "(x is null)" or "(x === null)" where x is an Object 7) replace "(x != null)" with "(x)" or "(x !== null)" where x is an Object 8) add cast keyword to C-style casts 9) remove Serializable interface, and transient keyword FWIW, I felt that the tricky part would be reworking some of the "dependencies" on how the Java notify and interrupt stuff operates. For example, here's one of Doug's quotes: "Why do so many methods perform notify within InterruptedException catches? Because when notify's and interrupt's happen at about the same time, JVMs are currently free to treat them independently, so a notified thread could return out as interrupted. In classes using notify rather than notifyAll, the extra notify in the catch clause is a safeguard to ensure that a non-interrupted thread, if one exists, will be notified. See my CPJ book for more details. " Then there's the D Thread class to consider: it currently does a linear array search when asked for the current Thread object. I figured that would need to be reworked for speed (Thread.currentThread() is employed in Doug's code). Not a big deal, except for getting it back into Phobos. While at it, I would have tried to remove Thread.pause and Thread.resume since there are reasonable arguments as to why they're prone to deadlock. Perhaps the biggest job in porting this stuff would be thorough testing. I really, really, hope you'll take this on <g>
Apr 18 2004
parent reply Mike Swieton <mike swieton.net> writes:
On Sun, 18 Apr 2004 19:34:31 -0700, Kris wrote:

 "Ben Hinkle" <bhinkle4 juno.com> wrote in message
 news:c5v9nt$rjv$1 digitaldaemon.com...
 Hmm. I'm surprised no-one seems interested. Are there any HOWTOs or
 scripts that can help with Java -> D conversions? Not in terms of
 library calls - I just mean syntax. It shouldn't be too hard to convert
 this code.

As far as porting Java goes, my meager experience identifies these candidates: 1) replace boolean with bool 2) replace implements with ':' or ',' 3) remove "throws X" syntax <snip> Perhaps the biggest job in porting this stuff would be thorough testing. I really, really, hope you'll take this on <g>

I would be interested in working on it. However, I highly recomend against any sort of mechanical (whether by human or machine) porting of his code: there are too many subtleties involved. The first issue that comes to mind is that D's (*nux) threading code is built on pthreads, the behavior of which is not fully consistent across platforms (it's *damn* close, but not 100%). This bit us in the ass over here a few months back. I'm hesitant to take on this alone, as it's a big can of worms that's not really my domain. Anyone interested in jumping in with me? Mike Swieton __ It's kind of fun to do the impossible. - Walt Disney
Apr 18 2004
next sibling parent "Ben Hinkle" <bhinkle4 juno.com> writes:
 I would be interested in working on it. However, I highly recomend against

 sort of mechanical (whether by human or machine) porting of his code:

 are too many subtleties involved. The first issue that comes to mind is

 D's (*nux) threading code is built on pthreads, the behavior of which is

 fully consistent across platforms (it's *damn* close, but not 100%). This

 us in the ass over here a few months back.

I wouldn't worry about making the threading cross-platform (yet). Any platform specific stuff that really needs to be there can be version'ed. I bet Walter would be very open to tweaks to std.thread to support this project, too.
 I'm hesitant to take on this alone, as it's a big can of worms that's not
 really my domain. Anyone interested in jumping in with me?

Do as much as you like/want/can, is my attitude. I've downloaded it and started looking at it. If we stick it up on www.dsource.org it should generate some more interest.
Apr 19 2004
prev sibling parent Sean Kelly <sean ffwd.cx> writes:
Mike Swieton wrote:
 On Sun, 18 Apr 2004 19:34:31 -0700, Kris wrote:
 
 I would be interested in working on it. However, I highly recomend against any
 sort of mechanical (whether by human or machine) porting of his code: there
 are too many subtleties involved. The first issue that comes to mind is that
 D's (*nux) threading code is built on pthreads, the behavior of which is not
 fully consistent across platforms (it's *damn* close, but not 100%). This bit
 us in the ass over here a few months back.

Even worse, winthreads are quite different from pthreads in some cases., and lack features present in pthreads in other cases. I'd suggest using the Boost threading library as a template for writing reliable cross-platform thread code. I've also got one or two sources for public domain condvar and other algorithms not implemented in Windows around here somewhere if licensing or copyright concerns are an issue.
 
 I'm hesitant to take on this alone, as it's a big can of worms that's not
 really my domain. Anyone interested in jumping in with me?

This is something I've been meaning to do but I've just been to darn busy. But I'd love to see "real" threading support in D. Sean
Apr 26 2004
prev sibling next sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
School wrote:
 I am just asking that. Although serialization is not a common technique,
 I am interested in having one in D.
 :-D

To give Kris's Dsc a run for its money (sorry Kris...) I've added an experimental serialization support to std.stream at http://home.comcast.net/~benhinkle/stream.d Comments welcome. Here's an example of how to use it: module mymodule; import std.stream; // Sample serializable class class Foo : Serializable { int some_field; // sample field void write(Stream s) { s.write(some_field); // write out object state } static Object read(Stream s, char[] id) { Foo obj = new Foo(); s.read(obj.some_field); return obj; } const char[] sID = "mymodule.Foo 1"; // verison 1 of mymodule.Foo char[] serializationID() {return sID;} } static this() { registerDeserializeFcn(Foo.sID, &Foo.read); // can register other handlers for past versions if any... } int main() { // let's write out an instance of Foo File f = new File("test.txt",FileMode.Out); Foo o=new Foo(); o.some_field = 42; f.write(o); f.close(); // let's try to read in what we wrote f = new File("test.txt",FileMode.In); Foo p = (Foo)f.read(); printf("p = %p, some_field = %d\n",p,p.some_field); return 0; }
Apr 12 2004
next sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Ben Hinkle wrote:

 School wrote:

 I am just asking that. Although serialization is not a common technique,
 I am interested in having one in D.
 :-D

To give Kris's Dsc a run for its money (sorry Kris...) I've added an experimental serialization support to std.stream at http://home.comcast.net/~benhinkle/stream.d Comments welcome. Here's an example of how to use it:

should link to one object. Something like this: //write pseudo if object already serialized (for this session) then //hint - use a map only write it's ID else write ID and object //read pseudo if object already built then assign object from the map of built objects else read and create object -- -Anderson: http://badmama.com.au/~anderson/
Apr 12 2004
parent Ben Hinkle <bhinkle4 juno.com> writes:
On Tue, 13 Apr 2004 09:13:03 +0800, J Anderson
<REMOVEanderson badmama.com.au> wrote:

Ben Hinkle wrote:

 School wrote:

 I am just asking that. Although serialization is not a common technique,
 I am interested in having one in D.
 :-D

To give Kris's Dsc a run for its money (sorry Kris...) I've added an experimental serialization support to std.stream at http://home.comcast.net/~benhinkle/stream.d Comments welcome. Here's an example of how to use it:

should link to one object. Something like this: //write pseudo if object already serialized (for this session) then //hint - use a map only write it's ID else write ID and object //read pseudo if object already built then assign object from the map of built objects else read and create object

Hmm. maybe. I thought about that briefly but decided against it since 1) it adds complexity 2) there are probably several caching strategies that could be used 3) it can be implemented on top of basic serialization 4) D doesn't have weak references (AFAIK) so no deserialized object could be garbage collected until the map of built objects is cleared. If an object or class (or classes) wants to cache then they have to cooperate. There is no default serialization implementation like in Java so user classes have to explicitly decide what they want to do. For example (assuming casting to interfaces is not evil...) class Foo { Object x,y; // assume x and y always refer to the same object void write(Stream s) { s.write(cast(Serializable)x); // ignore null for simplicity } static Object read(Stream s, char[] id) { Foo obj = new Foo(); obj.x = s.read(); obj.y = x; return obj; } }
Apr 12 2004
prev sibling parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
"Ben Hinkle" <bhinkle4 juno.com> wrote in message
news:c5fdg4$1k5v$1 digitaldaemon.com...
 To give Kris's Dsc a run for its money (sorry Kris...) I've added
 an experimental serialization support to std.stream at

Nice one Ben! Dsc registration is similar, but it registers the Object itself rather than a delegate/id pair. While I'm completely paranoid about allocating memory on a request-by-request basis within a server (Dsc.server allocates *zero* memory per request), I don't care a hoot about instantiating a living Object for registration purposes :-) Likewise, Dsc does nothing further than provide a framework for serialization. Over time it will gain all sorts of nifty distributed caching mechanisms, that one can leverage the hell out of for high-performance clustered environments. However, in such an environment one should not be tossing Object-trees around the place (yeehaw!); so Dsc will not "encourage" such behavior with cool mechanisms for doing just that. That's not to say there's not a need for serializing Object forests ... it's just not the focus for Dsc, and its a tough nut to crack whereby you can satisfy everyone. - Kris
Apr 13 2004
prev sibling parent reply h3r3tic <h3r3tic_member pathlink.com> writes:
yesterday i've independently coded a serialization lib... though its probably
ugly it may come to be useful in the struggle to create sth better :)
readme included.
Apr 13 2004
next sibling parent reply h3r3tic <h3r3tic_member pathlink.com> writes:
whooops...

change the file objfactory.d:
line  5:	static char[][ClassInfo]			o2n;
line 17:			factory.o2n[T.classinfo]		= T.classinfo.name;
line 23:			factory.o2n[T.classinfo]	= name;
line 29:		return o2n[obj.classinfo];

sorry, guess i was too sleepy...
Apr 13 2004
parent reply h3r3tic <h3r3tic_member pathlink.com> writes:
k, here's a slightly modified version in a slightly different zip archive
Apr 13 2004
parent reply h3r3tic <h3r3tic_member pathlink.com> writes:
please ignore the comment in main.d, line 120. i thought i had deleted it...
spotted sth else was causing the problem i mention there. sorry
Apr 13 2004
parent School <school users.sf.net> writes:
h3r3tic 提到:
 please ignore the comment in main.d, line 120. i thought i had deleted it...
 spotted sth else was causing the problem i mention there. sorry
 
 

-- School, yet another nickname for anonymous. :D ;-D
Apr 14 2004
prev sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
hmm. what's a rar file and can you post the files in something 
more conventional? something that WinZip can read, for instance.
Do I need to upgrade my WinZip?
thanks,
-Ben

On Tue, 13 Apr 2004 08:08:48 +0000 (UTC), h3r3tic
<h3r3tic_member pathlink.com> wrote:

yesterday i've independently coded a serialization lib... though its probably
ugly it may come to be useful in the struggle to create sth better :)
readme included.

Apr 13 2004
parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Ben Hinkle wrote:

hmm. what's a rar file and can you post the files in something 
more conventional? something that WinZip can read, for instance.
Do I need to upgrade my WinZip?
thanks,
-Ben

  

winase (winase does many formats). Everyone uses zip and it doesn't seem people are willing to upgrade to/pick a (slightly) better format (of which there are many competing factions).
On Tue, 13 Apr 2004 08:08:48 +0000 (UTC), h3r3tic
<h3r3tic_member pathlink.com> wrote:
  

yesterday i've independently coded a serialization lib... though its probably
ugly it may come to be useful in the struggle to create sth better :)
readme included.

    


-- -Anderson: http://badmama.com.au/~anderson/
Apr 13 2004
next sibling parent reply Stephan Wienczny <wienczny web.de> writes:
J Anderson wrote:
 Ben Hinkle wrote:
 
 Rar is a *slightly* better compression scheme then zip.  Get winrar or 
 winase (winase does many formats).  Everyone uses zip and it doesn't 
 seem people are willing to upgrade to/pick a (slightly) better format 
 (of which there are many competing factions).
 

Winase? Google suggest Winace did you mean that? If you want a good compression for text files, you should use bzip2 Stephan
Apr 13 2004
next sibling parent School <school users.sf.net> writes:
Stephan Wienczny wrote:
 J Anderson wrote:
 
 Ben Hinkle wrote:

 Rar is a *slightly* better compression scheme then zip.  Get winrar or 
 winase (winase does many formats).  Everyone uses zip and it doesn't 
 seem people are willing to upgrade to/pick a (slightly) better format 
 (of which there are many competing factions).

Winase? Google suggest Winace did you mean that? If you want a good compression for text files, you should use bzip2 Stephan

-- School, yet another nickname for anonymous. :D ;-D
Apr 13 2004
prev sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Stephan Wienczny wrote:

 J Anderson wrote:

 Ben Hinkle wrote:

 Rar is a *slightly* better compression scheme then zip.  Get winrar 
 or winase (winase does many formats).  Everyone uses zip and it 
 doesn't seem people are willing to upgrade to/pick a (slightly) 
 better format (of which there are many competing factions).

Winase? Google suggest Winace did you mean that? If you want a good compression for text files, you should use bzip2 Stephan

sorry, yeah. -- -Anderson: http://badmama.com.au/~anderson/
Apr 13 2004
parent J Anderson <REMOVEanderson badmama.com.au> writes:
 Java's serialization API goes this route. I haven't used it but it 
 looks very easy. The Effective Java book, though, cautions against 
 relying too heavily on the default behaviors.

Well of course you have to be aware of what is happening but much of the time, but you can always overload the default behaviour, and your back where we started before.
 The basic warning was that the default serialization scheme really 
 nails down your class implementation and it's easy to turn it on and 
 not realize you've just forced yourself into supporting a given 
 implementation. It's ironic - the more features it has the more 
 problematic it becomes

I also think you should be able to plugin your own handlers into the serialization engine but that's a different issue and doesn't have to do with default serialization.
Also I remember (but I haven't checked if it is still true) that the doc on the
Java serialization basically said not to use it since the format will change in
the future.
  

If you want to play around with serializing/deserializing graphs of objects
feel free to modify the stream.d code 

classes in the stream set. All I argue is that it should be in the standard somewhere. Parhaps called mapSerialization.
- if people/Walter like the result it could get into phobos. I personally have
no idea if Walter is paying attention to all this stream stuff anyway.

-Ben

-Anderson: http://badmama.com.au/~anderson/
Apr 13 2004
prev sibling parent reply "Phill" <phill pacific.net.au> writes:
"J Anderson" <REMOVEanderson badmama.com.au> wrote in message
news:c5gm7u$iac$1 digitaldaemon.com...
 Ben Hinkle wrote:

hmm. what's a rar file and can you post the files in something
more conventional? something that WinZip can read, for instance.
Do I need to upgrade my WinZip?
thanks,
-Ben

winase (winase does many formats). Everyone uses zip and it doesn't seem people are willing to upgrade to/pick a (slightly) better format (of which there are many competing factions).

Am I right in thinking that if you have Apache or Tomcat, you can just place the .rar file in one of your servers folders, and when you retstart the server the file unpacks itself? Or am I confused with something else? Phill. --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.656 / Virus Database: 421 - Release Date: 4/9/2004
Apr 14 2004
parent reply Brad Anderson <brad dsource.dot.org> writes:
Phill wrote:
 
 Am I right in thinking that if you have Apache or
 Tomcat, you can just place the .rar file in one of
 your servers folders, and when you retstart the
 server the file unpacks itself? Or am I confused
 with something else?

.war is the file type you're thinking of. .jar is a Java Archive .war is a Web Archive
Apr 15 2004
parent "Phill" <phill pacific.net.au> writes:
"Brad Anderson" <brad dsource.dot.org> wrote in message
news:c5nrt5$2210$1 digitaldaemon.com...
 Phill wrote:
 Am I right in thinking that if you have Apache or
 Tomcat, you can just place the .rar file in one of
 your servers folders, and when you retstart the
 server the file unpacks itself? Or am I confused
 with something else?

.war is the file type you're thinking of. .jar is a Java Archive .war is a Web Archive

Ah thats right, thanks, its been a while since I mucked around with Tomcat. Phill. --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.659 / Virus Database: 423 - Release Date: 4/15/2004
Apr 16 2004