www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Writing Classes to File

reply Jeremy <Jeremy_member pathlink.com> writes:
Is there a way I can just write a whole class to a file?

For example, why can't I do:

file.writeExact(someClass, someClass.sizeof);

This seems to just write 4 bytes, which must be the pointer.

The compiler complains when I do this:

file.writeExact(*someClass, (*someClass).sizeof);

Saying that '*' is only used infront of pointers -- but aren't class always
represented with pointers?

Anyway, I want to save a class to file, so I can load it back. I was hoping
there was a way to do this without having to write/read each element of the
class seperately.
May 08 2006
next sibling parent reply BCS <BCS pathlink.com> writes:
Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?
 
 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.
 
 


My solution is to declare a struct that has related data members in it. 
Then I give the class a DumpToFile method (generally named something 
else) that copies the data to the struct and does the writeExact thing. 
A readExact and revers translation do the other end. This has the 
advantage of providing a nice place to do some translations like endean 
swaps and recursive decent of trees (dump each child, recording where it 
is, place a FILE pointer in the struct and then dump the struct).

The biggest catch is keeping the struct<->class translators up to date 
with new rev of the class.

<script:coffee author.GetOn(SoapBox);)>

But that beings up my old dead horse: the "witheach" statement
http://www.digitalmars.com/d/archives/digitalmars/D/32232.html

<script:coffee author.GetOff(SoapBox);)>
May 08 2006
parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <e3ntd5$1nnb$1 digitaldaemon.com>, BCS says...

Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?
 
 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.
 
 


My solution is to declare a struct that has related data members in it. 
Then I give the class a DumpToFile method (generally named something 
else) that copies the data to the struct and does the writeExact thing. 
A readExact and revers translation do the other end. This has the 
advantage of providing a nice place to do some translations like endean 
swaps and recursive decent of trees (dump each child, recording where it 
is, place a FILE pointer in the struct and then dump the struct).

The biggest catch is keeping the struct<->class translators up to date 
with new rev of the class.

<script:coffee author.GetOn(SoapBox);)>

But that beings up my old dead horse: the "witheach" statement
http://www.digitalmars.com/d/archives/digitalmars/D/32232.html

<script:coffee author.GetOff(SoapBox);)>


So, could you just include the struct inside your class?

e.g.:

struct ClassVars {
int x, y;
int health;
};

class someClass {
private:
ClassVars myVars;
..
public:
void saveTo(BufferedFile someFile) {
someFile.writeExact(&myVars, myVars.sizeof);
}
};

And in processing that class (e.g. did this character gain some health) -- you
could check myVars.health etc.
May 08 2006
parent reply BCS <BCS_member pathlink.com> writes:
In article <e3o17r$1un7$1 digitaldaemon.com>, Jeremy says...

In article <e3ntd5$1nnb$1 digitaldaemon.com>, BCS says...

My solution is to declare a struct that has related data members in it. 
Then I give the class a DumpToFile method (generally named something 
else) that copies the data to the struct and does the writeExact thing. 




[...]

The biggest catch is keeping the struct<->class translators up to date 
with new rev of the class.

<script:coffee author.GetOn(SoapBox);)>

But that beings up my old dead horse: the "witheach" statement
http://www.digitalmars.com/d/archives/digitalmars/D/32232.html

<script:coffee author.GetOff(SoapBox);)>


So, could you just include the struct inside your class?


that would work and would reduce the different revs catch.

e.g.:

struct ClassVars {
int x, y;
int health;
};


I'd put the struct in the class scope as well

class
{
static sreuct args
{
}
args a;

}

class someClass {
private:
ClassVars myVars;
..
public:
void saveTo(BufferedFile someFile) {
someFile.writeExact(&myVars, myVars.sizeof);
}
};

And in processing that class (e.g. did this character gain some health) -- you
could check myVars.health etc.


But Ohh dose that get ungly. OTOH might not be to bad if you want to spilt out
some sets of args.
May 08 2006
parent reply Chad J <gamerChad _spamIsBad_gmail.com> writes:
I think you might want to check out http://www.runuo.com/
It emulates a server for the massively multiplayer game Ultima Online. 
I mention it only because it has a system for doing just this sort of 
thing - they call it serializing (saving) and deserializing (loading). 
Their setup may not be any better than the struct thing, but it is an 
alternative, and it's worked for this very large project.  I say it may 
not be any better because I have had my share of frustration with 
serializing and deserializing.  If you want, I'll write an example, but 
it would be quite lengthy (and I have differential equations homework to 
do) so maybe later if you say you want it.

That struct thing is interesting, and I'd like to see what kind of ideas 
there are for handling this task.

Also, beware the object reference...

class Mobile
{
    struct fields
    {
        int x, y;
        int health;

        char[] name;           // <---   There be monsters here!
        Container backpack;    // <---
    }
    fields myVars;

    ...

    void saveTo(BufferedFile someFile) {
        someFile.writeExact(&myVars, myVars.sizeof);

        // better add something like:
        someFile.writeExact(myVars.name.ptr, myVars.name.length);
        backpack.saveTo(someFile);
    }
    // careful when loading...
}

class Container : Item
{
     struct fields
     {
         Item[] contents;
         ...
     }
     fields myVars;
     ...
}

For handling the references, BCS, did you come up with something 
different by any chance?
May 08 2006
parent reply BCS <BCS pathlink.com> writes:
Chad J wrote:
[...]

 
 Also, beware the object reference...
 
 class Mobile
 {
    struct fields
    {
        int x, y;
        int health;
 
        char[] name;           // <---   There be monsters here!
        Container backpack;    // <---
    }
    fields myVars;


[...]


 For handling the references, BCS, did you come up with something 
 different by any chance?


I mentioned it a few posts back but to summarize, reference items were 
dumper to the file first (after recording the current file location) and 
then file pointers were used in the parent object. This has the odd 
effect that children end up in the file first but oh well. To help find 
things I put a header at the start of the file (to begin with it had 
dummy values) and then when I knew where things ended up, I overwrote 
the header.

I have a linked list example I can post if anyone is interested but it 
will have to wait till Monday (I'm books until then).
May 09 2006
parent BCS <BCS_member pathlink.com> writes:
In article <e3qg96$2vio$1 digitaldaemon.com>, BCS says...
[...]

I have a linked list example I can post if anyone is interested but it 
will have to wait till Monday (I'm books until then).


Um, I'm not a busy as I thoght. Here it is.

works on linux (dmd ~v.153 I think)
may need some tweeking for win
make shure ./test is not an important file as it get trashed.
--------------------------
import std.stdio;
import std.stream;


int main(char[][] argv)
{
LLItem List1, List2;

// construct a linked list
List1 = new LLItem(argv, 0);

// open a file
Stream str = new File("./test", FileMode.OutNew);
long pos, first;

// store the "header"
pos = str.position;
str.writeExact(&first, long.sizeof);

// dump list
first = List1.DumpToFile(str);

// over write header
str.position = pos;
str.writeExact(&first, long.sizeof);

// re open file
str.close;
str = new File("./test",FileMode.In);

// read header
str.readExact(&first, long.sizeof);

// read list
str.position = first;
List2 = LLItem.ReadFromFile(str);

// print list
while(null !is List2)
{
writef("%d:%s\n", List2.integer, List2.string);
List2 = List2.next;
}

return 0;
}




class LLItem
{
char[] string;
int integer;
LLItem next;

private this(){}

// turn list of args into a linked list
this(char[][] args, int i)
{
string = args[0];
integer = i;
if(1 < args.length)
next = new LLItem(args[1..$], i+1);
}

static struct LLItemDump
{
int strLength;
long string;
int integer;
long next;
}

long DumpToFile(Stream outs)
{
LLItemDump dump;

// deal with int
dump.integer = integer;

// deal with next
if(null is next)
dump.next = 0;
else
dump.next = next.DumpToFile(outs);

// deal with string
dump.strLength = string.length;
dump.string = outs.position;
outs.writeExact(string.ptr, string.length);

// dump
long pos = outs.position;
outs.writeExact(&dump, LLItemDump.sizeof);

return pos;
}

static LLItem ReadFromFile(Stream ins)
{
//get item
LLItemDump dump;
ins.readExact(&dump, LLItemDump.sizeof);

LLItem ret = new LLItem;

//deal with int
ret.integer = dump.integer;

// deal with next
if(0 == dump.next)
ret.next = null;
else
{
ins.position = dump.next;
ret.next = LLItem.ReadFromFile(ins);
}

//deal with string
ret.string.length = dump.strLength;
ins.position = dump.string;
ins.readExact(ret.string.ptr, dump.strLength);

return ret;
}
}
May 09 2006
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?


They are, but I think they don't have the same semantics as pointers 
insofar as the type is concerned.  This is what inspired me to make the 
"isizeof" proposal a few weeks ago, as there's currently no way to get a 
class' size at compile-time.  However, you can get it a run-time via:

someClass.classinfo.init.length


Sean
May 08 2006
parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...

Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?


They are, but I think they don't have the same semantics as pointers 
insofar as the type is concerned.  This is what inspired me to make the 
"isizeof" proposal a few weeks ago, as there's currently no way to get a 
class' size at compile-time.  However, you can get it a run-time via:

someClass.classinfo.init.length


Sean


Interesting... so could this be done:

file.writeExact(someClass, someClass.classinfo.init.length);

to write the entire class to a file? (I'd like to do some zlib compression on it
too :-D )
May 08 2006
next sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On Mon, 8 May 2006, Jeremy wrote:


 In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...

Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?


They are, but I think they don't have the same semantics as pointers 
insofar as the type is concerned.  This is what inspired me to make the 
"isizeof" proposal a few weeks ago, as there's currently no way to get a 
class' size at compile-time.  However, you can get it a run-time via:

someClass.classinfo.init.length


Sean


 
 Interesting... so could this be done:
 
 file.writeExact(someClass, someClass.classinfo.init.length);
 
 to write the entire class to a file? (I'd like to do some zlib compression on
it
 too :-D )


Even with this, direct writing of a class like this will only work in the 
simplest of cases.  It won't dive into sub structures, for example, 
char[]'s.  Object serialization for the general case is a lot more 
complicated.

Later,
Brad
May 08 2006
parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <Pine.LNX.4.64.0605082013370.2422 bellevue.puremagic.com>, Brad
Roberts says...

On Mon, 8 May 2006, Jeremy wrote:


 In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...

Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?


They are, but I think they don't have the same semantics as pointers 
insofar as the type is concerned.  This is what inspired me to make the 
"isizeof" proposal a few weeks ago, as there's currently no way to get a 
class' size at compile-time.  However, you can get it a run-time via:

someClass.classinfo.init.length


Sean


 
 Interesting... so could this be done:
 
 file.writeExact(someClass, someClass.classinfo.init.length);
 
 to write the entire class to a file? (I'd like to do some zlib compression on
it
 too :-D )


Even with this, direct writing of a class like this will only work in the 
simplest of cases.  It won't dive into sub structures, for example, 
char[]'s.  Object serialization for the general case is a lot more 
complicated.

Later,
Brad


Hrm... So after I write the class, I should also write the char[]'s seperately
with another write function. This shouldn't be too much of a problem... my class
consists mostly of floats/ints/doubles
May 08 2006
next sibling parent icee <icee_member pathlink.com> writes:
In article <e3p2me$oee$1 digitaldaemon.com>, Jeremy says...

In article <Pine.LNX.4.64.0605082013370.2422 bellevue.puremagic.com>, Brad
Roberts says...

On Mon, 8 May 2006, Jeremy wrote:


 In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...

Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?


They are, but I think they don't have the same semantics as pointers 
insofar as the type is concerned.  This is what inspired me to make the 
"isizeof" proposal a few weeks ago, as there's currently no way to get a 
class' size at compile-time.  However, you can get it a run-time via:

someClass.classinfo.init.length


Sean


 
 Interesting... so could this be done:
 
 file.writeExact(someClass, someClass.classinfo.init.length);
 
 to write the entire class to a file? (I'd like to do some zlib compression on
it
 too :-D )


Even with this, direct writing of a class like this will only work in the 
simplest of cases.  It won't dive into sub structures, for example, 
char[]'s.  Object serialization for the general case is a lot more 
complicated.

Later,
Brad


Hrm... So after I write the class, I should also write the char[]'s seperately
with another write function. This shouldn't be too much of a problem... my class
consists mostly of floats/ints/doubles



offset  contents  
0  pointer to vtable  
4  monitor  
8...  non-static members  

from the D abi def.

so you also write and read pointer to vtbl, is this safe enough? especially when
exchanging between different program?
May 08 2006
prev sibling parent dennis luehring <dl.soluz gmx.net> writes:
Jeremy schrieb:


 Hrm... So after I write the class


a class is called "object" after instantiation ...
May 08 2006
prev sibling parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <e3o138$1ujn$1 digitaldaemon.com>, Jeremy says...

In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...

Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?


They are, but I think they don't have the same semantics as pointers 
insofar as the type is concerned.  This is what inspired me to make the 
"isizeof" proposal a few weeks ago, as there's currently no way to get a 
class' size at compile-time.  However, you can get it a run-time via:

someClass.classinfo.init.length


Sean


Interesting... so could this be done:

file.writeExact(someClass, someClass.classinfo.init.length);

to write the entire class to a file? (I'd like to do some zlib compression on it
too :-D )


Just for everyone who reads this later -- the above works. I can write the
entire class to a file for later reading/reconstruction with only that
writeExact function call. Ahhh, my saving/loading troubles just got much easier
:)
May 08 2006
parent Jeremy <Jeremy_member pathlink.com> writes:
Interesting... so could this be done:

file.writeExact(someClass, someClass.classinfo.init.length);

to write the entire class to a file? (I'd like to do some zlib compression on it
too :-D )




OK, so I am able to write the class to file. However, I am writing many objects
of the same class to a file and I want to pass it through some compression. What
is the best way to do this?

1) Should I allocate memory the size of all my objects, writeExact all my
objects to that memory space, and then use zlib compression on that memory
space?

2) or write everything to the file, then ZIP up the file?

3) or zlib compress each object's data before writing to the file? (e.g. if I
was saving 10 objects, I'd do 10 individual zlib compressions and writes) --
would this result in the same space usage as option (1)?
May 09 2006
prev sibling parent reply Brad Anderson <brad dsource.org> writes:
Jeremy wrote:

 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?
 
 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.
 
 


Would Mango's pickling mechanism work here?

http://www.dsource.org/projects/mango

http://mango.dsource.org


BA
May 08 2006
parent Kyle Furlong <kylefurlong gmail.com> writes:
Brad Anderson wrote:

 Jeremy wrote:

 Is there a way I can just write a whole class to a file?

 For example, why can't I do:

 file.writeExact(someClass, someClass.sizeof);

 This seems to just write 4 bytes, which must be the pointer.

 The compiler complains when I do this:

 file.writeExact(*someClass, (*someClass).sizeof);

 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.


 
 Would Mango's pickling mechanism work here?
 
 http://www.dsource.org/projects/mango
 
 http://mango.dsource.org
 
 
 BA


Mango's solution is invasive, but for that technique, very well developed.

-- 
Kyle Furlong // Physics Undergrad, UCSB

"D is going wherever the D community wants it to go." - Walter Bright
May 08 2006