www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Writing Classes to File

reply Jeremy <Jeremy_member pathlink.com> writes:
Is there a way I can just write a whole class to a file?

For example, why can't I do:

file.writeExact(someClass, someClass.sizeof);

This seems to just write 4 bytes, which must be the pointer.

The compiler complains when I do this:

file.writeExact(*someClass, (*someClass).sizeof);

Saying that '*' is only used infront of pointers -- but aren't class always
represented with pointers?

Anyway, I want to save a class to file, so I can load it back. I was hoping
there was a way to do this without having to write/read each element of the
class seperately.
May 08 2006
next sibling parent reply BCS <BCS pathlink.com> writes:
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?
 
 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.
 
 

Then I give the class a DumpToFile method (generally named something else) that copies the data to the struct and does the writeExact thing. A readExact and revers translation do the other end. This has the advantage of providing a nice place to do some translations like endean swaps and recursive decent of trees (dump each child, recording where it is, place a FILE pointer in the struct and then dump the struct). The biggest catch is keeping the struct<->class translators up to date with new rev of the class. <script:coffee author.GetOn(SoapBox);)> But that beings up my old dead horse: the "witheach" statement http://www.digitalmars.com/d/archives/digitalmars/D/32232.html <script:coffee author.GetOff(SoapBox);)>
May 08 2006
parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <e3ntd5$1nnb$1 digitaldaemon.com>, BCS says...
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?
 
 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.
 
 

Then I give the class a DumpToFile method (generally named something else) that copies the data to the struct and does the writeExact thing. A readExact and revers translation do the other end. This has the advantage of providing a nice place to do some translations like endean swaps and recursive decent of trees (dump each child, recording where it is, place a FILE pointer in the struct and then dump the struct). The biggest catch is keeping the struct<->class translators up to date with new rev of the class. <script:coffee author.GetOn(SoapBox);)> But that beings up my old dead horse: the "witheach" statement http://www.digitalmars.com/d/archives/digitalmars/D/32232.html <script:coffee author.GetOff(SoapBox);)>

So, could you just include the struct inside your class? e.g.: struct ClassVars { int x, y; int health; }; class someClass { private: ClassVars myVars; .. public: void saveTo(BufferedFile someFile) { someFile.writeExact(&myVars, myVars.sizeof); } }; And in processing that class (e.g. did this character gain some health) -- you could check myVars.health etc.
May 08 2006
parent reply BCS <BCS_member pathlink.com> writes:
In article <e3o17r$1un7$1 digitaldaemon.com>, Jeremy says...
In article <e3ntd5$1nnb$1 digitaldaemon.com>, BCS says...
My solution is to declare a struct that has related data members in it. 
Then I give the class a DumpToFile method (generally named something 
else) that copies the data to the struct and does the writeExact thing. 


The biggest catch is keeping the struct<->class translators up to date 
with new rev of the class.

<script:coffee author.GetOn(SoapBox);)>

But that beings up my old dead horse: the "witheach" statement
http://www.digitalmars.com/d/archives/digitalmars/D/32232.html

<script:coffee author.GetOff(SoapBox);)>

So, could you just include the struct inside your class?

that would work and would reduce the different revs catch.
e.g.:

struct ClassVars {
int x, y;
int health;
};

I'd put the struct in the class scope as well class { static sreuct args { } args a; }
class someClass {
private:
ClassVars myVars;
..
public:
void saveTo(BufferedFile someFile) {
someFile.writeExact(&myVars, myVars.sizeof);
}
};

And in processing that class (e.g. did this character gain some health) -- you
could check myVars.health etc.

But Ohh dose that get ungly. OTOH might not be to bad if you want to spilt out some sets of args.
May 08 2006
parent reply Chad J <gamerChad _spamIsBad_gmail.com> writes:
I think you might want to check out http://www.runuo.com/
It emulates a server for the massively multiplayer game Ultima Online. 
I mention it only because it has a system for doing just this sort of 
thing - they call it serializing (saving) and deserializing (loading). 
Their setup may not be any better than the struct thing, but it is an 
alternative, and it's worked for this very large project.  I say it may 
not be any better because I have had my share of frustration with 
serializing and deserializing.  If you want, I'll write an example, but 
it would be quite lengthy (and I have differential equations homework to 
do) so maybe later if you say you want it.

That struct thing is interesting, and I'd like to see what kind of ideas 
there are for handling this task.

Also, beware the object reference...

class Mobile
{
    struct fields
    {
        int x, y;
        int health;

        char[] name;           // <---   There be monsters here!
        Container backpack;    // <---
    }
    fields myVars;

    ...

    void saveTo(BufferedFile someFile) {
        someFile.writeExact(&myVars, myVars.sizeof);

        // better add something like:
        someFile.writeExact(myVars.name.ptr, myVars.name.length);
        backpack.saveTo(someFile);
    }
    // careful when loading...
}

class Container : Item
{
     struct fields
     {
         Item[] contents;
         ...
     }
     fields myVars;
     ...
}

For handling the references, BCS, did you come up with something 
different by any chance?
May 08 2006
parent reply BCS <BCS pathlink.com> writes:
Chad J wrote:
[...]
 
 Also, beware the object reference...
 
 class Mobile
 {
    struct fields
    {
        int x, y;
        int health;
 
        char[] name;           // <---   There be monsters here!
        Container backpack;    // <---
    }
    fields myVars;

 For handling the references, BCS, did you come up with something 
 different by any chance?

I mentioned it a few posts back but to summarize, reference items were dumper to the file first (after recording the current file location) and then file pointers were used in the parent object. This has the odd effect that children end up in the file first but oh well. To help find things I put a header at the start of the file (to begin with it had dummy values) and then when I knew where things ended up, I overwrote the header. I have a linked list example I can post if anyone is interested but it will have to wait till Monday (I'm books until then).
May 09 2006
parent BCS <BCS_member pathlink.com> writes:
In article <e3qg96$2vio$1 digitaldaemon.com>, BCS says...
[...]
I have a linked list example I can post if anyone is interested but it 
will have to wait till Monday (I'm books until then).

Um, I'm not a busy as I thoght. Here it is. works on linux (dmd ~v.153 I think) may need some tweeking for win make shure ./test is not an important file as it get trashed. -------------------------- import std.stdio; import std.stream; int main(char[][] argv) { LLItem List1, List2; // construct a linked list List1 = new LLItem(argv, 0); // open a file Stream str = new File("./test", FileMode.OutNew); long pos, first; // store the "header" pos = str.position; str.writeExact(&first, long.sizeof); // dump list first = List1.DumpToFile(str); // over write header str.position = pos; str.writeExact(&first, long.sizeof); // re open file str.close; str = new File("./test",FileMode.In); // read header str.readExact(&first, long.sizeof); // read list str.position = first; List2 = LLItem.ReadFromFile(str); // print list while(null !is List2) { writef("%d:%s\n", List2.integer, List2.string); List2 = List2.next; } return 0; } class LLItem { char[] string; int integer; LLItem next; private this(){} // turn list of args into a linked list this(char[][] args, int i) { string = args[0]; integer = i; if(1 < args.length) next = new LLItem(args[1..$], i+1); } static struct LLItemDump { int strLength; long string; int integer; long next; } long DumpToFile(Stream outs) { LLItemDump dump; // deal with int dump.integer = integer; // deal with next if(null is next) dump.next = 0; else dump.next = next.DumpToFile(outs); // deal with string dump.strLength = string.length; dump.string = outs.position; outs.writeExact(string.ptr, string.length); // dump long pos = outs.position; outs.writeExact(&dump, LLItemDump.sizeof); return pos; } static LLItem ReadFromFile(Stream ins) { //get item LLItemDump dump; ins.readExact(&dump, LLItemDump.sizeof); LLItem ret = new LLItem; //deal with int ret.integer = dump.integer; // deal with next if(0 == dump.next) ret.next = null; else { ins.position = dump.next; ret.next = LLItem.ReadFromFile(ins); } //deal with string ret.string.length = dump.strLength; ins.position = dump.string; ins.readExact(ret.string.ptr, dump.strLength); return ret; } }
May 09 2006
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

They are, but I think they don't have the same semantics as pointers insofar as the type is concerned. This is what inspired me to make the "isizeof" proposal a few weeks ago, as there's currently no way to get a class' size at compile-time. However, you can get it a run-time via: someClass.classinfo.init.length Sean
May 08 2006
next sibling parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

They are, but I think they don't have the same semantics as pointers insofar as the type is concerned. This is what inspired me to make the "isizeof" proposal a few weeks ago, as there's currently no way to get a class' size at compile-time. However, you can get it a run-time via: someClass.classinfo.init.length Sean

Interesting... so could this be done: file.writeExact(someClass, someClass.classinfo.init.length); to write the entire class to a file? (I'd like to do some zlib compression on it too :-D )
May 08 2006
parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <e3o138$1ujn$1 digitaldaemon.com>, Jeremy says...
In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

They are, but I think they don't have the same semantics as pointers insofar as the type is concerned. This is what inspired me to make the "isizeof" proposal a few weeks ago, as there's currently no way to get a class' size at compile-time. However, you can get it a run-time via: someClass.classinfo.init.length Sean

Interesting... so could this be done: file.writeExact(someClass, someClass.classinfo.init.length); to write the entire class to a file? (I'd like to do some zlib compression on it too :-D )

Just for everyone who reads this later -- the above works. I can write the entire class to a file for later reading/reconstruction with only that writeExact function call. Ahhh, my saving/loading troubles just got much easier :)
May 08 2006
parent Jeremy <Jeremy_member pathlink.com> writes:
Interesting... so could this be done:

file.writeExact(someClass, someClass.classinfo.init.length);

to write the entire class to a file? (I'd like to do some zlib compression on it
too :-D )


OK, so I am able to write the class to file. However, I am writing many objects of the same class to a file and I want to pass it through some compression. What is the best way to do this? 1) Should I allocate memory the size of all my objects, writeExact all my objects to that memory space, and then use zlib compression on that memory space? 2) or write everything to the file, then ZIP up the file? 3) or zlib compress each object's data before writing to the file? (e.g. if I was saving 10 objects, I'd do 10 individual zlib compressions and writes) -- would this result in the same space usage as option (1)?
May 09 2006
prev sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On Mon, 8 May 2006, Jeremy wrote:

 In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

They are, but I think they don't have the same semantics as pointers insofar as the type is concerned. This is what inspired me to make the "isizeof" proposal a few weeks ago, as there's currently no way to get a class' size at compile-time. However, you can get it a run-time via: someClass.classinfo.init.length Sean

Interesting... so could this be done: file.writeExact(someClass, someClass.classinfo.init.length); to write the entire class to a file? (I'd like to do some zlib compression on it too :-D )

Even with this, direct writing of a class like this will only work in the simplest of cases. It won't dive into sub structures, for example, char[]'s. Object serialization for the general case is a lot more complicated. Later, Brad
May 08 2006
parent reply Jeremy <Jeremy_member pathlink.com> writes:
In article <Pine.LNX.4.64.0605082013370.2422 bellevue.puremagic.com>, Brad
Roberts says...
On Mon, 8 May 2006, Jeremy wrote:

 In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

They are, but I think they don't have the same semantics as pointers insofar as the type is concerned. This is what inspired me to make the "isizeof" proposal a few weeks ago, as there's currently no way to get a class' size at compile-time. However, you can get it a run-time via: someClass.classinfo.init.length Sean

Interesting... so could this be done: file.writeExact(someClass, someClass.classinfo.init.length); to write the entire class to a file? (I'd like to do some zlib compression on it too :-D )

Even with this, direct writing of a class like this will only work in the simplest of cases. It won't dive into sub structures, for example, char[]'s. Object serialization for the general case is a lot more complicated. Later, Brad

Hrm... So after I write the class, I should also write the char[]'s seperately with another write function. This shouldn't be too much of a problem... my class consists mostly of floats/ints/doubles
May 08 2006
next sibling parent icee <icee_member pathlink.com> writes:
In article <e3p2me$oee$1 digitaldaemon.com>, Jeremy says...
In article <Pine.LNX.4.64.0605082013370.2422 bellevue.puremagic.com>, Brad
Roberts says...
On Mon, 8 May 2006, Jeremy wrote:

 In article <e3o0d4$1tdc$1 digitaldaemon.com>, Sean Kelly says...
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

They are, but I think they don't have the same semantics as pointers insofar as the type is concerned. This is what inspired me to make the "isizeof" proposal a few weeks ago, as there's currently no way to get a class' size at compile-time. However, you can get it a run-time via: someClass.classinfo.init.length Sean

Interesting... so could this be done: file.writeExact(someClass, someClass.classinfo.init.length); to write the entire class to a file? (I'd like to do some zlib compression on it too :-D )

Even with this, direct writing of a class like this will only work in the simplest of cases. It won't dive into sub structures, for example, char[]'s. Object serialization for the general case is a lot more complicated. Later, Brad

Hrm... So after I write the class, I should also write the char[]'s seperately with another write function. This shouldn't be too much of a problem... my class consists mostly of floats/ints/doubles

offset contents 0 pointer to vtable 4 monitor 8... non-static members from the D abi def. so you also write and read pointer to vtbl, is this safe enough? especially when exchanging between different program?
May 08 2006
prev sibling parent dennis luehring <dl.soluz gmx.net> writes:
Jeremy schrieb:

 Hrm... So after I write the class

a class is called "object" after instantiation ...
May 08 2006
prev sibling parent reply Brad Anderson <brad dsource.org> writes:
Jeremy wrote:
 Is there a way I can just write a whole class to a file?
 
 For example, why can't I do:
 
 file.writeExact(someClass, someClass.sizeof);
 
 This seems to just write 4 bytes, which must be the pointer.
 
 The compiler complains when I do this:
 
 file.writeExact(*someClass, (*someClass).sizeof);
 
 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?
 
 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.
 
 

Would Mango's pickling mechanism work here? http://www.dsource.org/projects/mango http://mango.dsource.org BA
May 08 2006
parent Kyle Furlong <kylefurlong gmail.com> writes:
Brad Anderson wrote:
 Jeremy wrote:
 Is there a way I can just write a whole class to a file?

 For example, why can't I do:

 file.writeExact(someClass, someClass.sizeof);

 This seems to just write 4 bytes, which must be the pointer.

 The compiler complains when I do this:

 file.writeExact(*someClass, (*someClass).sizeof);

 Saying that '*' is only used infront of pointers -- but aren't class always
 represented with pointers?

 Anyway, I want to save a class to file, so I can load it back. I was hoping
 there was a way to do this without having to write/read each element of the
 class seperately.

Would Mango's pickling mechanism work here? http://www.dsource.org/projects/mango http://mango.dsource.org BA

Mango's solution is invasive, but for that technique, very well developed. -- Kyle Furlong // Physics Undergrad, UCSB "D is going wherever the D community wants it to go." - Walter Bright
May 08 2006