www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Binary with appended zip-file

reply Henning Hasemann <hhasemann web.de> writes:
Okay this might be of topic, if so sorry for posting this.

I'm about to build a little SDL-application which should run completely
from a single binary.
So my idea was to simple append a .zip-file with all other needed
files (images etc..) to it and extract it in RAM with std.zip.

The big problem is: How does the program know where in the binary the zip-file
begins?
Naturally this information is only available *after* compilation
(its the size of the plain binary then) but how can I insert
it to the program after it has been compiled?

I could reserve the space for a size-variable and compile with the dropped-in
value a second time but maybe someone has a better idea?

TIA
Henning
May 20 2007
next sibling parent Frank Benoit <keinfarbton googlemail.com> writes:
Henning Hasemann schrieb:
 Okay this might be of topic, if so sorry for posting this.
 
 I'm about to build a little SDL-application which should run completely
 from a single binary.
 So my idea was to simple append a .zip-file with all other needed
 files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the zip-file
 begins?
 Naturally this information is only available *after* compilation
 (its the size of the plain binary then) but how can I insert
 it to the program after it has been compiled?
 
 I could reserve the space for a size-variable and compile with the dropped-in
 value a second time but maybe someone has a better idea?
 
 TIA
 Henning

There are two binary to D converter i know of: * www.dprogramming.com * in the tioport project, the bintod folder So you can compile the zip into a ubyte[] and link it to your sources, then your app can simply read it.
May 20 2007
prev sibling next sibling parent Deewiant <deewiant.doesnotlike.spam gmail.com> writes:
Henning Hasemann wrote:
 I'm about to build a little SDL-application which should run completely from 
 a single binary. So my idea was to simple append a .zip-file with all other 
 needed files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the 
 zip-file begins? Naturally this information is only available *after* 
 compilation (its the size of the plain binary then) but how can I insert it 
 to the program after it has been compiled?
 
 I could reserve the space for a size-variable and compile with the dropped-in
  value a second time but maybe someone has a better idea?
 

http://www.digitalmars.com/d/expression.html#ImportExpression auto z = new ZipArchive(import("foo.zip"));
May 20 2007
prev sibling next sibling parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
(Note: This was typed before I read the post by Deewiant reminding me 
that the relatively new import-expression is pretty much what I'm 
hacking together here, and much cleaner (even though it returns char[] 
instead of void[]/ubyte[]/byte[]). I'll post anyway, just so I don't 
feel I wasted too much time :( )

Henning Hasemann wrote:
 I'm about to build a little SDL-application which should run completely
 from a single binary.
 So my idea was to simple append a .zip-file with all other needed
 files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the zip-file
 begins?
 Naturally this information is only available *after* compilation
 (its the size of the plain binary then) but how can I insert
 it to the program after it has been compiled?
 
 I could reserve the space for a size-variable and compile with the dropped-in
 value a second time but maybe someone has a better idea?

A hack that should work: use an assembler to include the .zip into an object file and link it in. Also, Manually construct an array data structure pointing to it (with correct length). Technically, this isn't what you asked for -- this won't be appending but embedding. IMHO it's a "cleaner" solution though. Step-by-step: (Note: I didn't test this, so there may be some mistakes in these steps) 1) Copy-paste the following: ===== section .rodata zip_start: incbin "resources.zip" zip_end: global zip_data zip_data: dd zip_end - zip_start ; Length dd zip_start ; Pointer ===== to a file (let's say zip-resources.asm). (For a 64-bit compiler (such as GDC for AMD64) replace both instances of 'dd' by 'dq') For object files other than ELF (the Linux default), you may need to add an underscore prefix to zip_data (or pass "--prefix _" to nasm when assembling) if you get link errors without it. 2) Install nasm (from nasm.sourceforge.net or your Linux package manager). (It should also work with yasm. In fact, yasm may be required to generate 64-bit object files unless you want to use the CVS version of nasm. I will be assuming nasm for the rest though, since that's what I know) 3) To assemble the file: nasm -f elf -o zip-resources.o zip-resources.asm 3a) For windows, replace 'nasm' with 'nasmw'. 3b) For DMD/win, replace 'elf' with 'obj'. You may also want to change the extension to ".obj" (also in the next step) to better fit the naming scheme used by DMD/win itself. 3c) For GDC/win(mingw), replace 'elf' with 'coff'. (Note: I'm not sure about 3b & 3c, but I /think/ those are the options to use) You'll probably want to add this command to your build system (to be executed after the resources.zip is updated, obviously). I'm not sure how to do this with build/rebuild, but it's trivial with a makefile. 4) Add zip-resources.o to your linker command line as one of the object files to link. 5) In your source, add the following line: extern extern(C) void[] zip_data; It's 'extern' because it's defined in another file, 'extern(C)' because I don't feel like figuring out what the name mangles to ;), and void[] because that's all std.zip requires and manipulating it manually should be discouraged. Once the new constness system is implemented & working, "final invariant" may be added as well if you want to be nice and const-correct. 6) Elsewhere, add auto resources = new std.zip.ZipArchive(zip_data); Of course you're free to rename variables and assembly labels if you prefer, just be sure to change them in all places where they're mentioned.
May 20 2007
parent Frank Benoit <keinfarbton googlemail.com> writes:
Frits van Bommel schrieb:
 (Note: This was typed before I read the post by Deewiant reminding me
 that the relatively new import-expression is pretty much what I'm
 hacking together here, and much cleaner (even though it returns char[]
 instead of void[]/ubyte[]/byte[]). I'll post anyway, just so I don't
 feel I wasted too much time :( )

Hehe, i think i can now drop my bintod in tioport :) Thats cool.
May 20 2007
prev sibling next sibling parent reply Henning Hasemann <hhasemann web.de> writes:
 There are two binary to D converter i know of:
 * www.dprogramming.com
 * in the tioport project, the bintod folder
 
 So you can compile the zip into a ubyte[] and link it to your sources,
 then your app can simply read it.

Thanks a lot I tried the bintod from dprogramming, works great! Exactly what I looked for! Henning
May 20 2007
parent reply BCS <ao pathlink.com> writes:
Reply to Henning,

 There are two binary to D converter i know of:
 * www.dprogramming.com
 * in the tioport project, the bintod folder
 So you can compile the zip into a ubyte[] and link it to your
 sources, then your app can simply read it.
 

Exactly what I looked for! Henning

another solution would be: auto zipstream = new MemoryStream(import("zipfile.zip)); the import("...") loads the file at compile time and stuffs it into the binary as a constant string, that with the memory stream lets you read it like a file
May 20 2007
parent BCS <ao pathlink.com> writes:
Reply to myself,
 
 auto zipstream = new MemoryStream(import("zipfile.zip));
 

note to self: read the whole thread befor posting :b
May 20 2007
prev sibling next sibling parent Henning Hasemann <hhasemann web.de> writes:
Thank you all for your fast answers,
thanks Frits for your detailed work though it shouldnt
be necessary to do it this "hard way".

Ill also try the import() thingy from deewiant,
should save one memory and time consuming step compared to bintod.

Henning
May 20 2007
prev sibling next sibling parent Henning Hasemann <hhasemann web.de> writes:
On Sun, 20 May 2007 19:17:00 +0200
Henning Hasemann <hhasemann web.de> wrote:

 
 Thank you all for your fast answers,
 thanks Frits for your detailed work though it shouldnt
 be necessary to do it this "hard way".
 
 Ill also try the import() thingy from deewiant,
 should save one memory and time consuming step compared to bintod.

whoot! This is dramatically faster regarding compile time *and* easier than bintod. (Clear if you imagine that bintod converts the binary to hex and the compiler converts it back to bin wheras import simply inserts the binary data). Henning
May 20 2007
prev sibling parent Jan Claeys <usenet janc.be> writes:
Op Sun, 20 May 2007 17:41:28 +0200
schreef Henning Hasemann <hhasemann web.de>:

 I'm about to build a little SDL-application which should run
 completely from a single binary.
 So my idea was to simple append a .zip-file with all other needed
 files (images etc..) to it and extract it in RAM with std.zip.
 
 The big problem is: How does the program know where in the binary the
 zip-file begins?

Check the size of the binary at runtime and subtract the size of the .zip file from it? :-) (That's how many virusses did it, back in the days of DOS...) -- Jan Claeys
May 26 2007