www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD2, std.zlib, and gzip

reply Glenn Lewis <nospam nospam.org> writes:
Content-Type: text/plain

If I'm not mistaken, I used to use std.zlib with DMD1, and it was compatible
with gzip.
Now that Andrei's awesome book is available on RoughCuts, I finally figured I
would take the plunge and switch up to DMD2.

I tried writing a file with std.zlib.compress, and gzip could not uncompress it
(both level=6 and level=9).
I tried gzip'ing a file, and std.zlib.uncompress could not uncompress it (see
attached D file).

Note that writing with std.zlib then later reading with std.zlib works just
fine... but am I dreaming that I used to also be able to use 'gzip' on the
resulting files?

Thanks!
-- Glenn Lewis
Jul 22 2009
parent reply "Vladimir Panteleev" <thecybershadow gmail.com> writes:
------------BUqlWPy7AzoYvIUM2Gj9JX
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes
Content-Transfer-Encoding: 8bit

On Thu, 23 Jul 2009 08:40:48 +0300, Glenn Lewis <nospam nospam.org> wrote:

 If I'm not mistaken, I used to use std.zlib with DMD1, and it was  
 compatible with gzip.
 Now that Andrei's awesome book is available on RoughCuts, I finally  
 figured I would take the plunge and switch up to DMD2.

If you compare DMD1 and DMD2's zlib.d, you'll notice that the differences are trivial, and the zlib library itself hasn't changed at all, so nothing could have changed to affect this.
 I tried writing a file with std.zlib.compress, and gzip could not  
 uncompress it (both level=6 and level=9).
 I tried gzip'ing a file, and std.zlib.uncompress could not uncompress it  
 (see attached D file).

What you seem to be confusing is that zlib is a compression algorithm, while gzip is a compression utility and file format. You need to add a header to zlib-compressed data. I've attached a simple module I use to encode/decode gzip files. Hope this helps. -- Best regards, Vladimir mailto:thecybershadow gmail.com ------------BUqlWPy7AzoYvIUM2Gj9JX Content-Disposition: attachment; filename=Gzip.d Content-Type: application/octet-stream; name=Gzip.d Content-Transfer-Encoding: Base64 LyoqCiAqIFNpbXBsZSBHemlwIGNvbXByZXNzaW9uL2RlY29tcHJlc3Npb24uCiAq CiAqIDIwMDcgIFZsYWRpbWlyIFBhbnRlbGVldiA8dGhlY3liZXJzaGFkb3dAZ21h aWwuY29tPgogKi8KCm1vZHVsZSBHemlwOwoKc3RhdGljIGltcG9ydCB6bGliID0g c3RkLnpsaWI7CnN0YXRpYyBpbXBvcnQgc3RkY3JjMzIgPSBjcmMzMjsKZGVidWcg aW1wb3J0IHN0ZC5zdGRpbywgc3RkLmZpbGU7Cgpwcml2YXRlIGVudW0KewoJRlRF WFQgPSAxLAoJRkhDUkMgPSAyLAoJRkVYVFJBID0gNCwKCUZOQU1FID0gOCwKCUZD T01NRU5UID0gMTYKfQoKdWludCBjcmMzMih2b2lkW10gZGF0YSkKewoJdWludCBj cmMgPSBzdGRjcmMzMi5pbml0X2NyYzMyKCk7Cglmb3JlYWNoKHY7Y2FzdCh1Ynl0 ZVtdKWRhdGEpCgkJY3JjID0gc3RkY3JjMzIudXBkYXRlX2NyYzMyKHYsIGNyYyk7 CglyZXR1cm4gfmNyYzsKfQoKdm9pZFtdIGNvbXByZXNzKHZvaWRbXSBkYXRhKQp7 Cgl1Ynl0ZVtdIGhlYWRlcjsKCWhlYWRlci5sZW5ndGggPSAxMDsKCWhlYWRlclsw XSA9IDB4MUY7CgloZWFkZXJbMV0gPSAweDhCOwoJaGVhZGVyWzJdID0gMHgwODsK CWhlYWRlclszLi44XSA9IDA7ICAvLyBUT0RPOiBzZXQgTVRJTUUKCWhlYWRlcls4 XSA9IDQ7CgloZWFkZXJbOV0gPSAzOyAgICAgLy8gVE9ETzogc2V0IE9TCgl1aW50 WzJdIGZvb3RlciA9IFtjcmMzMihkYXRhKSwgZGF0YS5sZW5ndGhdOwoJdWJ5dGVb XSBjb21wcmVzc2VkID0gY2FzdCh1Ynl0ZVtdKXpsaWIuY29tcHJlc3MoZGF0YSwg OSk7CglyZXR1cm4gaGVhZGVyIH4gY29tcHJlc3NlZFsyLi4kLTRdIH4gY2FzdCh1 Ynl0ZVtdKWZvb3RlcjsKfQoKdm9pZFtdIHVuY29tcHJlc3Modm9pZFtdIGRhdGEp CnsKCWFzc2VydChkYXRhLmxlbmd0aD49MTApOwoJdWJ5dGVbXSBoZWFkZXIgPSBj YXN0KHVieXRlW10pZGF0YVswLi4xMF07Cglhc3NlcnQoaGVhZGVyWzBdID09IDB4 MUYpOwoJYXNzZXJ0KGhlYWRlclsxXSA9PSAweDhCKTsKCWFzc2VydChoZWFkZXJb Ml0gPT0gMHgwOCk7Cglhc3NlcnQoaGVhZGVyWzNdID09IDApOyAgLy8gVE9ETzog c3VwcG9ydCAoc2tpcHBpbmcgb2YpIGV4dHJhIGZpZWxkcwoJdm9pZFtdIHVuY29t cHJlc3NlZCA9IHpsaWIudW5jb21wcmVzcyhkYXRhW2hlYWRlci5sZW5ndGguLiQt OF0sIDAsIC0xNSk7Cglhc3NlcnQodW5jb21wcmVzc2VkLmxlbmd0aCA9PSAqY2Fz dCh1aW50KikoJmRhdGFbJC00XSkpOwoJcmV0dXJuIHVuY29tcHJlc3NlZDsKfQo= ------------BUqlWPy7AzoYvIUM2Gj9JX--
Jul 22 2009
parent grauzone <none example.net> writes:
 I've attached a simple module I use to encode/decode gzip files. Hope 
 this helps.

zlib can do this for you. Then you don't need to put up so much pressure on the GC, and there's no need for manual header parsing/generation. Quoting the docs for deflateInit2: "windowBits can also be greater than 15 for optional gzip encoding. Add 16 to windowBits to write a simple gzip header and trailer around the compressed data instead of a zlib wrapper." You can also enable automatic header detection for zlib/gzip decoding in inflateInit2. But I think you'll have to stop using std.zlib to use this functionality.
Jul 23 2009