digitalmars.D.bugs - [Issue 2137] New: Data not compressed on fly when adding to zip archive
- d-bugmail puremagic.com (30/30) Jun 02 2008 http://d.puremagic.com/issues/show_bug.cgi?id=2137
- d-bugmail puremagic.com (4/4) Aug 14 2008 http://d.puremagic.com/issues/show_bug.cgi?id=2137
- d-bugmail puremagic.com (6/6) Aug 15 2008 http://d.puremagic.com/issues/show_bug.cgi?id=2137
- d-bugmail puremagic.com (8/8) Aug 15 2008 http://d.puremagic.com/issues/show_bug.cgi?id=2137
http://d.puremagic.com/issues/show_bug.cgi?id=2137 Summary: Data not compressed on fly when adding to zip archive Product: D Version: 2.014 Platform: PC OS/Version: All Status: NEW Severity: minor Priority: P2 Component: Phobos AssignedTo: bugzilla digitalmars.com ReportedBy: dsimcha yahoo.com The fact that, when an ArchiveMember is added via addMember() to a ZipArchive, the data in expandedData is not compressed on the fly, makes it very difficult to create large zip files without running out of memory. I have created a simple workaround by moving the following code from the build() method to the addMember() method: de.crc32 = std.zlib.crc32(0, cast(void[])de.expandedData); de.compressedData = cast(ubyte[])std.zlib.compress(cast(void[]))de.expandedData; de.compressedData = de.compressedData[2 .. de.compressedData.length - 4]; de.compressedSize = de.compressedData.length; de.expandedSize = de.expandedData.length; I also added the following to free the memory previously occupied by expandedData: de.expandedData=null; This is actually proving to be a significant issue for me, as I am attempting to use std.zip to archive large amounts of DNA sequence, which compresses extremely well, but won't fit in memory uncompressed. --
Jun 02 2008
http://d.puremagic.com/issues/show_bug.cgi?id=2137 Could you please email me the diffs? Thanks! --
Aug 14 2008
http://d.puremagic.com/issues/show_bug.cgi?id=2137 Created an attachment (id=269) --> (http://d.puremagic.com/issues/attachment.cgi?id=269&action=view) Diffs to make std.zip compress on fly. --
Aug 15 2008
http://d.puremagic.com/issues/show_bug.cgi?id=2137 Here they are. Note that I had changed some style stuff to make the code more readable for me when I was trying to understand it, so a little of that may show up in the diffs. Also, I hadn't bothered to include the switch statement to get store mode to work because I didn't need it, but this should be trivial to put back. --
Aug 15 2008