extracting large zip files

[royg]

I am evaluating VCLZip but get an out of memory error.

The typical zipped file is 65-75Mb with uncompressed data in the 1 - 1.5GB range.

It is a single file in the zip archive but can run to 10M+lines.

The real question, is there a way to read these files, saving portions of say 10-20MB a time to a txt file for later processing?

TIA

Roy

[Kevin]

Which version of Delphi are you using?

[royg]

Currently D2006 but will move to D2009 once other tools have been upgraded

Roy

[Kevin]

Do you get the error when opening up an archive or when you display it with your UI? I mean, if you just set the ZipName and then call ReadZip, do you get the out of memory error? The problem is not how large the compressed files are, it definitely is related to how many files there are in the archive.

I was going to suggest downloading and using the FastMM4 memory manager. But I believe Delphi actually incorporated FastMM4 starting with Delphi 2006. It still might be worth downloading the latest version since there have likely been updates. The old memory manager was very poor and I would always add FastMM to my applications at least in D7 and prior, when there was the chance that an archive with MANY MANY files might be opened. VCLZip creates a structure for each file in the archive when it is opened. The old memory manager would end up with fragmented memory while FastMM4 would not. However, 10 million entries in an archive probably does go beyond the capabilities of even FastMM4.

VCLZip does not have a way (at least not right now) to only read in only part of the archive. That is something I can add to the list of features to add in a future version however.