|
|
Post by mycwcgr on Oct 16, 2003 8:15:37 GMT -5
My program use VCLZIP PRO 3.03.
One of users ran a few tests of my program but
The test should backup approx. 34GB of data.
The tool uses approx 675MB RAM (incl. virtual Memory) and 1 - 2% CPU time...
OS System:
P4 2.6 GHZ 260 MB RAM (120GB HD)
WINDOWS 2003 Server
But Uncompressed, Compressed, and Archive file sizes can be up to 2^63-1 bytes in length (over 8,000,000,000 gig) in VCLZIP PRO 3.03
Could you tell me how much size your software could reach in test?
|
|
|
|
Post by Kevin on Oct 16, 2003 14:36:38 GMT -5
It's not the size of the files at all, it is the number of files. While theoretically, it is possible to compress up to 2147483647 files into an archive there is a memory feasibility limit which, of course, depends on the machine that is creating the archive. A small index entry for each file is kept in memory and this is what is driving up the memory usage when you try to zip really large number of files.
One problem with this is actually what has been called a bug in the Borland Memory Manager when an app creates lots of memory allocations. There is memory creep and fragmentation.
One recommendation that I have that works quite well is to use the MultiMM Memory Manager. This is free and the two files associated with this are actually included in the Delphi Zip Demo in the KPDEMOSD_PRO.EXE file on the web site. They are MultiMM.pas and HPMM.PAS.
What you need to do is include MultiMM as the very first entry in your PROJECT source USES list. That's not your main form's uses list, that's your PROJECT source. Make sure these two files are where your project can find them. That's all you need to do. You should see a dramatic improvement in memory usage. This may not eliminate your problem depending on just how many files you are trying to zip, but it might possibly.
I plan to make MultiMM available on the website soon with some explanation about why and how to use them.
I do plan to see how feasible it might be to eliminate as much of the memory requirements as possible in a future maintenance release.
BTW, I have had the same memory problem when zipping lots of files with PKZip for Windows.
Kevin
|
|
|
|
Post by mycwcgr on Oct 16, 2003 18:08:17 GMT -5
If I set MultiMode Property to mmBlocks, and every file block size is 2G. by doing so, could the backup size be over 34G ?
|
|
|
|
Post by Kevin on Oct 16, 2003 18:39:30 GMT -5
Yes, but the memory issue, as a result of the number of files being compressed, still stands.
|
|
|
|
Post by Kevin on Oct 16, 2003 19:25:25 GMT -5
BTW, just curious, about how many files are you trying to zip into the 32+gig archive?
|
|
|
|
Post by mycwcgr on Oct 17, 2003 6:20:21 GMT -5
This is only my one of users testing, I don't do that!
|
|
|
|
Post by Kevin on Oct 17, 2003 10:57:48 GMT -5
OK, well let me know if you give MultiMM memory manager a try and whether it helps with this particular test or not. I'd be interested to know.
Thanks!
Kevin
|
|
|
|
Post by Jon L Dee on Oct 17, 2003 10:58:53 GMT -5
Kevin,
Do you recommend MultiMM for D7?
Jon
|
|
|
|
Post by Kevin on Oct 17, 2003 11:57:04 GMT -5
Definitely. All versions. The same problem seems to be there in all of the versions.
|
|
|
|
Post by mycwcgr on Oct 29, 2003 7:36:09 GMT -5
Could you tell me whether MultiMM.pas and HPMM.PAS is only valid in these Delphi program that use VclZip Pro?
|
|
|
|
Post by Kevin on Oct 29, 2003 7:47:52 GMT -5
MultiMM is useful for ANY Delphi program (or BCB).
|
|
