PureZIP library : ZIP / UNZIP files

[gnozal]

Update [PB4]

- added thread-safe library version

This library exists in two versions :
- the standard version, located in %Purebasic%\PureLibraries\UserLibraries\ ;
- the thread-safe version, located in %Purebasic%\SubSystems\UserLibThreadSafe\PureLibraries\ .
In order to use this library in thread-safe mode (compiler option /THREAD), you have to enable the subsystem 'UserLibThreadSafe' in the PB IDE, or add '/SUBSYSTEM UserLibThreadSafe' to the PBCompiler arguments. In jaPBe, do nothing : it will automatically enable the 'UserLibThreadSafe' subsystem to use the threadsafe versions of the installed userlibraries.

[webbmeister]

Gnozal

I am using the miniUNZIP example supplied with your library to expand a 188MB zip containing one 900MB file. Memory usage goes up well beyond the 1/2 GB mark when unzipping using the compiled exe!! Why is memory usage so high? Surely the unzipped "chunks" are written straight to the destination HDD and not to memory. Any ideas?

Thanks

Paul.

[gnozal]

Gnozal

I am using the miniUNZIP example supplied with your library to expand a 188MB zip containing one 900MB file. Memory usage goes up well beyond the 1/2 GB mark when unzipping using the compiled exe!! Why is memory usage so high? Surely the unzipped "chunks" are written straight to the destination HDD and not to memory. Any ideas?

Yes, during decompression, PureZIP does one AllocateMemory(FileSize) and so does ZLIB's unzReadCurrentFile() ; the extracted file is written from memory to disk with WriteData(). I will see if I can change this, e.g. only allocate a small buffer and decompressing with a loop.

[webbmeister]

Thanks Gnozal, that would be a great help. Also, I am having zipping/ unzipping files that have 0KB size - ie a text file with no content.

[gnozal]

Thanks Gnozal, that would be a great help. Also, I am having zipping/ unzipping files that have 0KB size - ie a text file with no content.

Could you please try this lib : http://people.freenet.de/gnozal/PureZIP_BETA.zip
It should use less memory while decompressing and handle zero length files.

[webbmeister]

That seems a lot more memory friendly. Is there a simple way of checking that a zip has written correctly and is not corrupt?

[gnozal]

That seems a lot more memory friendly.

Thanks. Please test it well (all three extraction functions PureZIP_Archive_Extract()/PureZIP_ExtractFile()/PureZIP_ExtractFiles()) because some critical code has changed.

Is there a simple way of checking that a zip has written correctly and is not corrupt?

A suggestion :
You can get the CRC32 for each file in the ZIP

Code: Select all

; PureZIP example
If PureZIP_Archive_Read("c:\PureBasic400\Program\Test.zip")
  ReturnValue.l = PureZIP_Archive_FindFirst() = #UNZ_OK
  While ReturnValue = #UNZ_OK 
    Debug PureZIP_Archive_FileInfo(@myFileinfo.PureZIP_FileInfo)
    Debug "Filename: " + myFileinfo\FileName
    Debug "Compressed Size: " + Str(myFileinfo\CompressedSize)
    Debug "Uncompressed Size: "+ Str(myFileinfo\unCompressedSize)
    Debug "CRC32 : "+ Hex(myFileinfo\crc32) ; <------------ CRC of uncompressed file
    ReturnValue = PureZIP_Archive_FindNext()
  Wend
  PureZIP_Archive_Close()
EndIf

then use CRCFingerprint32() on the original file and compare both results.

[Inf0Byt3]

Hi gnozal, just tested it and it works very nice. I'll keep testing it...

[webbmeister]

It struggles with the following zip file...

.DS_STORE.ZIP

containing...

.DS_STORE

I get "No Files Extracted". Other than that, perfect!!!

[gnozal]

It struggles with the following zip file...
.DS_STORE.ZIP
containing...
.DS_STORE
I get "No Files Extracted". Other than that, perfect!!!

Should be fixed (wildcard problem) : http://people.freenet.de/gnozal/PureZIP_BETA.zip
I will update both libs (PB3.94 and PB4.00) tomorrow if there are no more bugs found.

[gnozal]

Update to V1.40

Both libs : for PB3.94 and PB4.0x

Changes
- decompression is more memory friendly
- some fixes for zero length files
- fixed wildcard problem
- PB4.0x version : added thread-safe lib (see help how to use it)

[gnozal]

Update to V1.50

Both libs : for PB3.94 and PB4.0x

Changes
- Renamed function PureZIP_SetCallback() to PureZIP_SetProgressionCallback()
- Added function PureZIP_SetCompressionCallback()

Set PureZIP progression callback address for the functions :
- PureZIP_ExtractFiles()
- PureZIP_AddFiles()
The callback is called each time a new file is processed.
Usefull to indicate the progress when many files are packed / unpacked.

Set PureZIP (de)compression callback address for the functions :
- PureZIP_Archive_Compress()
- PureZIP_Archive_Extract()
- PureZIP_ExtractFile()
- PureZIP_AddFile()
- PureZIP_ExtractFiles()
- PureZIP_AddFiles()
The callback is called in realtime during compression or decompression.
Usefull to indicate the progress when a big file is packed / unpacked.

See the new MiniZIP/MiniUNZIP sources for an example using both callbacks.

[oryaaaaa]

Can you do jar(zip) of Firefox in extracting?

Cannot you do it because of Zlib?

[gnozal]

Can you do jar(zip) of Firefox in extracting?
Cannot you do it because of Zlib?

It worked with the java .jar archives I have tested (they are standard ZIP files).

[oryaaaaa]

Thanks

I cannot have done Firefox zip, I will use 7-Zip for command line.