Page 1 of 6
How to compress with zlib
Posted: Wed Oct 16, 2013 6:39 am
by moob
Hi...
I need some help.
I need open an dds file and compress with zlib and save compressed.
Please any help will be great.
Thanks.
Re: How to compress with zlib
Posted: Wed Oct 16, 2013 7:06 am
by JHPJHP
Check out this thread:
http://www.purebasic.fr/english/viewtop ... 13&t=56969
My last post... open your dds file with ReadFile... WriteData.
Re: How to compress with zlib
Posted: Wed Oct 16, 2013 7:15 am
by moob
thanks but I have test this and I use PB 5.00 and give me an error
Re: How to compress with zlib
Posted: Wed Oct 16, 2013 7:17 am
by JHPJHP
There is a dll version (decompression) near the top of the thread that can be modified for compression, but if you want to post some code - I'll have a look... I'm on PB 5.20 though; not sure if it's using the same zlib library?
Re: How to compress with zlib
Posted: Thu Oct 17, 2013 3:48 am
by moob
I have a problem with this.
Code: Select all
OpenLibrary(0, "zlib1.dll")
file$ = "temp\teste.dds"
ReadFile(0, file$)
SourceLen = Lof(0)
*SourceMem= AllocateMemory(SourceLen)
ReadData(0, *SourceMem,SourceLen)
CloseFile(0)
;calculate buffer, allocate memory and compress
DestLen = ((Sourcelen +(((Sourcelen)/1000)+1)+12))
*DestMem=AllocateMemory(DestLen)
result = CallCFunction(0, "compress", *DestMem, @DestLen, *SourceMem, SourceLen)
;write compressed memory to file
If result = 0
CreateFile(1, "temp\teste2.dds")
WriteData(1, *DestMem, DestLen+1)
CloseFile(1)
EndIf
CloseLibrary(0)
the file "teste.dds" have 2MB, when compressed the size is 56kb, but if I uncompress the file compressed the size is the same 56kb and not 2MB
what I do wrong?
Re: How to compress with zlib
Posted: Thu Oct 17, 2013 4:37 am
by skywalk
I use ImportC and "_compress2".
Code: Select all
ImportC "zlib.lib"
zl_version.i() As "_zlibVersion"
zl_compress2.i(*mTo_i, *lenTo_i, *mFrom_i, lenFrom.i, Level.i) As "_compress2"
zl_uncompress.i(*mTo_i, *lenTo_i, *mFrom_i, lenFrom.i) As "_uncompress"
zl_crc32.i(crc32.i, *buf_i, len.i) As "_crc32"
zl_compressBound.i(lenFrom.i) As "_compressBound" ; Bugs fixed in v1.2.4
EndImport
; compress snippet...
Level = #ZL_BEST_COMPRESSION ;#ZL_DEFAULT_COMPRESSION
If *mFrom
If lenFrom = #PB_Default
lenFrom = MemorySize(*mFrom)
EndIf
; LenTo >= 0.1% of LenFrom + 12 bytes
Protected.i lenTo = zl_compressBound(lenFrom) ; Round(lenFrom * 1.001,#PB_Round_Up) + 12
*mTo = AllocateMemory(lenTo)
If *mTo
*p\err = zl_compress2(*mTo, @lenTo, *mFrom, lenFrom, Level)
If *p\err = #ZL_OK
*mTo = ReAllocateMemory(*mTo, lenTo)
Else
FreeMemory(*mTo)
*mTo = 0
EndIf
EndIf
EndIf
; *mTo = pointer to compressed memory
; uncompress snippet...
; Return lenTo or negative error code.
Protected.i lenFrom = MemorySize(*mFrom)
Protected.i lenTo = MemorySize(*mTo)
*p\err = zl_uncompress(*mTo, @lenTo, *mFrom, lenFrom)
Re: How to compress with zlib
Posted: Thu Oct 17, 2013 4:51 am
by moob
skywalk wrote:I use ImportC and "_compress2".
Code: Select all
ImportC "zlib.lib"
zl_version.i() As "_zlibVersion"
zl_compress2.i(*mTo_i, *lenTo_i, *mFrom_i, lenFrom.i, Level.i) As "_compress2"
zl_uncompress.i(*mTo_i, *lenTo_i, *mFrom_i, lenFrom.i) As "_uncompress"
zl_crc32.i(crc32.i, *buf_i, len.i) As "_crc32"
zl_compressBound.i(lenFrom.i) As "_compressBound" ; Bugs fixed in v1.2.4
EndImport
; compress snippet...
Level = #ZL_BEST_COMPRESSION ;#ZL_DEFAULT_COMPRESSION
If *mFrom
If lenFrom = #PB_Default
lenFrom = MemorySize(*mFrom)
EndIf
; LenTo >= 0.1% of LenFrom + 12 bytes
Protected.i lenTo = zl_compressBound(lenFrom) ; Round(lenFrom * 1.001,#PB_Round_Up) + 12
*mTo = AllocateMemory(lenTo)
If *mTo
*p\err = zl_compress2(*mTo, @lenTo, *mFrom, lenFrom, Level)
If *p\err = #ZL_OK
*mTo = ReAllocateMemory(*mTo, lenTo)
Else
FreeMemory(*mTo)
*mTo = 0
EndIf
EndIf
EndIf
; *mTo = pointer to compressed memory
; uncompress snippet...
; Return lenTo or negative error code.
Protected.i lenFrom = MemorySize(*mFrom)
Protected.i lenTo = MemorySize(*mTo)
*p\err = zl_uncompress(*mTo, @lenTo, *mFrom, lenFrom)
Hi...
this example give me an error "#ZL_BEST_COMPRESSION constant not found" I use PB 5.00
------
I have use this code below also and the result is the same with dll
Code: Select all
CompilerSelect #PB_Compiler_OS
CompilerCase #PB_OS_Linux
ImportC #PB_Compiler_Home + "purelibraries/linux/libraries/zlib.a"
CompilerCase #PB_OS_MacOS
ImportC "/usr/lib/libz.dylib"
CompilerCase #PB_OS_Windows
ImportC "zlib.lib"
CompilerEndSelect
compress2(*dest,*destlen,*source,sourcelen,Level)
EndImport
File$ = "temp\teste.dds"
ReadFile(0, File$)
sourcelen = Lof(0)
*SourceMem= AllocateMemory(sourcelen)
ReadData(0, *SourceMem,sourcelen)
CloseFile(0)
destLen = ((sourcelen +(((sourcelen)/1000)+1)+12))
*DestMem=AllocateMemory(destLen)
Result = compress2(*DestMem,@destLen,*SourceMem,sourcelen,9)
If Result = 0
CreateFile(1, "temp\teste2.dds")
WriteData(1, *DestMem, destLen+1)
CloseFile(1)
FreeMemory(*sourceMem)
FreeMemory(*destMem)
EndIf
the orginal file have a size 2MB if compress and uncompress the size is 56kb
Re: How to compress with zlib
Posted: Thu Oct 17, 2013 5:17 am
by skywalk
Sorry, those are constants from the zlib website. I'm not seeing much wrong with what you're doing, but I am on v5.2. Can you try to simplify the problem by compressing/uncompressing a smaller file?
Code: Select all
#ZL_OK = 0
#ZL_STREAM_END = 1
#ZL_NEED_DICT = 2
#ZL_ERRNO = -1
#ZL_STREAM_ERROR = -2
#ZL_DATA_ERROR = -3
#ZL_MEM_ERROR = -4
#ZL_BUF_ERROR = -5
#ZL_VERSION_ERROR = -6
#ZL_CRC32_ERROR = -15
#ZL_NO_COMPRESSION = 0
#ZL_BEST_SPEED = 1
#ZL_BEST_COMPRESSION = 9
#ZL_DEFAULT_COMPRESSION = -1
; Compression level must be #ZL_DEFAULT_COMPRESSION, or between 0 and 9.
; 1 gives best speed, 9 gives best compression, 0 = no compression (input data is simply copied a block at a time).
; #ZL_DEFAULT_COMPRESSION = Level 6, compromise between speed and compression.
Re: How to compress with zlib
Posted: Thu Oct 17, 2013 6:20 am
by moob
skywalk wrote:Sorry, those are constants from the zlib website. I'm not seeing much wrong with what you're doing, but I am on v5.2. Can you try to simplify the problem by compressing/uncompressing a smaller file?
Code: Select all
#ZL_OK = 0
#ZL_STREAM_END = 1
#ZL_NEED_DICT = 2
#ZL_ERRNO = -1
#ZL_STREAM_ERROR = -2
#ZL_DATA_ERROR = -3
#ZL_MEM_ERROR = -4
#ZL_BUF_ERROR = -5
#ZL_VERSION_ERROR = -6
#ZL_CRC32_ERROR = -15
#ZL_NO_COMPRESSION = 0
#ZL_BEST_SPEED = 1
#ZL_BEST_COMPRESSION = 9
#ZL_DEFAULT_COMPRESSION = -1
; Compression level must be #ZL_DEFAULT_COMPRESSION, or between 0 and 9.
; 1 gives best speed, 9 gives best compression, 0 = no compression (input data is simply copied a block at a time).
; #ZL_DEFAULT_COMPRESSION = Level 6, compromise between speed and compression.
I use PB 5.00 because I need to use an rwapper of DevIL lib and only found one compatible with PB 5.00.
You can test my code abobe in anything, the result of compression is an corrupt file.
I try uncompress with an external program and the compression generated with my code is corrupt
and I don't now what wrong on the my code abobe to generate an corrupt file.
sorry for my english
Re: How to compress with zlib
Posted: Fri Oct 18, 2013 2:15 am
by JHPJHP
What error (if any) are you receiving when you run the following?
- uses PureBasic's built-in zlib library (to use the DLL - create Prototypes)
- works with text based files / binary files require some modification to the code
- don't know why, but deflateBound is not doing its job
-- to decrease the size of the deflated file, adjust this line:
LengthToWrite = deflateBound(@strm, LengthToRead)
--- to something like:
LengthToWrite = deflateBound(@strm, LengthToRead) / 2
Code: Select all
#ENABLE_GZIP = 16
#Z_NULL = 0
#ZLIB_VERSION = "1.2.8"
#Z_FINISH = 4
#Z_DEFAULT_COMPRESSION = -1
#Z_DEFLATED = 8
#MAX_MEM_LEVEL = 9
#Z_DEFAULT_STRATEGY = 0
Structure Z_STREAM Align #PB_Structure_AlignC
*next_in.Byte
avail_in.l
total_in.l
*next_out.Byte
avail_out.l
total_out.l
*msg.Byte
*state
zalloc.l
zfree.l
opaque.l
data_type.i
adler.l
reserved.l
CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
alignment.l
CompilerEndIf
EndStructure
ImportC "zlib.lib"
inflateInit2_(*strm, windowBits.i, Version.s, strm_size)
inflate(*strm, flush.i)
inflateEnd(*strm)
deflateInit2_(*strm, level.i, method.i, windowBits.i, memlevel.i, strategy.i, Version.s, strm_size)
deflateBound(*strm, sourceLen)
deflate(*strm, flush.i)
deflateEnd(*strm)
EndImport
Procedure.s InflatePayload(*Payload, windowBits.i)
LengthToRead = MemorySize(*Payload)
LengthToWrite = LengthToRead * 8
*Output = AllocateMemory(LengthToWrite)
strm.Z_STREAM
strm\next_in = *Payload
strm\avail_in = LengthToRead
strm\next_out = *Output
strm\avail_out = LengthToWrite
strm\zalloc = #Z_NULL
strm\zfree = #Z_NULL
strm\opaque = #Z_NULL
inflateInit2_(@strm, windowBits, #ZLIB_VERSION, SizeOf(Z_STREAM))
inflate(@strm, #Z_FINISH)
inflateEnd(@strm)
sOutput.s = PeekS(*Output, -1, #PB_UTF8)
FreeMemory(*Output)
ProcedureReturn sOutput
EndProcedure
Procedure DeflatePayload(sInput.s, windowBits.i)
LengthToRead = StringByteLength(sInput)
strm.Z_STREAM
strm\next_in = @sInput
strm\avail_in = LengthToRead
strm\zalloc = #Z_NULL
strm\zfree = #Z_NULL
strm\opaque = #Z_NULL
deflateInit2_(@strm, #Z_DEFAULT_COMPRESSION, #Z_DEFLATED, windowBits, #MAX_MEM_LEVEL, #Z_DEFAULT_STRATEGY, #ZLIB_VERSION, SizeOf(Z_STREAM))
LengthToWrite = deflateBound(@strm, LengthToRead)
*Payload = AllocateMemory(LengthToWrite)
strm\next_out = *Payload
strm\avail_out = LengthToWrite
deflate(@strm, #Z_FINISH)
deflateEnd(@strm)
ProcedureReturn *Payload
EndProcedure
If ReadFile(0, "Buglist.txt")
While Eof(0) = 0
sInput.s + ReadString(0) + #CRLF$
Wend
CloseFile(0)
*Payload = DeflatePayload(sInput, 15 | #ENABLE_GZIP)
If CreateFile(0, "Buglist.gz")
WriteData(0, *Payload, MemorySize(*Payload))
CloseFile(0)
EndIf
FreeMemory(*Payload)
EndIf
If OpenFile(0, "Buglist.gz")
LengthToRead = FileSize("Buglist.gz")
*Payload = AllocateMemory(LengthToRead)
ReadData(0, *Payload, LengthToRead)
CloseFile(0)
sOutput.s = InflatePayload(*Payload, 15 | #ENABLE_GZIP)
Debug sOutput
If CreateFile(0, "Buglist2.txt")
WriteString(0, sOutput)
CloseFile(0)
EndIf
EndIf
FreeMemory(*Payload)
Re: How to compress with zlib
Posted: Fri Oct 18, 2013 4:04 am
by moob
JHPJHP wrote:What error (if any) are you receiving when you run the following?
- uses PureBasic's built-in zlib library (to use the DLL - create Prototypes)
- works with text based files / binary files require some modification to the code
- don't know why, but deflateBound is not doing its job
-- to decrease the size of the deflated file, adjust this line:
LengthToWrite = deflateBound(@strm, LengthToRead)
--- to something like:
LengthToWrite = deflateBound(@strm, LengthToRead) / 2
Code: Select all
#ENABLE_GZIP = 16
#Z_NULL = 0
#ZLIB_VERSION = "1.2.8"
#Z_FINISH = 4
#Z_DEFAULT_COMPRESSION = -1
#Z_DEFLATED = 8
#MAX_MEM_LEVEL = 9
#Z_DEFAULT_STRATEGY = 0
Structure Z_STREAM Align #PB_Structure_AlignC
*next_in.Byte
avail_in.l
total_in.l
*next_out.Byte
avail_out.l
total_out.l
*msg.Byte
*state
zalloc.l
zfree.l
opaque.l
data_type.i
adler.l
reserved.l
CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
alignment.l
CompilerEndIf
EndStructure
ImportC "zlib.lib"
inflateInit2_(*strm, windowBits.i, Version.s, strm_size)
inflate(*strm, flush.i)
inflateEnd(*strm)
deflateInit2_(*strm, level.i, method.i, windowBits.i, memlevel.i, strategy.i, Version.s, strm_size)
deflateBound(*strm, sourceLen)
deflate(*strm, flush.i)
deflateEnd(*strm)
EndImport
Procedure.s InflatePayload(*Payload, windowBits.i)
LengthToRead = MemorySize(*Payload)
LengthToWrite = LengthToRead * 8
*Output = AllocateMemory(LengthToWrite)
strm.Z_STREAM
strm\next_in = *Payload
strm\avail_in = LengthToRead
strm\next_out = *Output
strm\avail_out = LengthToWrite
strm\zalloc = #Z_NULL
strm\zfree = #Z_NULL
strm\opaque = #Z_NULL
inflateInit2_(@strm, windowBits, #ZLIB_VERSION, SizeOf(Z_STREAM))
inflate(@strm, #Z_FINISH)
inflateEnd(@strm)
sOutput.s = PeekS(*Output, -1, #PB_UTF8)
FreeMemory(*Output)
ProcedureReturn sOutput
EndProcedure
Procedure DeflatePayload(sInput.s, windowBits.i)
LengthToRead = StringByteLength(sInput)
strm.Z_STREAM
strm\next_in = @sInput
strm\avail_in = LengthToRead
strm\zalloc = #Z_NULL
strm\zfree = #Z_NULL
strm\opaque = #Z_NULL
deflateInit2_(@strm, #Z_DEFAULT_COMPRESSION, #Z_DEFLATED, windowBits, #MAX_MEM_LEVEL, #Z_DEFAULT_STRATEGY, #ZLIB_VERSION, SizeOf(Z_STREAM))
LengthToWrite = deflateBound(@strm, LengthToRead)
*Payload = AllocateMemory(LengthToWrite)
strm\next_out = *Payload
strm\avail_out = LengthToWrite
deflate(@strm, #Z_FINISH)
deflateEnd(@strm)
ProcedureReturn *Payload
EndProcedure
If ReadFile(0, "Buglist.txt")
While Eof(0) = 0
sInput.s + ReadString(0) + #CRLF$
Wend
CloseFile(0)
*Payload = DeflatePayload(sInput, 15 | #ENABLE_GZIP)
If CreateFile(0, "Buglist.gz")
WriteData(0, *Payload, MemorySize(*Payload))
CloseFile(0)
EndIf
FreeMemory(*Payload)
EndIf
If OpenFile(0, "Buglist.gz")
LengthToRead = FileSize("Buglist.gz")
*Payload = AllocateMemory(LengthToRead)
ReadData(0, *Payload, LengthToRead)
CloseFile(0)
sOutput.s = InflatePayload(*Payload, 15 | #ENABLE_GZIP)
Debug sOutput
If CreateFile(0, "Buglist2.txt")
WriteString(0, sOutput)
CloseFile(0)
EndIf
EndIf
FreeMemory(*Payload)
the error I receive when open with an external tool is "File parsing failed, invalid or unsupported file format"
I can't test your code in PB 5.00 because give me an error "Align is not a valid operator"
Can you please adapt your code to work with PB 5.00
thanks
Re: How to compress with zlib
Posted: Fri Oct 18, 2013 6:48 am
by JHPJHP
That error doesn't make sense (to me) - Align is a part of the Structure syntax?
If your in the 32bit version of PureBasic - change the Z_STREAM structure to this:
- also test with a basic text file and see if you get an error
- what error (if any) do you get now
Code: Select all
Structure Z_STREAM
*next_in.Byte
avail_in.l
total_in.l
*next_out.Byte
avail_out.l
total_out.l
*msg.Byte
*state
zalloc.l
zfree.l
opaque.l
data_type.i
adler.l
reserved.l
EndStructure
Re: How to compress with zlib
Posted: Fri Oct 18, 2013 7:19 am
by moob
JHPJHP wrote:That error doesn't make sense (to me) - Align is a part of the Structure syntax?
If your in the 32bit version of PureBasic - change the Z_STREAM structure to this:
- also test with a basic text file and see if you get an error
- what error (if any) do you get now
Code: Select all
Structure Z_STREAM
*next_in.Byte
avail_in.l
total_in.l
*next_out.Byte
avail_out.l
total_out.l
*msg.Byte
*state
zalloc.l
zfree.l
opaque.l
data_type.i
adler.l
reserved.l
EndStructure
I write my name "moob" in "Buglist.txt" but in "Buglist2.txt" not appear my name only the first letter.
And I tried with dds file and not working too.
Re: How to compress with zlib
Posted: Fri Oct 18, 2013 8:03 am
by JHPJHP
Check to see if you have Unicode selected under options - if so remove it... Let me know if that resolves the problem?
Re: How to compress with zlib
Posted: Fri Oct 18, 2013 5:09 pm
by moob
JHPJHP wrote:Check to see if you have Unicode selected under options - if so remove it... Let me know if that resolves the problem?
With txt file work, but with dds file crash.
And I need is for compress dds file with zlib
Here is my dds file for test:
http://www9.zippyshare.com/v/80941158/file.html