How to compress with zlib

[Thunder93]

JHPJHP. Mine works but I just lazy to support 32bit compile. Like you said all you need to-do is surround the alignments I made with...

Code: Select all

CompilerIf #PB_Compiler_Processor = #PB_Processor_x64 
  PB_Alignment?.b[?]
CompilerEndIf

[Thunder93]

Btw, your version corrupts under x64 because you aren't padding correctly.

[JHPJHP]

Hi Thunder93,

I just noticed there was a new page... when did you get here? ... I just kept editing my previous post until I was happy with it, not knowing you already responded.

[Thunder93]

JHPJHP.

Code: Select all

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l
  zfree.l
  opaque.l
  data_type.i
  adler.l
  reserved.l
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    alignment.l
  CompilerEndIf
EndStructure

This version of your code does not work with x64 PB compiler, your padding is simply wrong. The compressed file will not decompress, so there is no .dds file to compare with the original .dds file hash.

[Thunder93]

My version produces the following;

x64 PB
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
b6c1f472b21d542185e4e4a78a569b12 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

And my extraction w/another decompressor of the test.gz file
8e8c5f619da1de67d67480671fc2827c test

x86 PB
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
b6c1f472b21d542185e4e4a78a569b12 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

And my extraction w/another decompressor of the test.gz file
8e8c5f619da1de67d67480671fc2827c test

[JHPJHP]

Yes - I see my mistake, I may be posting without Align #PB_Structure_AlignC, but I'm still using it in my code.
- thanks for the clarification... I'll correct my previous posts

[moob]

Thunder93 and JHPJHP

Not work the continues corrupt

I'm use PB 5.00 x86 under windows 8 x64

Why 'gz'? #ENABLE_GZIP is the same of ZLIB compression??


Why create test.gz and not compressed dds?

When I compress with external program the compessed file size is 107 KB and work very well

with this example is 116 KB still corrupt I can't open this file.


You can zlib and unzlib with this program is free http://www26.zippyshare.com/v/37726668/file.html

[Thunder93]

Are you sure you using exactly the following to work for both PB x64 and x86?

Code: Select all

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l     
  zfree.l  
  opaque.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  
  data_type.i
  adler.l
  reserved.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[8]
  CompilerEndIf  
EndStructure

Here are my tests again... but with older versions of PB w/the above code.


PureBasic v4.60 x64
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
------------

PureBasic v4.41 x86
8e8c5f619da1de67d67480671fc2827c test.dds
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz
8e8c5f619da1de67d67480671fc2827c test2.dds

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
-------------

[moob]

Are you sure you using exactly the following to work for both PB x64 and x86?

Code: Select all

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l     
  zfree.l  
  opaque.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  
  data_type.i
  adler.l
  reserved.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[8]
  CompilerEndIf  
EndStructure

Here are my tests again... but with older versions of PB w/the above code.


PureBasic v4.60 x64
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
------------

PureBasic v4.41 x86
8e8c5f619da1de67d67480671fc2827c test.dds
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz
8e8c5f619da1de67d67480671fc2827c test2.dds

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
-------------

Yes, still corrupt file.

I use this code

Code: Select all

#ENABLE_GZIP = 16
#Z_NULL = 0
#ZLIB_VERSION = "1.2.8"
#Z_FINISH = 4
#Z_DEFAULT_COMPRESSION = -1
#Z_DEFLATED = 8
#MAX_MEM_LEVEL = 9
#Z_DEFAULT_STRATEGY = 0

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l     
  zfree.l 
  opaque.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  
  data_type.i
  adler.l
  reserved.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[8]
  CompilerEndIf 
EndStructure

ImportC "zlib.lib"
  inflateInit2_(*strm, windowBits.i, Version.s, strm_size)
  inflate(*strm, flush.i)
  inflateEnd(*strm)
  deflateInit2_(*strm, level.i, method.i, windowBits.i, memlevel.i, strategy.i, Version.s, strm_size)
  deflateBound(*strm, sourceLen)
  deflate(*strm, flush.i)
  deflateEnd(*strm)
EndImport

Procedure InflatePayload(*Payload, windowBits.i)
  LengthToRead = MemorySize(*Payload)
  LengthToWrite = FileSize("test.dds")
  *Output = AllocateMemory(LengthToWrite)
  strm.Z_STREAM
  strm\next_in = *Payload
  strm\avail_in = LengthToRead
  strm\next_out = *Output
  strm\avail_out = LengthToWrite
  strm\zalloc = #Z_NULL
  strm\zfree = #Z_NULL
  strm\opaque = #Z_NULL
  inflateInit2_(@strm, windowBits, #ZLIB_VERSION, SizeOf(Z_STREAM))
  inflate(@strm, #Z_FINISH)
  inflateEnd(@strm)
  ProcedureReturn *Output
EndProcedure

Procedure DeflatePayload(*Input, windowBits.i)
  LengthToRead = MemorySize(*Input)
  strm.Z_STREAM
  strm\next_in = *Input
  strm\avail_in = LengthToRead
  strm\zalloc = #Z_NULL
  strm\zfree = #Z_NULL
  strm\opaque = #Z_NULL
  deflateInit2_(@strm, #Z_DEFAULT_COMPRESSION, #Z_DEFLATED, windowBits, #MAX_MEM_LEVEL, #Z_DEFAULT_STRATEGY, #ZLIB_VERSION, SizeOf(Z_STREAM))
  LengthToWrite = deflateBound(@strm, LengthToRead) / 20
  *Payload = AllocateMemory(LengthToWrite)
  strm\next_out = *Payload
  strm\avail_out = LengthToWrite
  deflate(@strm, #Z_FINISH)
  deflateEnd(@strm)
  ProcedureReturn *Payload
EndProcedure

If OpenFile(0, "test.dds")
  LengthToRead = FileSize("test.dds")
  *Input = AllocateMemory(LengthToRead)
  ReadData(0, *Input, LengthToRead)
  CloseFile(0)
  *Payload = DeflatePayload(*Input, 15 | #ENABLE_GZIP)

  If CreateFile(0, "test2.dds")
    WriteData(0, *Payload, MemorySize(*Payload))
    CloseFile(0)
  EndIf
  FreeMemory(*Payload)
EndIf

If OpenFile(0, "test2.dds")
  LengthToRead = FileSize("test2.dds")
  *Payload = AllocateMemory(LengthToRead)
  ReadData(0, *Payload, LengthToRead)
  CloseFile(0)
  *Output = InflatePayload(*Payload, 15 | #ENABLE_GZIP)

  If CreateFile(0, "test3.dds")
    WriteData(0, *Output, MemorySize(*Output))
    CloseFile(0)
  EndIf
  FreeMemory(*Output)
  FreeMemory(*Payload)
EndIf

using this code try uncompress with the program I post abobe, you can't because the dds is corrupt

[Thunder93]

No problem here with both x64 and x86 of PureBasic v5.20 running on Windows 7 x64..

8e8c5f619da1de67d67480671fc2827c test.dds
f5be8972032d759ee3fd8071f45e63f7 test2.dds
8e8c5f619da1de67d67480671fc2827c test3.dds


Edit: ... with PB 4.51 x86 on Windows 7 x64

8e8c5f619da1de67d67480671fc2827c test.dds
f5be8972032d759ee3fd8071f45e63f7 test2.dds
8e8c5f619da1de67d67480671fc2827c test3.dds

identical results always.

[Thunder93]

Maybe a PB userlib or a tool extension is messing with your results?

[JHPJHP]

One more thing to try:

Change your code to the following:

Code: Select all

*Payload = DeflatePayload(*Input, -15)

Code: Select all

*Output = InflatePayload(*Payload, -15)

Don't worry that the compressed .dds file cannot be opened like a .gz file... this is raw deflate.
- the inflated file should be identical to the original
- refer to the documentation: http://zlib.net/manual.html

[Thunder93]

8e8c5f619da1de67d67480671fc2827c test.dds
8e8c5f619da1de67d67480671fc2827c test3.dds

f5be8972032d759ee3fd8071f45e63f7 test2.dds.gz <---- renamed from test2.dds
8e8c5f619da1de67d67480671fc2827c test2.dds <---- Manual decompression w/another decompressor.

[Thunder93]

I tested my structure on two different computers with identical results.... JHP you share moob experiences? Or does the structure I provided work for you too?

[moob]

One more thing to try:

Change your code to the following:

Code: Select all

*Payload = DeflatePayload(*Input, -15)

Code: Select all

*Output = InflatePayload(*Payload, -15)

Don't worry that the compressed .dds file cannot be opened like a .gz file... this is raw deflate.
- the inflated file should be identical to the original
- refer to the documentation: http://zlib.net/manual.html

Not work either, still corrupt file

Maybe a PB userlib or a tool extension is messing with your results?

but I have try with external zlib.dll and the result is the same see the first page

I don't now why not work if work for you both why not work for me??

its a mystery

this is the file after compress with the code I post abobe if you try uncompress with program I post give an error is corrupt

http://www64.zippyshare.com/v/56317546/file.html