PureBasic Forums - English

http://www.purebasic.com
https://www.purebasic.fr/english/

How to compress with zlib

https://www.purebasic.fr/english/viewtopic.php?t=57055

Page 3 of 6

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:10 am
by Thunder93
JHPJHP. Mine works but I just lazy to support 32bit compile. Like you said all you need to-do is surround the alignments I made with...

Code: Select all

CompilerIf #PB_Compiler_Processor = #PB_Processor_x64 
  PB_Alignment?.b[?]
CompilerEndIf

:wink:

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:15 am
by Thunder93
Btw, your version corrupts under x64 because you aren't padding correctly.

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:22 am
by JHPJHP
Hi Thunder93,

I just noticed there was a new page... when did you get here? :P ... I just kept editing my previous post until I was happy with it, not knowing you already responded. :lol:

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:39 am
by Thunder93
JHPJHP.

Code: Select all

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l
  zfree.l
  opaque.l
  data_type.i
  adler.l
  reserved.l
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    alignment.l
  CompilerEndIf
EndStructure
This version of your code does not work with x64 PB compiler, your padding is simply wrong. The compressed file will not decompress, so there is no .dds file to compare with the original .dds file hash.

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:47 am
by Thunder93
My version produces the following;

x64 PB
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
b6c1f472b21d542185e4e4a78a569b12 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

And my extraction w/another decompressor of the test.gz file
8e8c5f619da1de67d67480671fc2827c test

x86 PB
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
b6c1f472b21d542185e4e4a78a569b12 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

And my extraction w/another decompressor of the test.gz file
8e8c5f619da1de67d67480671fc2827c test

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:49 am
by JHPJHP
Yes - I see my mistake, I may be posting without Align #PB_Structure_AlignC, but I'm still using it in my code.
- thanks for the clarification... I'll correct my previous posts

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 5:52 pm
by moob
Thunder93 and JHPJHP

Not work the continues corrupt

I'm use PB 5.00 x86 under windows 8 x64

Why 'gz'? #ENABLE_GZIP is the same of ZLIB compression??


Why create test.gz and not compressed dds?

When I compress with external program the compessed file size is 107 KB and work very well

with this example is 116 KB still corrupt I can't open this file.


You can zlib and unzlib with this program is free http://www26.zippyshare.com/v/37726668/file.html

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 6:50 pm
by Thunder93
Are you sure you using exactly the following to work for both PB x64 and x86?

Code: Select all

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l     
  zfree.l  
  opaque.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  
  data_type.i
  adler.l
  reserved.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[8]
  CompilerEndIf  
EndStructure

Here are my tests again... but with older versions of PB w/the above code.


PureBasic v4.60 x64
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
------------

PureBasic v4.41 x86
8e8c5f619da1de67d67480671fc2827c test.dds
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz
8e8c5f619da1de67d67480671fc2827c test2.dds

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
-------------

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 7:11 pm
by moob
Thunder93 wrote:Are you sure you using exactly the following to work for both PB x64 and x86?

Code: Select all

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l     
  zfree.l  
  opaque.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  
  data_type.i
  adler.l
  reserved.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[8]
  CompilerEndIf  
EndStructure

Here are my tests again... but with older versions of PB w/the above code.


PureBasic v4.60 x64
8e8c5f619da1de67d67480671fc2827c test.dds -- Original
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
------------

PureBasic v4.41 x86
8e8c5f619da1de67d67480671fc2827c test.dds
2a0c70c5d54651e0bc59b5bf11ee0697 test.gz
8e8c5f619da1de67d67480671fc2827c test2.dds

8e8c5f619da1de67d67480671fc2827c test -- Extraction manually via another decompressor
-------------
Yes, still corrupt file.

I use this code

Code: Select all

#ENABLE_GZIP = 16
#Z_NULL = 0
#ZLIB_VERSION = "1.2.8"
#Z_FINISH = 4
#Z_DEFAULT_COMPRESSION = -1
#Z_DEFLATED = 8
#MAX_MEM_LEVEL = 9
#Z_DEFAULT_STRATEGY = 0

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l     
  zfree.l 
  opaque.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment1.b[4]
  CompilerEndIf
  
  data_type.i
  adler.l
  reserved.l
  
  CompilerIf #PB_Compiler_Processor = #PB_Processor_x64
    PB_Alignment2.b[8]
  CompilerEndIf 
EndStructure

ImportC "zlib.lib"
  inflateInit2_(*strm, windowBits.i, Version.s, strm_size)
  inflate(*strm, flush.i)
  inflateEnd(*strm)
  deflateInit2_(*strm, level.i, method.i, windowBits.i, memlevel.i, strategy.i, Version.s, strm_size)
  deflateBound(*strm, sourceLen)
  deflate(*strm, flush.i)
  deflateEnd(*strm)
EndImport

Procedure InflatePayload(*Payload, windowBits.i)
  LengthToRead = MemorySize(*Payload)
  LengthToWrite = FileSize("test.dds")
  *Output = AllocateMemory(LengthToWrite)
  strm.Z_STREAM
  strm\next_in = *Payload
  strm\avail_in = LengthToRead
  strm\next_out = *Output
  strm\avail_out = LengthToWrite
  strm\zalloc = #Z_NULL
  strm\zfree = #Z_NULL
  strm\opaque = #Z_NULL
  inflateInit2_(@strm, windowBits, #ZLIB_VERSION, SizeOf(Z_STREAM))
  inflate(@strm, #Z_FINISH)
  inflateEnd(@strm)
  ProcedureReturn *Output
EndProcedure

Procedure DeflatePayload(*Input, windowBits.i)
  LengthToRead = MemorySize(*Input)
  strm.Z_STREAM
  strm\next_in = *Input
  strm\avail_in = LengthToRead
  strm\zalloc = #Z_NULL
  strm\zfree = #Z_NULL
  strm\opaque = #Z_NULL
  deflateInit2_(@strm, #Z_DEFAULT_COMPRESSION, #Z_DEFLATED, windowBits, #MAX_MEM_LEVEL, #Z_DEFAULT_STRATEGY, #ZLIB_VERSION, SizeOf(Z_STREAM))
  LengthToWrite = deflateBound(@strm, LengthToRead) / 20
  *Payload = AllocateMemory(LengthToWrite)
  strm\next_out = *Payload
  strm\avail_out = LengthToWrite
  deflate(@strm, #Z_FINISH)
  deflateEnd(@strm)
  ProcedureReturn *Payload
EndProcedure

If OpenFile(0, "test.dds")
  LengthToRead = FileSize("test.dds")
  *Input = AllocateMemory(LengthToRead)
  ReadData(0, *Input, LengthToRead)
  CloseFile(0)
  *Payload = DeflatePayload(*Input, 15 | #ENABLE_GZIP)

  If CreateFile(0, "test2.dds")
    WriteData(0, *Payload, MemorySize(*Payload))
    CloseFile(0)
  EndIf
  FreeMemory(*Payload)
EndIf

If OpenFile(0, "test2.dds")
  LengthToRead = FileSize("test2.dds")
  *Payload = AllocateMemory(LengthToRead)
  ReadData(0, *Payload, LengthToRead)
  CloseFile(0)
  *Output = InflatePayload(*Payload, 15 | #ENABLE_GZIP)

  If CreateFile(0, "test3.dds")
    WriteData(0, *Output, MemorySize(*Output))
    CloseFile(0)
  EndIf
  FreeMemory(*Output)
  FreeMemory(*Payload)
EndIf
using this code try uncompress with the program I post abobe, you can't because the dds is corrupt

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 7:27 pm
by Thunder93
No problem here with both x64 and x86 of PureBasic v5.20 running on Windows 7 x64..

8e8c5f619da1de67d67480671fc2827c test.dds
f5be8972032d759ee3fd8071f45e63f7 test2.dds
8e8c5f619da1de67d67480671fc2827c test3.dds


Edit: ... with PB 4.51 x86 on Windows 7 x64

8e8c5f619da1de67d67480671fc2827c test.dds
f5be8972032d759ee3fd8071f45e63f7 test2.dds
8e8c5f619da1de67d67480671fc2827c test3.dds

identical results always. :?

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 7:28 pm
by Thunder93
Maybe a PB userlib or a tool extension is messing with your results?

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 7:40 pm
by JHPJHP
One more thing to try:

Change your code to the following:

Code: Select all

*Payload = DeflatePayload(*Input, -15)

Code: Select all

*Output = InflatePayload(*Payload, -15)
Don't worry that the compressed .dds file cannot be opened like a .gz file... this is raw deflate.
- the inflated file should be identical to the original
- refer to the documentation: http://zlib.net/manual.html

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 7:52 pm
by Thunder93
8e8c5f619da1de67d67480671fc2827c test.dds
8e8c5f619da1de67d67480671fc2827c test3.dds

f5be8972032d759ee3fd8071f45e63f7 test2.dds.gz <---- renamed from test2.dds
8e8c5f619da1de67d67480671fc2827c test2.dds <---- Manual decompression w/another decompressor. :wink:

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 8:31 pm
by Thunder93
I tested my structure on two different computers with identical results.... JHP you share moob experiences? Or does the structure I provided work for you too?

Re: How to compress with zlib

Posted: Sun Oct 20, 2013 8:53 pm
by moob
JHPJHP wrote:One more thing to try:

Change your code to the following:

Code: Select all

*Payload = DeflatePayload(*Input, -15)

Code: Select all

*Output = InflatePayload(*Payload, -15)
Don't worry that the compressed .dds file cannot be opened like a .gz file... this is raw deflate.
- the inflated file should be identical to the original
- refer to the documentation: http://zlib.net/manual.html

Not work either, still corrupt file
Thunder93 wrote:Maybe a PB userlib or a tool extension is messing with your results?
but I have try with external zlib.dll and the result is the same see the first page

I don't now why not work if work for you both why not work for me??

its a mystery

this is the file after compress with the code I post abobe if you try uncompress with program I post give an error is corrupt

http://www64.zippyshare.com/v/56317546/file.html
All times are UTC+01:00
Page 3 of 6

Powered by phpBB® Forum Software © phpBB Limited