PureBasic Forums - English

The following code will work as is, but you should read the documentation on the zlib site, then optimize it to fit your needs.
- the current example tries to inflate / deflate all at once, but this may not be the best solution for your requirements
- as I mentioned in a previous post - if your file is not text based you would have to modify the code a bit
-- I suggest comparing the two examples to see how they work with the different formats
- notice that I set the compressed size with: LengthToWrite = deflateBound(@strm, LengthToRead) / 2
-- not sure why delatebound is not optimizing compression - thus the reason for "dividing by two" (modify to whatever works)

JHPJHP wrote:The following code will work as is, but you should read the documentation on zlib site, then optimize it to fit your needs.
- the current example tries to inflate / deflate all at once, but this may not be the best solution for your requirements
- as I mentioned in a previous post - if your file is not text based you would have to modify the code a bit
-- I suggest comparing the two examples to see how they work with the different formats
- notice that I set the compressed size with: LengthToWrite = deflateBound(@strm, LengthToRead) / 2
-- not sure why delatebound is not optimizing compression - thus the reason for "dividing by two" (modify to whatever works)

Code: Select all

#ENABLE_GZIP = 16
#Z_NULL = 0
#ZLIB_VERSION = "1.2.8"
#Z_FINISH = 4
#Z_DEFAULT_COMPRESSION = -1
#Z_DEFLATED = 8
#MAX_MEM_LEVEL = 9
#Z_DEFAULT_STRATEGY = 0

Structure Z_STREAM
  *next_in.Byte
  avail_in.l
  total_in.l
  *next_out.Byte
  avail_out.l
  total_out.l
  *msg.Byte
  *state
  zalloc.l
  zfree.l
  opaque.l
  data_type.i
  adler.l
  reserved.l
EndStructure

ImportC "zlib.lib"
  inflateInit2_(*strm, windowBits.i, Version.s, strm_size)
  inflate(*strm, flush.i)
  inflateEnd(*strm)
  deflateInit2_(*strm, level.i, method.i, windowBits.i, memlevel.i, strategy.i, Version.s, strm_size)
  deflateBound(*strm, sourceLen)
  deflate(*strm, flush.i)
  deflateEnd(*strm)
EndImport

Procedure InflatePayload(*Payload, windowBits.i)
  LengthToRead = MemorySize(*Payload)
  LengthToWrite = FileSize("test.dds")
  *Output = AllocateMemory(LengthToWrite)
  strm.Z_STREAM
  strm\next_in = *Payload
  strm\avail_in = LengthToRead
  strm\next_out = *Output
  strm\avail_out = LengthToWrite
  strm\zalloc = #Z_NULL
  strm\zfree = #Z_NULL
  strm\opaque = #Z_NULL
  inflateInit2_(@strm, windowBits, #ZLIB_VERSION, SizeOf(Z_STREAM))
  inflate(@strm, #Z_FINISH)
  inflateEnd(@strm)
  ProcedureReturn *Output
EndProcedure

Procedure DeflatePayload(*Input, windowBits.i)
  LengthToRead = MemorySize(*Input)
  strm.Z_STREAM
  strm\next_in = *Input
  strm\avail_in = LengthToRead
  strm\zalloc = #Z_NULL
  strm\zfree = #Z_NULL
  strm\opaque = #Z_NULL
  deflateInit2_(@strm, #Z_DEFAULT_COMPRESSION, #Z_DEFLATED, windowBits, #MAX_MEM_LEVEL, #Z_DEFAULT_STRATEGY, #ZLIB_VERSION, SizeOf(Z_STREAM))
  LengthToWrite = deflateBound(@strm, LengthToRead) / 2
  *Payload = AllocateMemory(LengthToWrite)
  strm\next_out = *Payload
  strm\avail_out = LengthToWrite
  deflate(@strm, #Z_FINISH)
  deflateEnd(@strm)
  ProcedureReturn *Payload
EndProcedure

If OpenFile(0, "test.dds")
  LengthToRead = FileSize("test.dds")
  *Input = AllocateMemory(LengthToRead)
  ReadData(0, *Input, LengthToRead)
  CloseFile(0)
  *Payload = DeflatePayload(*Input, 15 | #ENABLE_GZIP)

  If CreateFile(0, "test.gz")
    WriteData(0, *Payload, MemorySize(*Payload))
    CloseFile(0)
  EndIf
  FreeMemory(*Payload)
EndIf

If OpenFile(0, "test.gz")
  LengthToRead = FileSize("test.gz")
  *Payload = AllocateMemory(LengthToRead)
  ReadData(0, *Payload, LengthToRead)
  CloseFile(0)
  *Output = InflatePayload(*Payload, 15 | #ENABLE_GZIP)

  If CreateFile(0, "test2.dds")
    WriteData(0, *Output, MemorySize(*Output))
    CloseFile(0)
  EndIf
  FreeMemory(*Output)
  FreeMemory(*Payload)
EndIf

In the last example debug give an error in line 75 "*Input = AllocateMemory(LengthToRead)", the error is "Can't allocate a memory block of size 0"

thanks for your effort

but I need compress dds file and not txt and no one of your example work in dds file.

You have done much trying help me thanks for that, I'm a noob in this.

thank you

NB*: These are only examples, you will need to modify the code to fit your existing script.

Double check the file names: LengthToRead is the file size of your .dds file, if your getting "Can't allocate a memory block of size 0" then your .dds file is size 0, or you're not pointing to the correct file.

Test Only - hard code the value:

Hi moob,
Please don't reply with full quotes of previous post.
Also, break your problem down. If you cannot compress/uncompress small amounts of data, don't bother with your large dds file.
A text file is still just binary data. Of course, text will compress more than some random binary data, but that is not the point.

A couple good points skywalk, I should have mentioned earlier about "full quotes".

You probably haven't read through all the posts (and who could blame you), but moob has already compressed and decompressed a small text file; the new scripts I gave him are for binary data... I think he's just making some noob / moob mistakes.

The scripts you provided earlier would have worked just as well.

Haha, yes I applaud your patience so far.
I got confused by moob's questions since I am compressing/uncompressing all day long with my backup tool.
File sizes or types don't matter at least for the method I posted.

Hi moob,

I did another test using the .dds file you supplied - increasing the compression. I modified the division from 2 to 20... as I mentioned in previous posts - change this value to fit your needs. I tested the output file against your original file using UltraCompare, and the files were identical. If you going to increase the compression using this method, be sure to check for data loss.

NB*: I sent an email to Mark Adler (co-creator of zlib) to clarify the use of deflateBound.

Cheers!

JHPJHP wrote:Hi moob,

I did another test using the .dds file you supplied - increasing the compression. I modified the division from 2 to 20... as I mentioned in previous posts - change this value to fit your needs.

Code: Select all

LengthToWrite = deflateBound(@strm, LengthToRead) / 20

I tested the output file against your original file using UltraCompare, and the files were identical. If you going to increase the compression using this method, be sure to check for data loss.

NB*: I sent an email to Mark Adler (co-creator of zlib) to clarify the use of deflateBound.

Cheers!

Ok thanks I can compress, but there is something wrong with compression, the file is corrupt after compressed.


This is what inside of my test.dds without compression.




And this is what inside after compression



as you see there is nothing inside after compression.

the program used can open dds file compressed with zlib

And I have trying uncompress the file with other program and the result is this, the same result.




I can compress with your example and the example of skywalk

but the result is an corrupt file.

I can't uncompress with external program the compressed file with examples presented by you and skywalk here, because the result is an corrupt dds file.

english is not my first language, I hope you both can understand what I write and not misunderstood it.

and yes I'm noob in PB, I not understand many code you write here because of this I ask for help.

Hi moob. Revisit JHPJHP post - http://www.purebasic.fr/english/viewtop ... 84#p428484 and re-copy the code. Now REM out the old Z_STREAM structure and use the following...
I did reproduce your experiences with the corruption, and this update will address it.

8e8c5f619da1de67d67480671fc2827c test.dds -- Original
b6c1f472b21d542185e4e4a78a569b12 test.gz -- Created
8e8c5f619da1de67d67480671fc2827c test2.dds -- Extracted

And my extraction w/another decompressor of the test.gz file
8e8c5f619da1de67d67480671fc2827c test

And doesn't matter if you using 32bit or 64bit compiler.

Hi Thunder93,

I gave moob the same Structure layout that your suggesting, but that caused a different set of problems (see previous posts)... the modified Structure was only to test with, as was the compression ratio... but definitely worth revisiting - good call.

Hi moob,

Try testing the compressed .gz file from http://2zip.org/ (site: thanks to Thunder93) - see if that makes a difference?
- please note that all the examples and suggestions have been tested before hand - and found working
-- this may indicate an incorrect setting under "Compiler Options" or elsewhere

'Align' keyword support for structure was introduced w/Version 5.10 of PB and moob using 5.00. I can't personally imagine not updating already or installing the later versions along side of that current version.

If moob is coding under x64 system, you'll have to have two source files one with and another without the structure paddings.

Thunder93 wrote:'Align' keyword support for structure was introduced w/Version 5.10 of PB and moob using 5.00. I can't personally imagine not updating already or installing the later versions along side of that current version.

If moob is coding under x64 system, you'll have to have two source files one with and another without the structure paddings.

I'm stuck in PB 5.00 because I need to use the DevIL lib for convert png to dds and only found an wrapper working in version 5.00

And I don't know how to update the code of wrapper to work in PB 5.20.

I'm stuck in version 5.00 because of this and 'Align' keyword not work in PB 5.00

JHPJHP wrote:Hi Thunder93,

I gave moob the same Structure layout that your suggesting, but that caused a different set of problems (see previous posts)... the modified Structure was only to test with, as was the compression ratio... but definitely worth revisiting - good call.

Hi moob,

Try testing the compressed .gz file from http://2zip.org/ (site: thanks to Thunder93) - see if that makes a difference?
- please note that all the examples and suggestions have been tested before hand - and found working
-- this may indicate an incorrect setting under "Compiler Options" or elsewhere

No diference, the size is the same 116.81KB

Thanks moob. I overlooked you mentioning that earlier.

Replace the previous structure for 5.00 with this one, and try your luck

Hi Thunder93,

The file size being returned from moob's tests seem to indicate a separate problem?