TSI include, new image file format
- DoubleDutch
- Addict
- Posts: 3220
- Joined: Thu Aug 07, 2003 7:01 pm
- Location: United Kingdom
- Contact:
Re: TSI include, new image file format
When you have multiple palettes for the same image, you have to make sure that whatever compression system is used that it doesn't rearrange colours or is colour lossy - otherwise only the palette used for the original image will be correct. I sometimes use a 16 colour palette system that has 8 colours (0 to 7, 0 being transparent) that are consistant and then the next 4 colours can be picked from a 4 colour palette and the next 4 colours can also be picked from a 4 colour palette. This system is great for bullets, powerups, etc. Other times I've had a 16 colour image (no transparency) and a 16 shade alpha level - so each pixel is 8 bits. The 16 colour image having its 16 colours picked from a palette. This is great for bullets, explosions, effects, etc
https://deluxepixel.com <- My Business website
https://reportcomplete.com <- School end of term reports system
https://reportcomplete.com <- School end of term reports system
Re: TSI include, new image file format
Just to let you people know: I was sucked into the game Torchlight and have made no progress on the TSI format so far. But i am back on the project now and decided to redesign the format. Mainly for adding more image data formats (pixel formats).
So the format version will go up to 2 with the next version of the include and it will no longer support TSI version 1 files.
The format version 2 will support a total of 22 image data formats:
Every indexed format have 3 subformats. For example a 8bit indexed image (256 color image) can have a palette with 24bit entries (RGB) or 32bit entries (RGBA). So you can make transparency a part of the palette, if you want so. But you can use the "8bit indexed 24bit colors + alpha channel" format if you want to have your 256 colors and a seperate alpha channel with 8 additional bit per pixel. So a 8bit indexed image with alpha channel will need 16bit per pixel. I think this is very usefull if you want to use indexed images with palette changing.
Another feature will be the multi palette support. You can store multiple palettes in one TSI file. This is for easy palette swapping. Entries in the palettes will not be rearranged by the encoder. So you can manipulate them at runtime, if you want.
So the format version will go up to 2 with the next version of the include and it will no longer support TSI version 1 files.
The format version 2 will support a total of 22 image data formats:
- 32bit BGRA
- 32bit RGBA
- 24bit BGR
- 24bit RGB
- 1bit indexed 32bit colors
- 1bit indexed 24bit colors
- 1bit indexed 24bit colors + alpha channel
- 2bit indexed 32bit colors
- 2bit indexed 24bit colors
- 2bit indexed 24bit colors + alpha channel
- 4bit indexed 32bit colors
- 4bit indexed 24bit colors
- 4bit indexed 24bit colors + alpha channel
- 8bit indexed 32bit colors
- 8bit indexed 24bit colors
- 8bit indexed 24bit colors + alpha channel
- 1bit greyscale
- 2bit greyscale
- 4bit greyscale
- 8bit greyscale
- 16bit greyscale
- 32bit greyscale
Every indexed format have 3 subformats. For example a 8bit indexed image (256 color image) can have a palette with 24bit entries (RGB) or 32bit entries (RGBA). So you can make transparency a part of the palette, if you want so. But you can use the "8bit indexed 24bit colors + alpha channel" format if you want to have your 256 colors and a seperate alpha channel with 8 additional bit per pixel. So a 8bit indexed image with alpha channel will need 16bit per pixel. I think this is very usefull if you want to use indexed images with palette changing.
Another feature will be the multi palette support. You can store multiple palettes in one TSI file. This is for easy palette swapping. Entries in the palettes will not be rearranged by the encoder. So you can manipulate them at runtime, if you want.
- DoubleDutch
- Addict
- Posts: 3220
- Joined: Thu Aug 07, 2003 7:01 pm
- Location: United Kingdom
- Contact:
Re: TSI include, new image file format
nice. 
Would have been handier if you could specify the number of bits per pixel (for 32 colour sprites)

Would have been handier if you could specify the number of bits per pixel (for 32 colour sprites)

https://deluxepixel.com <- My Business website
https://reportcomplete.com <- School end of term reports system
https://reportcomplete.com <- School end of term reports system
Re: TSI include, new image file format
That would be 5bit per pixel. I can guarantee you that this would be very slow to decode. Because there would be a lot of bit shifting involved, which would slow down decoding and encoding dramaticly.DoubleDutch wrote: Would have been handier if you could specify the number of bits per pixel (for 32 colour sprites)
Why do you need 5bit depth? Why not just use 8bit depth?
Re: TSI include, new image file format
Great work, Thorium.
It beats PNG (optimized) in size on all files I tried! And the speed - simply ownage!
Hope you will improve it even more (more pixel data formats).
PS Ever though about improving TSI even more (TSI2 or something) sometime in future by adding animation support?
Nowadays there's no modern image format supporting it (GIF is old shit). APNG is some alternative, but sadly it's not widely supported. TSI would be better as FASTER and possibly SMALLER.
It beats PNG (optimized) in size on all files I tried! And the speed - simply ownage!
Hope you will improve it even more (more pixel data formats).
PS Ever though about improving TSI even more (TSI2 or something) sometime in future by adding animation support?
Nowadays there's no modern image format supporting it (GIF is old shit). APNG is some alternative, but sadly it's not widely supported. TSI would be better as FASTER and possibly SMALLER.

Re: TSI include, new image file format
Thanksklaver wrote:Great work, Thorium.
It beats PNG (optimized) in size on all files I tried! And the speed - simply ownage!
Hope you will improve it even more (more pixel data formats).
PS Ever though about improving TSI even more (TSI2 or something) sometime in future by adding animation support?
Nowadays there's no modern image format supporting it (GIF is old shit). APNG is some alternative, but sadly it's not widely supported. TSI would be better as FASTER and possibly SMALLER.
and yes it will be called TSA and will have a special compression that saves differences between the frames. A alpha version of a TSA encoder and decoder allready works but missing some important features. I want to concentrate on the more simple TSI first.
Re: TSI include, new image file format
Project is still alive.
After i redesigned the format a few times i am finaly pleased with it.
Today i finished the new core decoder procedure and the decoder procedures for transformations and filtering for the new image data formats. So, the most work should be done. At least the hard work is done. Now i have to do some more easy work and i hope to release the all new TSI version in one week.
I even added 3 more image data formats as i wrote: 16bit, 48bit and 64bit

After i redesigned the format a few times i am finaly pleased with it.
Today i finished the new core decoder procedure and the decoder procedures for transformations and filtering for the new image data formats. So, the most work should be done. At least the hard work is done. Now i have to do some more easy work and i hope to release the all new TSI version in one week.
I even added 3 more image data formats as i wrote: 16bit, 48bit and 64bit
Re: TSI include, new image file format
Not realy.klaver wrote:Any news on TSI & TSA?
I was been asked to join a professional game development team and thats a chance i can't refuse. ^^
So i put all my time in our projects, which are currently only for the iPhone. And my time is very limited because i do a regular job 8 hours per day until we generated some money.
I will write a TSI decoder for iPhone OpenGL ES and i will work on the PB include but i can't say when i actualy have the time to do that.
Re: TSI include, new image file format
Thus an end to a great project
RIP TSI 


Re: TSI include, new image file format
Hey Thorium, any chance on giving some open source to the community?
If only to make your name in developers land, wouldn't it be great if you would be famous for donating a new improved image standard to the world? Do it for the fame and honour!

If only to make your name in developers land, wouldn't it be great if you would be famous for donating a new improved image standard to the world? Do it for the fame and honour!

( PB6.00 LTS Win11 x64 Asrock AB350 Pro4 Ryzen 5 3600 32GB GTX1060 6GB)
( The path to enlightenment and the PureBasic Survival Guide right here... )
( The path to enlightenment and the PureBasic Survival Guide right here... )
Re: TSI include, new image file format
Put in on GoogleCode... or Sourceforge
Re: TSI include, new image file format
Well it isnt a lib. It's just source code in a include. So it's open source if you want to call it so.blueznl wrote:Hey Thorium, any chance on giving some open source to the community?
If only to make your name in developers land, wouldn't it be great if you would be famous for donating a new improved image standard to the world? Do it for the fame and honour!
And the include for TSI1 is still online and will be until TSI2 is out.
The project is not dead it's just delayed, i promise.

The thing with TSI2 is that i need some time for it that i currently not have but i will have that time. We will use TSI and TSA in our projects, so it will become part of my development. But for now we have no time for it.
I could post the core decoder for TSI2 but for most people it would be useless without the procedures for putting it into a image or sprite or texture. And the core encoder isnt finished.
Re: TSI include, new image file format
If you want the format specifications:blueznl wrote:Hey Thorium, any chance on giving some open source to the community?
If only to make your name in developers land, wouldn't it be great if you would be famous for donating a new improved image standard to the world? Do it for the fame and honour!
TSI v1
Code: Select all
TSI (Thorium Sprite Image) format structure
------
header
------
QWord signatur = 'ThorSpIm'
Byte format version = 1
Byte flags
Word width
Word height
-------------------------------------------------------------
compression header (only in file if image data is compressed)
-------------------------------------------------------------
DWord uncompressed size of image data
DWord compressed size of image data
----------
image data
----------
32bit per pixel (BGRA)
Flags: Bit 0: image data compressed with ZLib
Bit 1: image data compressed with Thorium compression
Bit 2: image data is transformed with alpha transformation
Bit 3: image data is transformed with channels transformation
Bit 4: image data is filtered with sub filter
Bit 5: image data is filtered with up filter
Bit 4+5: image data is filtered with average filter
Transformations are moving the bytes of the image data around in order to make it bedder compressible, only one transformation can be used per file.
Alpha Channel Transformation: The Alpha channel is seperated from the pixel data and wrote to the beginning of the image data. So if that bit is set the image data looks like that: "alpha channel as byte array followed by BGR array".
Channels Transformation: All channels are seperated so if that bit is set image data looks like that: "blue channel as byte array followed by green channel as byte array followed by red channel as byte array followed by alpha channel as byte array"
Filters are changing the values of the pixels in order to make the image bedder compressible. The used filters are 100% reversible, so they are lossless. Only one filter can be used per file. TSI supports 3 filters of the PNG format: Sub Filter, Up Filter and Average Filter. For information how they work take a look here: http://www.w3.org/TR/PNG-Filters.html
TSI v2
TSI2 supports a lot more. First off all it supports different image data formats, not only 32bit BGRA. Take a look of the image data format constants list. Second it does support LZMA compression in addition to ZLib. Third it does support multible palettes for the indexed image data formats.
And it supports a bunch of optimization setting to optimize a TSI for a specific plattform or API. It supports BGR(A), (A)BGR, RGB(A), (A)RGB, reversed Y order and big endian byte order. All optional optimizations so your target plattform don't have to convert any data and save a lot of time on decoding. Especially interessting for devices with slow CPU's, pixel convertion is very pricey on that devices.
If you want to make your custom decoder or loader for another plattform or API it does not have to support all features of the TSI, only whats relevant for the plattform or API. If you use it on games you can even cut of the checks if the TSI have the right settings for your decoder. For example your decoder does not have to check for the byte order if all your images for your game are the same byte order.
So here is the new structure of the TSI file format. It has been simplified a bit and changed here and there.
Code: Select all
TSI (Thorium Sprite Image) format structure
/-----------\
| structure |
\-----------/
------
header
------
8 Bytes signatur = 'ThorSpIm'
Byte format version = 2
Byte flags
Byte compression
Byte filter
Byte transformation
Byte image data format
Byte count of transparent colors
Byte count of paletts
Word width
Word height
DWord compressed size
-----------------------------
transparent color information
-----------------------------
xxx transparent colors
-----------------------------
paletts
-----------------------------
image data
-----------------------------
/-------\
| flags |
\-------/
Bit 0: big endian byte order (little endian if not set)
Bit 1: image data is Y reversed
Bit 2: pixel format is RGB (BGR if not set)
Bit 3: alpha channel is first channel (alpha channel is last channel if not set)
/-----------------------\
| compression constants |
\-----------------------/
000: data is not compressed
001: data is compressed with ZLib
002: data is compressed with LZMA
003: data is compressed with Thorium compression (for future use)
/------------------\
| filter constants |
\------------------/
000: image is not filtered
001: image is filtered with sub filter
002: image is filtered with up filter
003: image is filtered with average filter
/--------------------------\
| transformation constants |
\--------------------------/
000: image is not transformed
001: image is transformed with alpha transformation
002: image is trandformed with channels transformation
/-----------------------------\
| image data format constants |
\-----------------------------/
001: 16bit 3 channels
002: 24bit 3 channels
003: 32bit 4 channels
004: 48bit 3 channels
005: 64bit 4 channels
006: 1bit indexed 24bit colors
007: 1bit indexed 32bit colors
008: 1bit indexed 24bit colors + alpha channel
009: 2bit indexed 24bit colors
010: 2bit indexed 32bit colors
011: 2bit indexed 24bit colors + alpha channel
012: 4bit indexed 24bit colors
013: 4bit indexed 32bit colors
014: 4bit indexed 24bit colors + alpha channel
015: 8bit indexed 24bit colors
016: 8bit indexed 32bit colors
017: 8bit indexed 24bit colors + alpha channel
018: 1bit greyscale (1 channel)
019: 2bit greyscale (1 channel)
020: 4bit greyscale (1 channel)
021: 8bit greyscale (1 channel)
022: 16bit greyscale (1 channel)
023: 32bit greyscale (1 channel)