libjpeg-turbo static

[Keya]

So I "simply" embed the DLL in my exe and load the DLL from memory, there is some lib (which you can successfully link!) for that in the forum.
Or at worst you can extract it to temp and load from there.

sounds interesting, ive heard of people loading DLLs from memory before but not tried it
i found this thread http://www.purebasic.fr/english/viewtop ... 13&t=42118 is that the one you were thinking of or another?

[sys64802]

Yes, that's it.
The author, Joachim Bauch, has been probably the first one to make some code available publicly to do that.
I saw some others in the past in hacking "circles", but I think the one above is the first being widely used.
In short, you have to manually do what the loader of the OS does.
You have to parse the PE file and allocate virtual space where to copy it, extract the dependencies from the import section of the PE, manually load all the required dlls, and patch the address of every single function exported from them.
Then do somethings similar from the exported functions of the dll you are trying to embed, and relocate them if it was not possible to load the dll at the base address specified as the preferred one in the dll itself.
Then you have to set the permissions of the various sections of the PE you copied into the virtual space accordingly to what specified in the sections table, and some other minor stuff.
Then you have to write your own GetProcAddress() because the dll you manually loaded is not known to the OS (I think the handle of the module you created should be in some kernel table or such, and I don't know of a user mode API to do so, but I don't know the precise details), so you can't use the predefined API function to retrieve the address of the exported functions you want to use, and the same thing goes for releasing the dll (and its dependencies you loaded). You have to write your code for that too.
I wrote my own code for all of the above directly in PB, even if for now I consider it in alpha status (another way to say who knows if it's working correctly), I have to test it better.

[Michael Vogel]

Sounds quite complicate to include the dll, but would allow to use the 32 and 64 bit variants.
Can you give an example code, how the functions of the turbojpeg.dll could be used from PureBasic?

[wilbert]

What I'm more or less using with the static lib is something like this (cross platform).
It might help you out.

Code: Select all

Enumeration
  #TJPF_RGB    
  #TJPF_BGR    
  #TJPF_RGBX    
  #TJPF_BGRX
  #TJPF_XBGR
  #TJPF_XRGB
  #TJPF_GRAY
  #TJPF_RGBA
  #TJPF_BGRA
EndEnumeration

Enumeration
  #TJSAMP_444    
  #TJSAMP_422    
  #TJSAMP_420    
EndEnumeration

#TJFLAG_BOTTOMUP    = 2
#TJFLAG_FASTDCT     = 2048
#TJFLAG_ACCURATEDCT = 4096

CompilerSelect #PB_Compiler_OS
  CompilerCase #PB_OS_Windows
    #LIBTURBO = "turbojpeg-static.lib"
  CompilerCase #PB_OS_MacOS
    #LIBTURBO = "libturbojpeg_osx.a"
  CompilerCase #PB_OS_Linux
    #LIBTURBO = "libturbojpeg_lin64.a"
CompilerEndSelect

ImportC #LIBTURBO
  tjAlloc(bytes)
  tjCompress2.l(handle, *srcBuf, width, pitch, height, pixelFormat, *p_jpegBuf, *jpegSize, jpegSubsamp, jpegQual, flags)
  tjDecompress2.l(handle, *jpegBuf, jpegSize, *dstBuf, width, pitch, height, pixelFormat, flags)
  tjDecompressHeader3.l(handle, *jpegBuf, jpegSize, *width, *height, *jpegSubsamp, *jpegColorspace)
  tjFree(*buffer)
  tjGetErrorStr()   
  tjInitCompress()
  tjInitDecompress()
EndImport

Global tjC, tjD
tjC = tjInitCompress()
tjD = tjInitDecompress()

; TJPF from PB PixelFormat
Procedure TJPF_(Format)
  If Format & #PB_PixelFormat_32Bits_RGB
    ProcedureReturn #TJPF_RGBA
  ElseIf Format & #PB_PixelFormat_24Bits_RGB
    ProcedureReturn #TJPF_RGB
  ElseIf Format & #PB_PixelFormat_32Bits_BGR
    ProcedureReturn #TJPF_BGRA
  Else
    ProcedureReturn #TJPF_BGR
  EndIf
EndProcedure

Procedure.i CatchImage_(Image, *Memory, Size)
  Protected.i result, w, h, subs, cs, r, pf, flags
  If PeekL(*Memory) & $ffffff <> $ffd8ff
    ProcedureReturn CatchImage(Image, *Memory, Size)
  ElseIf tjDecompressHeader3(tjD, *Memory, Size, @w, @h, @subs, @cs) = 0 And w And h
    result = CreateImage(Image, w, h, 24)
    If result
      If Image = #PB_Any : Image = result : EndIf
      StartDrawing(ImageOutput(Image))
      pf = DrawingBufferPixelFormat() : flags = #TJFLAG_ACCURATEDCT
      If pf & #PB_PixelFormat_ReversedY : flags | #TJFLAG_BOTTOMUP : EndIf
      r = tjDecompress2(tjD, *Memory, Size, DrawingBuffer(), w, DrawingBufferPitch(), h, TJPF_(pf), flags)
      StopDrawing()
      If r
        FreeImage(Image) : result = 0
      EndIf
    EndIf    
  EndIf
  ProcedureReturn result
EndProcedure


Procedure LoadJPEG(Image, FileName.s)
  Protected.i result, fhandle, size, w, h, subs, cs, r, pf, flags, sig.l, *buffer.Long
  
  fhandle = ReadFile(#PB_Any, FileName)
  If fhandle
    size = Lof(fhandle)
    If size > 96
      sig = ReadLong(fhandle)
      If sig & $ffffff = $ffd8ff; possible jpeg
        *buffer = AllocateMemory(size, #PB_Memory_NoClear)
        If *buffer
          *buffer\l = sig
          ReadData(fhandle, *buffer + 4, size - 4)
          ; jpeg decompression
          If tjDecompressHeader3(tjD, *buffer, size, @w, @h, @subs, @cs) = 0 And w And h
            ; create image
            result = CreateImage(Image, w, h, 24)
            If result
              If Image = #PB_Any : Image = result : EndIf
              StartDrawing(ImageOutput(Image))
              pf = DrawingBufferPixelFormat() : flags = #TJFLAG_ACCURATEDCT
              If pf & #PB_PixelFormat_ReversedY : flags | #TJFLAG_BOTTOMUP : EndIf
              r = tjDecompress2(tjD, *buffer, size, DrawingBuffer(), w, DrawingBufferPitch(), h, TJPF_(pf), flags)
              StopDrawing()
              If r
                FreeImage(Image) : result = 0
              EndIf
            EndIf
          EndIf
          FreeMemory(*buffer) 
        EndIf
      EndIf
    EndIf
    CloseFile(fhandle)
  EndIf
  
  ProcedureReturn result  
EndProcedure

Procedure SaveJPEG(Image, FileName.s, quality = 95)
  Protected.i result, pf, flags, r, size, fhandle, *buffer
  If IsImage(Image) And StartDrawing(ImageOutput(Image))
    pf = DrawingBufferPixelFormat()
    If quality < 1 : quality = 1 : ElseIf quality > 100 : quality = 100 : EndIf
    If quality > 80 : flags = #TJFLAG_ACCURATEDCT : Else : flags = #TJFLAG_FASTDCT : EndIf
    If pf & #PB_PixelFormat_ReversedY : flags | #TJFLAG_BOTTOMUP : EndIf
    r = tjCompress2(tjC, DrawingBuffer(), OutputWidth(), DrawingBufferPitch(), OutputHeight(), TJPF_(pf), @*buffer, @size, #TJSAMP_444, quality, flags)
    StopDrawing()
    If r = 0
      fhandle = CreateFile(#PB_Any, FileName)
      If fhandle
        If WriteData(fhandle, *buffer, size) = size
          result = #True
        EndIf
        CloseFile(fhandle)
      EndIf
    EndIf
    tjFree(*buffer)
  EndIf
  ProcedureReturn result
EndProcedure

[Michael Vogel]

Thank you, Wilbert - this will help a lot!

I am only wondering why it is not possible to include UseJPEGImageDecoder() and UseJPEGImageEncoder() to allow LoadJpeg().
You normally wouldn't need it anymore, because the turbo library seems to be around 50% faster and allow to set a standard quality value when saving jpeg images.

But...
... I am using more than 50 jpeg and png images in my exe file (IncludeBinary) which get loaded by CatchImage. And now it gets complicated...

Code: Select all

CompilerIf #UseTurboJpegLibrary=0
	UseJPEGImageDecoder()
	UseJPEGImageEncoder()

CompilerElse
	:
	Procedure LoadJPEG(Image,FileName.s)
		:
	EndProcedure
	Procedure SaveJPEG(Image,FileName.s,quality = 95)
		:
	EndProcedure
	Macro LoadImage(Image,Filename)
		LoadJPEG(Image,Filename)
	EndMacro
	Macro SaveImage(Image,Filename,Quality)
		SaveJPEG(Image,Filename,Quality*10)
	EndMacro
CompilerEndIf

When I exclude the PureBasic encoder/decoder, CatchImage will fail for the (jpeg) images, when I include the encoder/decoder, the turbo lib doesn't work. Maybe a new CatchImage replacement can be done, but then I have to create one which supports png format as well. Or Fred will include the library to be used with standard commands (okay, this won't happen).

Maybe you can give me a hint...

[wilbert]

When I exclude the PureBasic encoder/decoder, CatchImage will fail for the (jpeg) images, when I include the encoder/decoder, the turbo lib doesn't work. Maybe a new CatchImage replacement can be done, but then I have to create one which supports png format as well. Or Fred will include the library to be used with standard commands (okay, this won't happen).

Maybe you can give me a hint...

libjpeg-turbo has two api sets. One is the one I'm using, the other the libjpeg api. It can be a drop in replacement for the normal libjpeg lib.
So if Fred would like to switch, that would be easy for him and it's not some kind of obscure lib; libjpeg-turbo is also used by Chrome, Firefox and IrfanView.
This compatibility with libjpeg might explain why you can't use it together with the jpeg encoder / decoder that comes with PureBasic.
Fortunately adding a catch image procedure isn't that difficult. I added it to the code above. The only thing is that it needs the size of the memory area. If that's a problem, some extra code would be required which first determines the size of the jpeg if there is one.

[Michael Vogel]

Oh, thanks...
...started imediately to write a jpeg parser to get the memory size, but after a while I also tried to set different values for size: 0 fails but $fffffff works fine for catchimage.

So I changed the procedure code to CatchImage_(Image,*Memory,Size=$FFFFFFF)

Anyhow, it gets more and more complicate to adapt my program, because it uses CatchImage (and SaveImage as well) with a different number of parameters (so macros can't do the job) and allows to load/save different file formats (so it will take a while to adapt all these things)

Fre e e e e d!
What about adding UseJPEGTurboImageEncoder() and UseJPEGTurboImageDecoder() which can be used alternatively to the standard library? That would be great and would allow to get rid of the quality level issue (and add some additional sampling method flags) even without any compatibility issues.

Something like this could be the result for a adapted source...

Code: Select all

CompilerIf #UseTurboJpegLibrary=1

	UseJPEGTurboImageDecoder()
	UseJPEGTurboImageEncoder()
	
	#JpegQualityFactor=10
	#JpegPlugin=#PB_ImagePlugin_JpegTurbo
	#JpegFilter=#PB_Image_DefaultSampling

CompilerElse

	UseJPEGImageDecoder()
	UseJPEGImageEncoder()
	
	#JpegQualityFactor=1
	#JpegPlugin=#PB_ImagePlugin_JPEG
	#JpegFilter=#Null
	
CompilerEndIf

SaveImage(0,"Test.jpg",#JpegPlugin|#JpegFilter,9*#JpegQualityFactor)

[wilbert]

Anyhow, it gets more and more complicate to adapt my program, because it uses CatchImage (and SaveImage as well) with a different number of parameters (so macros can't do the job) and allows to load/save different file formats (so it will take a while to adapt all these things)

What you could do is what I did with CatchImage_ ; forward to the existing PB commands when it's not a jpeg image.

Fre e e e e d!
What about adding UseJPEGTurboImageEncoder() and UseJPEGTurboImageDecoder() which can be used alternatively to the standard library? That would be great and would allow to get rid of the quality level issue (and add some additional sampling method flags) even without any compatibility issues.

If Fred would do that, that would of course be the easiest option

[davido]

@Michael Vogel,

if libjpeg-turbo is an improvement on the current offering, then...

+1 for UseJPEGTurboImageEncoder() and UseJPEGTurboImageDecoder()

[Michael Vogel]

A new version of the turbo jpeg library is available (1.5), was anyone able to use it together with purebasic? A quick try did not work, a lot of unresolved external symbols have been listed...

Code: Select all

Enumeration
	#TJPF_RGB
	#TJPF_BGR
	#TJPF_RGBX
	#TJPF_BGRX
	#TJPF_XBGR
	#TJPF_XRGB
	#TJPF_GRAY
	#TJPF_RGBA
	#TJPF_BGRA
EndEnumeration

Enumeration
	#TJSAMP_444
	#TJSAMP_422
	#TJSAMP_420
EndEnumeration

#TJFLAG_BOTTOMUP=	2
#TJFLAG_FASTDCT=		2048
#TJFLAG_ACCURATEDCT=	4096

ImportC "turbojpeg-static.lib"
	tjAlloc(bytes)
	tjCompress2.l(handle,*srcBuf,width,pitch,height,pixelFormat,*p_jpegBuf,*jpegSize,jpegSubsamp,jpegQual,flags)
	tjDecompress2.l(handle,*jpegBuf,jpegSize,*dstBuf,width,pitch,height,pixelFormat,flags)
	tjDecompressHeader3.l(handle,*jpegBuf,jpegSize,*width,*height,*jpegSubsamp,*jpegColorspace)
	tjFree(*buffer)
	tjGetErrorStr()
	tjInitCompress()
	tjInitDecompress()
EndImport

Global tjC,tjD
tjC=tjInitCompress()
tjD=tjInitDecompress()

[wilbert]

A new version of the turbo jpeg library is available (1.5), was anyone able to use it together with purebasic? A quick try did not work, a lot of unresolved external symbols have been listed...

Thanks for mentioning Michael
The 1.5 release should provide for faster encoding of color images

The static version for OSX seems to work fine without any changes.
On Windows those symbol errors were present on 1.4 also initially. Keya did a great job at fixing this.
If Keya wants to do the same for the 1.5 release, that would be great. If not, I might try myself but I'm not as good at this as Keya.

[Michael Vogel]

On Windows those symbol errors were present on 1.4 also initially. Keya did a great job at fixing this.
If Keya wants to do the same for the 1.5 release, that would be great. If not, I might try myself but I'm not as good at this as Keya.

32Bit = +1
64Bit = +2

[Keya]

I'm not actually using this lib myself so really it would be best if somebody using it could compile it, then you can keep on top of all the latest releases straight away, besides I thought I'd solved and documented in this thread all the challenges in compiling it Plus it's just a great feeling when you get those darn things to compile! lol (the powertrip, whoa!). So I encourage you to grab a free copy of Visual Studio and check back over my notes and try to compile it yourselves, it was actually fairly easy in the end - just figuring out those steps wasn't (I don't do any C programming so it was all reaching around in the dark for me), but if you don't have any luck of course I will do my best

[Keya]

days of silence = mega guilt trip, lol

Hopefully some of you had a go at least! Anyway i will try to list here step-by-step instructions for building this library and setup of all tools required.

Setup - only need to do once, will then have Visual Studio and accessories nicely set up ready for future libs.

1. Install Visual Studio 2013
2010 also works, 2008 might also work, 2015 is the one recommended by libjpeg itself but I had problems registering that, so im just using 2013 for this. Here is ISO for VS2013 Ultimate (2.8gb) - http://download.microsoft.com/download/ ... LT_ENU.iso

2. Install CMake, and during install tell it to add itself to the Path environment variable.
http://cmake.org/download/

3. Install Nasm
http://www.nasm.us

4. add Nasm to the Path variable
(Find that Computer icon, usually on your Desktop or in Start menu)...
Computer -> Properties -> Advanced Settings -> Advanced -> Environment Variables
go to PATH and add ";c:\nasm" or whatever to the end - the CMake path will probably be just before it.

5. If you have any console windows or Visual Studio open then close them down and restart them for new Path to take effect.

SETUP COMPLETE!!!

Building a lib
Now to compile a lib, for this example using libjpegturbo ...
1. Download the source bundle from https://sourceforge.net/projects/libjpe ... rce=navbar
Or specifically 1.5.0: https://sourceforge.net/projects/libjpe ... z/download

Opening it up we see it doesn't come with Visual Studio solutions! So their actual recommendation from the command-line is like this:

Code: Select all

cd c:\libjpeg
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release c:\libjpeg
nmake

And that works, so that's one option!

But using VISUAL STUDIO makes it easier i think.

1. Extract the source code to TWO directories, for example C:\libjpeg32 and C:\libjpeg64
Yes just two copies of the exact same code.

2. Open a new command prompt:

Code: Select all

cd c:\libjpeg32
cmake -G "Visual Studio 12 2013"

As simple as that it generates full VS2013 x86 solution for us!
Now do similar for x64:

Code: Select all

cd c:\libjpeg64
cmake -G "Visual Studio 12 2013 Win64"

You should notice the x64 vs x86 differences in the output of both.
Anyway, solutions for x86 and x64 are now ready!

3. Open VS2013, File -> Open Solution, open 32-bit one

4. Right-click on the Solution and click Configuration Manager menu, and change the dropdown "Configuration:" from Debug to Release. Also make sure the Platform dropdown is Win32 or x64 depending what you want to build

5. Right-click not on the Solution but a Project under it, and click Properties. Here you can specify compiler optimizations, for example you can Disable Security Checks, and ensure Whole Program Optimization is off also - these two are important for PB use.

6. Right-click on the solution and click the Build menu (or just press F7), and wait for the build to complete.

Then restart VS2013 and open the x64 solution and do the same with that. You should now have everything built including static libs, in \Release\ and \x64\Release\ directories, or \Debug\ if you didn't correctly change the Configuration over.

Then if you encounter any of the previous problems we had when initially trying to figure this all out, like having to comment out the vsprintf line, see back for my previous comments about how those particular ones at least were fixed. (the vsnprintf one i think is only an XP issue)

[wilbert]

days of silence = mega guilt trip, lol

Hopefully some of you had a go at least!

Not yet, have some other things on my mind at the moment.
I am planning to give it a try somewhere next week.