libjpeg-turbo static

[wilbert]

Has anyone here ever successfully used the static lib of libjpeg-turbo ?

I wanted to try it to see if it decodes jpeg much faster or not.
On both OSX and Linux I could use the pre-build static lib without much problems but on Windows 10 I get linker errors I don't know how to solve.
The unresolved external symbols are __ms_vsnprintf and _putenv .

Example code

Code: Select all

DataSection
  jpeg_start:
  IncludeBinary "MyImage.jpg"
  jpeg_end:
EndDataSection

Enumeration
  #TJPF_RGB 	
  #TJPF_BGR 	
  #TJPF_RGBX 	
  #TJPF_BGRX 	
EndEnumeration

Enumeration
  #TJSAMP_444 	
  #TJSAMP_422 	
  #TJSAMP_420 	
EndEnumeration

ImportC "libturbojpeg.a"
  tjAlloc(bytes)
  tjCompress2.l(handle, *srcBuf, width, pitch, height, pixelFormat, *p_jpegBuf, *jpegSize, jpegSubsamp, jpegQual, flags)
  tjDecompress2.l(handle, *jpegBuf, jpegSize, *dstBuf, width, pitch, height, pixelFormat, flags)
  tjDecompressHeader3.l(handle, *jpegBuf, jpegSize, *width, *height, *jpegSubsamp, *jpegColorspace)
  tjFree(*buffer)
  tjGetErrorStr()	
  tjInitCompress()
  tjInitDecompress()
EndImport


handle = tjInitDecompress()
tjDecompressHeader3(handle, ?jpeg_start, ?jpeg_end - ?jpeg_start, @width, @height, @subsamp, @cs)
*m = AllocateMemory(width * height * 4, #PB_Memory_NoClear)
tjDecompress2(handle, ?jpeg_start, ?jpeg_end - ?jpeg_start, *m, width, 0, height, #TJPF_RGBX, 0)
End

[JHPJHP]

Hi wilbert,

Works with just the DLL:
- ImportC "turbojpeg.lib" works if both turbojpeg.lib and turbojpeg.dll are included (v1.4.2)

Code: Select all

Enumeration
  #TJPF_RGB    
  #TJPF_BGR    
  #TJPF_RGBX    
  #TJPF_BGRX    
EndEnumeration

PrototypeC protoInitDecompress()
Global tjInitDecompress.protoInitDecompress

PrototypeC.l protoDecompressHeader3(handle, *jpegBuf, jpegSize, *width, *height, *jpegSubsamp, *jpegColorspace)
Global tjDecompressHeader3.protoDecompressHeader3

PrototypeC.l protoDecompress2(handle, *jpegBuf, jpegSize, *dstBuf, width, pitch, height, pixelFormat, flags)
Global tjDecompress2.protoDecompress2

libturbojpeg = OpenLibrary(#PB_Any, "libturbojpeg.dll")

If libturbojpeg
  tjInitDecompress = GetFunction(libturbojpeg, "tjInitDecompress")
  tjDecompressHeader3 = GetFunction(libturbojpeg, "tjDecompressHeader3")
  tjDecompress2 = GetFunction(libturbojpeg, "tjDecompress2")
  handle = tjInitDecompress()
  tjDecompressHeader3(handle, ?jpeg_start, ?jpeg_end - ?jpeg_start, @width, @height, @subsamp, @cs)
  *m = AllocateMemory(width * height * 4, #PB_Memory_NoClear)
  tjDecompress2(handle, ?jpeg_start, ?jpeg_end - ?jpeg_start, *m, width, 0, height, #TJPF_BGRX, 0)
EndIf : End

DataSection
  jpeg_start:
  IncludeBinary "test.jpg"
  jpeg_end:
EndDataSection

[Keya]

this page mentions a similar unresolved api issue: https://github.com/libjpeg-turbo/libjpe ... /issues/45
"If you need to use jpeg-static.lib or turbojpeg-static.lib, you'll need to either build your application with an older version of Visual C++ or build libjpeg-turbo using Visual C++ 2015."
also when building Visual Studio libs for PB it seems important to turn off global optimization in VS options, i dont know if thats related or not just putting it out there!
i have three versions of VS installed now so i might be able to help, only 32bit though and ive only ever built 1 lib before so hopefully its smooth sailing!
which versions have you tried?

[wilbert]

@JHPJHP,
My goal is to make something portable with no external dependancies so I'm trying to avoid a dll if possible.

@Keya,
I tried to install Vistual Studio myself but I'm so unfamiliar with it that I don't know where to start to build such a lib.
My initial tests on OSX and Linux seem to indicate that libjpeg-turbo decodes a jpeg file in about half the time it takes the decoder that comes with PB so it would be nice if it could be used.
The source can be found here https://sourceforge.net/projects/libjpe ... z/download
32 bit would be fine for Windows. The compiled file that is required for use in PB is named turbojpeg-static.lib

[JHPJHP]

Hi wilbert,

The DLL or LIB + DLL versions work in both x86 / x64 -- Ascii / Unicode.

My goal is to make something portable with no external dependancies

The Microsoft Visual Studio C/C++ Runtime Library is an external dependancy.
- Window OS, architecture, and RT build version need to be considered

[Keya]

well, this is intimidating for me as ive only built one LIB before and that was with a Visual Studio Solution file that was click-and-compile and didnt give any errors/warnings

but this has no Visual Studio files! for this i had to also install CMake, and then also NASM! as per https://github.com/libjpeg-turbo/libjpe ... UILDING.md

https://github.com/libjpeg-turbo/libjpe ... UILDING.md
Visual C++ (command-line)
cd {build_directory}
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release {source_directory}
nmake

Here are the results of the compile, lots of warnings and a couple of macro errors but they look minor: http://pastebin.com/xVgpAxqR
But it did compile! 32bit jpeg-static.lib: https://www.sendspace.com/file/ml55q0
(604k lib -> 159k zip) but ive got no idea if it still has the same problems as the existing precompiled binaries you refered to in OP. You might want to try the libs from a really old version just to check if the problem you found is related to new builds
This is for v1.4.2 btw, the last non-beta, and i used VC2005
I will try compile a couple other versions now

I had problems compiling the very latest beta, ill try to have another go soon, but I also tried a really early version, v1.2.1 from March 2014, and it compiled in VC2005 really smoothly - no warnings or errors! and it compiled EVERYTHING, whereas my previous compile was incomplete (but at least it didnt fall over until one of the libs was built)
https://www.sendspace.com/file/rla0v4
(442kb zip: jpeg-static.lib, turbojpeg-static.lib, turbojpeg.lib, and turbojpeg.dll)
Anyway hopefully by going back to 1.2.1 it will work and we can then move forward to find optimal version to use that works!

1.4.1 compiled clean too! 533kb, same four files - https://www.sendspace.com/file/svchez

One thing though is that ive just run cmake/nmake "as is", i havent been able to turn off that Global Optimiztion thing that was necessary in a previous lib (not that i know if thats required for this or not) as that was a Visual Studio setting and im just running this from the commandline as per its build instructions

I retried the very latest beta, as well as 1.4.2 but ive had no luck with those too, so it seems 1.4.1 is the best/latest i can do but at least thats still fairly recent (Sept 2015) and it was an error-free build. But my builds might just be the same as the precompiled binaries that come with it so the problem you mentioned might still be the same, i dont know
anyway fingers crossed there's some good results and i will await your feedback

btw i jst noticed in your source code demo you import a ".a" instead of .lib ... i noticed that in the GCC .exe download the \lib\ dir just has .a files, whereas the VC version just has .lib files. Youre probably already aware of that but just thought it was worth mentioning!

perhaps if none of these work MingW or Cygwin might be worth a try, though i have hardly any experience with those either. Or perhaps it just needs a different VC build like that other person suggested in my quote, but i only have VC 2005/2008/2010, so of course it suggests 2015 lol

[wilbert]

To use the turbo api which is easier to use, turbojpeg-static.lib is required (jpeg-static.lib won't do).
I tried all of your versions. Unfortunately they give the same kind of polink errors
Someone on the internet suggested to link against legacy_stdio_definitions.lib but I don't know if that makes a difference.
From the precompiled builds offered from their website, I tried both the VC and GCC versions. The GCC ones give a few less errors but still won't run.
I downloaded the community version of Visual Studio 2015 with VC2015 but it seems very complicated.
From what I've read, it also seems possible to cross compile (build the Windows lib on OSX or Linux) but that seems even more complicated.

To check if the lib works with PB, all you need are a few lines

Code: Select all

ImportC "turbojpeg-Static.lib"
  tjInitDecompress()
EndImport

handle = tjInitDecompress()
Debug handle

If everything went fine, debug should show a non zero value.

[Keya]

Someone on the internet suggested to link against legacy_stdio_definitions.lib but I don't know if that makes a difference.

im not sure how to do that and i checked all the files in my TDM-GCC-32 and all three VC directories but there's no *legacy* anything. Let me know if you want me to try anything against VC2005/2008/2010. It's compiler issues like this that are the main reason I dont like C/C++ lol, makes building just as slow/tricky as programming and i find it so frustrating getting stuck on issues that aren't core programming issues. Guitarists don't want to spend hours changing strings. "Portable", yeah if you never move the damn thing and always use the same version of the same compiler with the same version of the same headers linked against the same libs on the same OS I certainly don't take it for granted how easy PB makes cross-OS-compiling, and that's why you're stuck with me sorry Fred lol

[wilbert]

@Keya, I think I'll have to give it up for now
I was able to compile the source with the community version of Visual Studio 2015 but still the same linker errors when I try the lib in PB.
Maybe some settings need to be changed or some libs linked but I just haven't got the knowledge about this.
If you have a Windows version other than 10 (I only have Windows 10) and want to try the static lib with that, I'd appreciate it.
It's very well possible that it will work on a system without the new universal c runtime.

[Keya]

I came upon this thread http://www.purebasic.fr/english/viewtop ... =5&t=41247
and it gave me the idea to update the \PB\Purelibraries\Windows\Libraries\msvcrt.lib with the one from VC

so I did that and it got rid of all the errors except for the one for "_putenv":


I tried with 2005, 2008, 2010, same result - they all seemed to fix all the errors except for the _putenv one
I also searched *.lib for putenv and copied the small handful of files with that to the PB lib dir, but still the _putenv error remains, but it seems if we can solve that then it might be ok? i dont know how required it is, perhaps we can comment it out, or make a dummy lib with _putenv export!? lol

btw if you check that thread youll see Fred mentioned a fix which might also apply here...
in order to remove the the problem of unresolved symbol "__p__iob" Fred said to use "/D__iob_func=__p__iob" with the preprocessor, so it seems worth a try with _putenv, but I don't know exactly how/where
I just found MSDN "/D" page https://msdn.microsoft.com/en-us/library/hhzbb5c8.aspx
it suggests its a CL.EXE flag so ill try it now when building the lib, although it uses CMake which does that call so ... well, i'll try!

btw i think yes maybe we can comment-out putenv ...

Code: Select all

	if(flags&TJFLAG_FORCEMMX) putenv("JSIMD_FORCEMMX=1");
	else if(flags&TJFLAG_FORCESSE) putenv("JSIMD_FORCESSE=1");
	else if(flags&TJFLAG_FORCESSE2) putenv("JSIMD_FORCESSE2=1");

[wilbert]

I came upon this thread http://www.purebasic.fr/english/viewtop ... =5&t=41247
and it gave me the idea to update the \PB\Purelibraries\Windows\Libraries\msvcrt.lib with the one from VC

Does this also work the other way around ?
I mean if you compile the lib with the same C version that was used to compile msvcrt it should work ?

[Keya]

I dont know sorry but no i dont think so as i was still getting the putenv error when using any of the VC msvcrt.lib's
but i just tried then after commenting-out the calls to putenv (just five or so 3-line blocks like i posted above, all in the main turbojpeg.c file), and that removed the putenv POLINK error, so the Debug output from your minimal.pb test is now non-zero valid handle
1.4.1 libs minus putenv - https://www.sendspace.com/file/29kfg1
1.4.1 is the highest i can seemingly compile sorry but its Sept 2015 so not too bad. But perhaps from commenting-out the putenv's ive changed whether it uses or forces MMX/SSE etc, ive got no idea at the moment, but its a start! I think it's ok though because those flags only seem to be used for FORCING MMX/SSE, which i dont think we want anyway? perhaps the code can be modified to use normal variables instead of environment vars. I would still like to know how to apply the /D switch to cl.exe but am not sure as its hidden in the cmake/makefile fu
btw here is msvcrt(+d).lib from VC2005, VC2008, VC2010 [edit] ooh... msvcrtd is just the Debug build lol, ignore those ones i guess
*looks around for waiter* GONNA NEED SOME COFFEE OVER HERE!

[wilbert]

Thanks Keya, that's something to work with
The msvcrt.lib from VC2010 works best. For that one I don't have to replace the one that comes with PureBasic.
Simply adding

Code: Select all

ImportC "msvcrt2010.lib"
EndImport

also seems to do the trick (I renamed it to msvcrt2010.lib).

[wilbert]

I tried the workaround today on Windows 7.
It complained about MSVCRT100.dll missing.
Seems it's complicated on Windows to create something without dependencies on dll's.

[Keya]

i wonder if its as simple as searching for MSVCRT100 and replacing with MSVCRT? i mean it doesn't seem like a library that absolutely has to have the latest msvcrt, it seems like it should work with any? [edit] just had a quick search tho, couldnt find string msvcrt100 hey it was still a good idea!

I googled and found this page http://stackoverflow.com/questions/1016 ... in-vc-10-0
it mentions a light version of msvcrt which might be of use. But then other pages http://stackoverflow.com/questions/1788 ... ver-msvcrt mention its better to use msvcrt100, so maybe it's best to provide link to the redistributable. i know thats not what you wanted to hear though!
also interesting http://www.syndicateofideas.com/posts/f ... t-dll-hell

or how about this?
http://stackoverflow.com/questions/1063 ... or-windows

Right-click your project in the Solution Explorer window, Properties, C/C++, Code Generation, Runtime Library setting. Change it to /MTd. Repeat for the Release configuration, pick /MT

You will now link the static version of the CRT, any functions you use get directly linked into your EXE instead of getting them from msvcr100.dll at runtime. So you no longer have the requirement to deploy the DLL along with your program.

Avoid using this option if you create your own DLLs. It then gets to be important that those DLLs and the EXE use the exact same CRT so they'll all share the same CRT state and the same heap. Because if they don't then you'll have nasty problems with passing C++ objects or pointers that need to be released from one chunk of code to another. An AccessViolation if you are lucky, a memory leak if you are not.

there's a catch though - there is no Visual Studio project with this, it's cmake/nmake! but should still be possible via cmdline i think