PureBasic Interface to OpenCV

[wilbert]

I wrote some of these example over a year ago, but your post reminds me that people are seeing them for the first time...

Yes, I always skipped this thread because it was Windows only.

Is there enough information in the Constant #CV_DESCRIPTION (tooltip) for someone new to the interface to view the examples full effect?

I think in general there is. It's just that I usually don't expect to get this kind information as a tooltip. But once you know, it does the job fine.

The new example cv_poisson_blending_1.pb uses a powerful blending algorithm to merge two images seamlessly.

That's another nice example.

I got a question for you also about the supported image formats.
cvShowImage supports a lot of different pixel formats.
Is 8 bit unsigned what functions usually output or are there also functions that output 16 or 32 bit integer or floating point rgb values ?

[JHPJHP]

Hi wilbert,

Let me know if you would like any additional information.

Supported image formats (IplImage / CvMat):

•IPL_DEPTH_8U - unsigned 8-bit integer. Equivalent to CV_8U in matrix types.
•IPL_DEPTH_8S - signed 8-bit integer. Equivalent to CV_8S in matrix types.
•IPL_DEPTH_16U - unsigned 16-bit integer. Equivalent to CV_16U in matrix types.
•IPL_DEPTH_16S - signed 8-bit integer. Equivalent to CV_16S in matrix types.
•IPL_DEPTH_32S - signed 32-bit integer. Equivalent to CV_32S in matrix types.
•IPL_DEPTH_32F - single-precision floating-point number. Equivalent to CV_32F in matrix types.
•IPL_DEPTH_64F - double-precision floating-point number. Equivalent to CV_64F in matrix types.

cvShowImage is the de facto Function for viewing all formats.

In order to save the image to a PNG, JPG, etc. it would first need to be converted to 8U: cvConvertScale.

If creating a PureBasic / OpenCV hybrid (demo examples) then the Function cvShowImage is not needed, as long as the format is first converted to standard RGB(A) / 8U.
- optimally all the heavy lifting is done using the proven OpenCV functions, with the results viewed from a PureBasic ImageGadget

[wilbert]

Hi JHPJHP,

cvShowImage is the de facto Function for viewing all formats.

In order to save the image to a PNG, JPG, etc. it would first need to be converted to 8U: cvConvertScale.

If creating a PureBasic / OpenCV hybred (demo examples) then the Function cvShowImage is not needed, as long as the format is first converted to standard RGB(A) / 8U.

That's what I was looking at; what the fastest way would be when using PureBasic windows and event loop.
cvShowImage internally converts first to CV_8UC3 using cvConvertImage but since OSX doesn't support 24 bit images, it expands it to 32 bit after this.
Encode the image with OpenCV to bmp and use CatchImage is very slow so I was thinking about what formats would need to be supported for a direct conversion.

Is there any reason you mention cvConvertScale instead of cvConvertImage ?

[JHPJHP]

Hi wilbert,

Were you looking at the example pb_sepia.pb in the OSX demo package? Take a look at the example pb_webcam.pb, I used a different approach.

I don't believe the Function cvConvertImage supports type conversion, this is what the Function cvConvertScale is for; I've used it extensively.

I've updated the OSX interface with a couple more examples. Your previous instructions were helpful with the conversion.

The following examples were a couple of my favorites:
- cv_seam_carving_1.pb: deleting vertical seams in an image by calculating the intensity of color pixels (experimental)
- le_stitching_1.pb: stitch together 2 images in stages; based on the SIFT (Scale Invariant Feature Transform) algorithm

cv_seam_carving_1.pb (based on information I read in Wikipedia)
- use the mouse to select (horizontally) the woman looking up at the castle (all columns will be selected)
- use the mouse to select (horizontally) the castle (all columns will be selected)
- press the Spacebar to start the process
- press the Spacebar again after the process has finished to enlarge the image

NB*: This technique could easily be expanded by narrowing the selection to an area, adding horizontal carving, etc., but my intention was to only demonstrate the process.

[wilbert]

Take a look at the example pb_webcam.pb, I used a different approach.

I have seen that but was thinking of trying to use asm or the vImage conversion functions (they are very fast) to speed things up.
https://developer.apple.com/library/mac ... index.html

I've updated the OSX interface with a couple more examples. Your previous instructions were helpful with the conversion.

Glad to hear my feedback was helpful
The stitching looks very professional.
With the seam carving example there's a problem with the enter key.

[JHPJHP]

Hi wilbert,

The examples have been fixed, as it turned out le_stitching_1.pb had a problem as well. Thank you for the feedback.

There are 202 color options available using the Function cvCvtColor.
- #CV_BGR2BGRA: converts an image from BGR to BGRA (cvCreateImage :: 4 channels)
- /includes/cv_constants.pbi (lines 657 - 859)

Converting between types:

Code: Select all

*image.IplImage = cvLoadImage("image.jpg", #CV_LOAD_IMAGE_ANYDEPTH | #CV_LOAD_IMAGE_ANYCOLOR)
*image32.IplImage = cvCreateImage(*image\width, *image\height, #IPL_DEPTH_32F, *image\nChannels)
cvConvertScale(*image, *image32, 1 / 255, 0)
...
*final.IplImage = cvCreateImage(*image\width, *image\height, #IPL_DEPTH_8U, *image\nChannels)
cvConvertScale(*image32, *final, 255, 0)

NB*: See the example cv_emboss_1.pb for a demonstration of these Functions.

[wilbert]

Hi JHPJHP,
It's impressive to see the quality of that stitching and carving.

There are 202 color options available using the Function cvCvtColor.

I didn't see any option to convert from BGR2RGBA (swapped red and blue) which would be the correct one for OSX.
Your approach in the pb_webcam.pb example is a good approach. It's just that using a little bit of asm helps a lot.
If you compare the original OpenCV2PBImage procedure you coded with the one below, you will see that it does exactly the same but the cpu load decreases.

Code: Select all

ProcedureC OpenCV2PBImage(*frame.IplImage, pbImage, nWidth, nHeight)
  Select *frame\nChannels
    Case 1
      If StartDrawing(ImageOutput(pbImage))
        widthStep = DrawingBufferPitch()
        *src = *frame\imageData
        *dst = DrawingBuffer()
        While nHeight
          !mov rax, [p.p_src]
          !mov rdx, [p.p_dst]
          !mov rcx, [p.v_nWidth]
          !push rbx
          !cv2pb_8uc1_l0:
          !movzx ebx, byte [rax]
          !imul ebx, 0x010101
          !or ebx, 0xff000000
          !mov [rdx], ebx
          !add rax, 1
          !add rdx, 4
          !sub rcx, 1
          !jnz cv2pb_8uc1_l0
          !pop rbx
          *src + *frame\widthStep
          *dst + widthStep
          nHeight - 1
        Wend          
        StopDrawing()
      EndIf
    Case 3
      If StartDrawing(ImageOutput(pbImage))
        widthStep = DrawingBufferPitch()
        *src = *frame\imageData
        *dst = DrawingBuffer()
        While nHeight
          !mov rax, [p.p_src]
          !mov rdx, [p.p_dst]
          !mov rcx, [p.v_nWidth]
          !push rbx
          !movzx ebx, word [rax + 1]
          !shl ebx, 16
          !mov bh, [rax]
          !jmp cv2pb_8uc3_l1
          !cv2pb_8uc3_l0:
          !mov ebx, [rax - 1]
          !cv2pb_8uc3_l1:
          !bswap ebx
          !or ebx, 0xff000000
          !mov [rdx], ebx
          !add rax, 3
          !add rdx, 4
          !sub rcx, 1
          !jnz cv2pb_8uc3_l0
          !pop rbx          
          *src + *frame\widthStep
          *dst + widthStep
          nHeight - 1
        Wend
        StopDrawing()
      EndIf
  EndSelect
EndProcedure

[JHPJHP]

Hi wilbert,

I didn't see any option to convert from BGR2RGBA

See line 661 in the file /includes/cv_constants.pbi.

Thank you for the ASM version of the Procedure OpenCV2PBImage.
- it has been included in all the interfaces as an additional option, but defaulted in the OSX Demo webcam example

[wilbert]

See line 661 in the file /includes/cv_constants.pbi.

You are right; I didn't see that. Thanks for mentioning
Maybe it's faster as what I came up with.

Thank you for the ASM version of the Procedure OpenCV2PBImage.
- it has been included in all the interfaces as an additional option, but defaulted in the OSX Demo webcam example

This exact same routine run on Windows will result in swapped red and blue channels (Windows uses BGRA and OSX uses RGBA).

[Keya]

Linux also uses RGBA like OSX

[wilbert]

Linux also uses RGBA like OSX

In that case it will also help when porting things to Linux

[JHPJHP]

Hi wilbert, Keya,

You'd think I'd remember the pixel format with the cross-platform work I've done, but it's just the opposite, things are getting more confusing working between operating systems
- I removed the ASM Function from the Windows interface; speed and CPU usage isn't an issue in the Window OS

Added four more converted examples:
- cv_mov_runningavg_1.pb: calculates the weighted sum of the input image and the accumulator, used to detect moving objects of a certain size
-- replace the AVI file with an M4V version
- cv_mov_scene_detection.pb: detect scene changes on a movie file using similarity measurements between frames
-- replace the AVI file with an M4V version
- cv_enhance_detail.pb: enhance the details of an image
- cv_fractal_1.pb: generates an iterative Mandelbrot Set, the most popular geometrical fractal

How are the videos playing from the movie examples? My systems configuration makes it hard for me to determine the playback speed.

[wilbert]

You'd think I'd remember the pixel format with the cross-platform work I've done, but it's just the opposite, things are getting more confusing working between operating systems

It can be confusing sometimes. It would be easier if all operating systems supported the same layout for pixels but unfortunately they don't.

The cpu load for the examples using movies is fine on my iMac.

I like the cv_mov_runningavg_1.pb example.

[wilbert]

I tried cvCvtColor to convert BGR to RGBA like this

Code: Select all

cvInitImageHeader(@image.IplImage, OutputHeight() << 32 | OutputWidth(), 8, 4, 0, 4)
cvSetData(@image, DrawingBuffer(), DrawingBufferPitch())
cvCvtColor(*frame, @image, #CV_BGR2RGBA)

It worked fine but was not as fast as the asm enhanced procedure.
However it seemed to be a bit better compared to the encode / catch method and could also be used to copy directly to a CanvasGadget .

[JHPJHP]

Hi wilbert,

That's a really good algorithm, I particularly like that it can write directly to the CanvasGadget.
- if you write any hybrid (OpenCV / PureBasic) examples, I'll add them to the demo package

Converted another four examples:
- cv_cam_histogram.pb: calculates a histogram for the Red, Green, and Blue channels of webcam frames
- cv_color_transfer.pb: performs a color transfer between two images
- cv_norm.pb: using a default image, the absolute difference is calculated against a directory of images, returning the closest match
- cv_saliency.pb: a fine-grained saliency algorithm is applied to an image

NB*: I hope this thread finds OSX users that can benefit from the work being done here.