PureBasic Forum
https://www.purebasic.fr/english/

Another Webcam Demo Program
https://www.purebasic.fr/english/viewtopic.php?f=12&t=69059		 Page 1 of 5
Author:  chris319 [ Mon Aug 28, 2017 9:05 pm ]
Post subject:  Another Webcam Demo Program
Here is a webcam demo program I have written. It uses escapi and processes each pixel individually. The processing code runs fast enough to keep up with the frame rate provided the debugger is off. I get a full 30 fps if the screen resolution is 640 x 360. The frame rate goes down if the resolution is 1280 x 720. The camera itself seems to be the frame-rate bottleneck, depending on the resolution.

Press CTL-Q to exit the program.

The program uses look-up tables rather than performing floating-point calculations on the fly. It also draws to an image gadget.

For testing purposes you can skip the pixel-by-pixel processing code by uncommenting line 86.

http://sol.gfxile.net/escapi/

Code:
;WORKS WITH ESCAPI
;UPDATED 8/28/2017
;PRESS CTL-Q TO QUIT
;INCORPORATES PROC AMP

ExamineDesktops()

gamma.f = 1.34 ;WEBCAM OUTPUT IS ALREADY GAMMA CORRECTED
#PEDESTAL = 16
gain.f = (235-16)/235
gain * 0.97

factor.f = 255/Pow(255,gamma)

;SET WEBCAM TO DEFAULTS AND SHARPNESS TO 255

;MAKE LUTS
Dim gammaTable.f(256)
For cnt = 0 To 255
cnf.f = cnt
gammaTable.f(cnt)=cnt;(Pow(cnf,gamma) * factor * gain) + #PEDESTAL
Next

Dim x3table.l(1921)
For cnt = 0 To 1920
x3table(cnt) = cnt * 3
Next

LoadFont(1,"Arial",24)

IncludeFile "escapi.pbi"
Global WIDTH = 640;DesktopWidth(0)
Global HEIGHT = 360;DesktopHeight(0)
Global WIDTHM1 = WIDTH - 1
Global HEIGHTM1 = HEIGHT - 1
Global pixCount = (WIDTH * HEIGHT) - 2

Global Dim pixcolor.l(WIDTH, HEIGHT): Global Dim unsmoothedY.d(WIDTH, HEIGHT)
Global Dim Cr.d(WIDTH, HEIGHT): Global Dim Y.d(WIDTH, HEIGHT): Global Dim Cb.d(WIDTH, HEIGHT)
Global imHeight, imWidth, xCoord, yCoord,Rd,Gd,Bd

#DEVICE = 0   
If setupESCAPI() = #Null
      MessageRequester("Error", "Unable to initialize ESCAPI.")
    End
EndIf

    bufSize = WIDTH * HEIGHT * 4
    scp.SimpleCapParams
    scp\mWidth = WIDTH
    scp\mHeight = HEIGHT
    scp\mTargetBuf = AllocateMemory(bufSize)
    *buf = scp\mTargetBuf

    If initCapture(#DEVICE, @scp)
     
image = CreateImage(1, WIDTH, HEIGHT, 24)
OpenWindow(1, 0, 0, WIDTH, HEIGHT,"",#PB_Window_BorderLess)
AddKeyboardShortcut(1, #PB_Shortcut_Control|#PB_Shortcut_Q, 113);CTL Q TO QUIT
ImageGadget(0, 0, 0, WIDTH, HEIGHT, ImageID(1))
Quit = #False

StartDrawing(ImageOutput(1))
*writeBuffer = DrawingBuffer()
pitch = DrawingBufferPitch()
StopDrawing()

Repeat
If WindowEvent() = #PB_Event_Menu ;KEYBOARD INPUT
If EventMenu() = 113
  Quit = #True
EndIf
EndIf       
     
doCapture(#DEVICE)

Repeat: Delay(1):Until isCaptureDone(#DEVICE) <> #False

;If isCaptureDone(#DEVICE) <> #False
;If isCaptureDone(#DEVICE) = #False

;PIXEL-BY-PIXEL READING AND WRITING
hm1 = *writebuffer + (HEIGHTM1 * pitch)
*bufoff = *buf

;Goto skip
For y = 0 To HEIGHTM1
For x = 0 To WIDTHM1
x3 = hm1 + x3table(x)

p1.l = PeekL(*bufoff)

PokeA(x3,gammaTable(p1 & 255))
PokeA(x3+1,gammaTable(p1 >> 8 & 255))
PokeA(x3+2,gammaTable(p1 >> 16))

*bufoff + 4
Next
hm1 - pitch
Next

skip:
SetGadgetState(0, ImageID(1))

StartDrawing(WindowOutput(1))
DrawingFont(FontID(1))

now.f = ElapsedMilliseconds()
fps.f = now.f-then.f

If fps > 0:fps$ = StrF((1/fps)*1000,2) ;:EndIf
If (1/fps)*1000 < 10: fps$ = "0" + fps$: EndIf
EndIf

DrawText(100, 200, fps$+" fps",#White)
then.f = ElapsedMilliseconds()
StopDrawing()

Until Quit = #True

deinitCapture(#DEVICE)
FreeImage(1)
FreeMemory(scp\mTargetBuf)
CloseWindow(1)

    Else
      Debug "Init capture failed."
    EndIf

    End
Author:  walbus [ Tue Aug 29, 2017 8:22 am ]
Post subject:  Re: Another Webcam Demo Program
Hi,
i have maked a quick look to your code (Before the first coffee)
Primary, i think
The frames text output is immediately, this is not good
This make also immediately a start and stop drawing in the main loop

I think, its recommended to change this, at sample so :
Set a windowed counter all ~500ms, then make this output
And looking before you make this output, is the frame rate changed
Is the frame rate not changed make this text output not !
The output directly on the window is also not a good idea, i think

The PokeA method is mostly fast enough for a image sized about 1024*768, but with more, its to slow
Older machines, i think, can not output this 30 frames
So, i think, further, a other buffer handling is recommended

The Delay(1) construction inside the main loop is a handbrake and looking a little strange
(I have not looking the pbi to time)
Author:  chris319 [ Tue Aug 29, 2017 10:21 am ]
Post subject:  Re: Another Webcam Demo Program
This is a demo program. In actual use you wouldn't display the fps text over video.

Quote:
Set a windowed counter all ~500ms, then make this output


You can't do that with video. The program must synchronize with an external device (the camera).

Quote:
Older machines, i think, can not output this 30 frames


Then they'll either have to get a better computer or live with a lower frame rate.
Author:  walbus [ Tue Aug 29, 2017 12:18 pm ]
Post subject:  Re: Another Webcam Demo Program
I self think, your primary main loop construction is the problem, never this can works good

And the world is full older machines, we should try making software for all
Not all users have money for new machines $
Also i see not a problem for a frames/s output
Author:  IdeasVacuum [ Tue Aug 29, 2017 1:49 pm ]
Post subject:  Re: Another Webcam Demo Program
Thanks for sharing your code chris319 8)

A couple of questions:
Where is file escapi.pbi?
What version of escapi is required?
Author:  chris319 [ Tue Aug 29, 2017 7:35 pm ]
Post subject:  Re: Another Webcam Demo Program
Here are some links for escapi. I'm using the latest version, 3.0.

http://sol.gfxile.net/escapi/

https://github.com/jarikomppa/escapi

https://github.com/jarikomppa/escapi/tr ... /purebasic

Kudos to Jari for supporting pb.

There are several posts regarding pb and escapi on this forum.

escapi.pbi can be found here, but it needs to be renamed with the .pbi extension. https://github.com/jarikomppa/escapi/bl ... /escapi.pb

Here are the contents of escapi.pbi:

Code:
;/* Extremely Simple Capture API */

Structure SimpleCapParams
  *mTargetBuf ; Must be at least mWidth * mHeight * SizeOf(int) of size!
  mWidth.l
  mHeight.l
EndStructure

;/* Return the number of capture devices found */
PrototypeC countCaptureDevicesProc()

; /* initCapture tries To open the video capture device.
;  * Returns 0 on failure, 1 on success.
;  * Note: Capture parameter values must Not change While capture device
;  *       is in use (i.e. between initCapture And deinitCapture).
;  *       Do *Not* free the target buffer, or change its pointer!
;  */
PrototypeC initCaptureProc(deviceno, *aParams.SimpleCapParams)

;/* deinitCapture closes the video capture device. */
PrototypeC deinitCaptureProc(deviceno)

;/* doCapture requests video frame To be captured. */
PrototypeC doCaptureProc(deviceno)

;/* isCaptureDone returns 1 when the requested frame has been captured.*/
PrototypeC isCaptureDoneProc(deviceno)

;/* Get the user-friendly name of a capture device. */
PrototypeC getCaptureDeviceNameProc(deviceno, *namebuffer, bufferlength)

;/* Returns the ESCAPI DLL version. 0x200 For 2.0 */
PrototypeC ESCAPIDLLVersionProc()

; marked as "internal" in the example
PrototypeC initCOMProc()

Global countCaptureDevices.countCaptureDevicesProc
Global initCapture.initCaptureProc
Global deinitCapture.deinitCaptureProc
Global doCapture.doCaptureProc
Global isCaptureDone.isCaptureDoneProc
Global getCaptureDeviceName.getCaptureDeviceNameProc
Global ESCAPIDLLVersion.ESCAPIDLLVersionProc

Procedure setupESCAPI()
 
  ; load library
  capdll = OpenLibrary(#PB_Any, "escapi.dll")
  If capdll = 0
    ProcedureReturn 0
  EndIf
 
  ;/* Fetch function entry points */
  countCaptureDevices = GetFunction(capdll, "countCaptureDevices")
  initCapture = GetFunction(capdll, "initCapture")
  deinitCapture = GetFunction(capdll, "deinitCapture")
  doCapture = GetFunction(capdll, "doCapture")
  isCaptureDone = GetFunction(capdll, "isCaptureDone")
  initCOM.initCOMProc = GetFunction(capdll, "initCOM")
  getCaptureDeviceName = GetFunction(capdll, "getCaptureDeviceName")
  ESCAPIDLLVersion = GetFunction(capdll, "ESCAPIDLLVersion")
 
  If countCaptureDevices = 0 Or initCapture = 0 Or deinitCapture = 0 Or doCapture = 0 Or isCaptureDone = 0 Or initCOM = 0 Or getCaptureDeviceName = 0 Or ESCAPIDLLVersion = 0
    ProcedureReturn 0
  EndIf
 
  ;/* Verify DLL version */
  If ESCAPIDLLVersion() < $200
    ProcedureReturn 0
  EndIf
 
  ;/* Initialize COM.. */
  initCOM();
 
  ; returns number of devices found
  ProcedureReturn countCaptureDevices()
EndProcedure
Author:  chris319 [ Tue Aug 29, 2017 8:05 pm ]
Post subject:  Re: Another Webcam Demo Program
walbus wrote:
I self think, your primary main loop construction is the problem, never this can works good

And the world is full older machines, we should try making software for all
Not all users have money for new machines $
Also i see not a problem for a frames/s output


See the other thread in Coding Questions in which you have participated. The bottleneck is the webcam itself. Even the OpenCV code slows down when it has to draw a 1280 x 720 bitmap. It slows down in my program even if you skip all of my processing code.

If a 30 or even 60 fps frame rate is desired, the hardware has to keep up. That includes the webcam and the host computer. Again, the frame rate is dictated by an external device.
Author:  chris319 [ Tue Aug 29, 2017 8:25 pm ]
Post subject:  Re: Another Webcam Demo Program
Here is a version of the program with a smaller 640 x 360 image and incorporating Walbus's suggestion for not calling StartDrawing() repeatedly. In addition, in creating the lookup table, any pixel value greater than 254 is forced to 254, and any pixel value less than 1 is forced to 1. In professional video, the values of 0 and 255 are reserved for sync. This code can be eliminated if desired.

Code:
;WORKS WITH ESCAPI
;UPDATED 8/28/2017
;PRESS CTL-Q TO QUIT
;INCORPORATES PROC AMP

ExamineDesktops()

gamma.f = 1.34 ;WEBCAM OUTPUT IS ALREADY GAMMA CORRECTED
#PEDESTAL = 16
gain.f = (235-16)/235
gain * 0.97

factor.f = 255/Pow(255,gamma)

;SET WEBCAM TO DEFAULTS AND SHARPNESS TO 255

;MAKE LUTS
Dim gammaTable.f(256)
For cnt = 0 To 255
cnf.f = cnt
gammaTable.f(cnt)=cnt;(Pow(cnf,gamma) * factor * gain) + #PEDESTAL
If gammaTable(cnt) > 254: gammaTable(cnt) = 254
ElseIf gammaTable(cnt) < 1: gammaTable(cnt) = 1
EndIf
Next

Dim x3table.l(1921)
For cnt = 0 To 1920
x3table(cnt) = cnt * 3
Next

LoadFont(1,"Arial",24)

IncludeFile "escapi.pbi"
Global WIDTH = 640;DesktopWidth(0)
Global HEIGHT = 360;DesktopHeight(0)
Global WIDTHM1 = WIDTH - 1
Global HEIGHTM1 = HEIGHT - 1
Global pixCount = (WIDTH * HEIGHT) - 2

Global Dim pixcolor.l(WIDTH, HEIGHT): Global Dim unsmoothedY.d(WIDTH, HEIGHT)
Global Dim Cr.d(WIDTH, HEIGHT): Global Dim Y.d(WIDTH, HEIGHT): Global Dim Cb.d(WIDTH, HEIGHT)

Global imHeight, imWidth, xCoord, yCoord,Rd,Gd,Bd

#DEVICE = 0   
If setupESCAPI() = #Null
      MessageRequester("Error", "Unable to initialize ESCAPI.")
    End
EndIf

    bufSize = WIDTH * HEIGHT * 4
    scp.SimpleCapParams
    scp\mWidth = WIDTH
    scp\mHeight = HEIGHT
    scp\mTargetBuf = AllocateMemory(bufSize)
    *buf = scp\mTargetBuf

    If initCapture(#DEVICE, @scp)
     
image = CreateImage(1, WIDTH, HEIGHT, 24)
OpenWindow(1, 0, 0, WIDTH, HEIGHT,"",#PB_Window_BorderLess)

AddKeyboardShortcut(1, #PB_Shortcut_Control|#PB_Shortcut_Q, 113);CTL Q TO QUIT
ImageGadget(0, 0, 0, WIDTH, HEIGHT, ImageID(1))
Quit = #False

StartDrawing(ImageOutput(1))
*writeBuffer = DrawingBuffer()
pitch = DrawingBufferPitch()
StopDrawing()

StartDrawing(WindowOutput(1))
;StartDrawing(ImageOutput(1))
DrawingFont(FontID(1))

Repeat
If WindowEvent() = #PB_Event_Menu ;KEYBOARD INPUT
If EventMenu() = 113
  Quit = #True
EndIf
EndIf       
     
doCapture(#DEVICE)

Repeat: Delay(1):Until isCaptureDone(#DEVICE) <> #False

;If isCaptureDone(#DEVICE) <> #False
;If isCaptureDone(#DEVICE) = #False

;PIXEL-BY-PIXEL READING AND WRITING
hm1 = *writebuffer + (HEIGHTM1 * pitch)
*bufoff = *buf

;Goto skip
For y = 0 To HEIGHTM1
For x = 0 To WIDTHM1
x3 = hm1 + x3table(x)

p1.l = PeekL(*bufoff)

PokeA(x3,gammaTable(p1 & 255))
PokeA(x3+1,gammaTable(p1 >> 8 & 255))
PokeA(x3+2,gammaTable(p1 >> 16))

*bufoff + 4
Next
hm1 - pitch
Next

skip:
SetGadgetState(0, ImageID(1))

;StartDrawing(WindowOutput(1))
;DrawingFont(FontID(1))

now.f = ElapsedMilliseconds()
fps.f = now.f-then.f

If fps > 0:fps$ = StrF((1/fps)*1000,2) ;:EndIf
If (1/fps)*1000 < 10: fps$ = "0" + fps$: EndIf
EndIf

DrawText(100, 200, fps$+" fps",#White)
then.f = ElapsedMilliseconds()
;StopDrawing()

Until Quit = #True
StopDrawing()
deinitCapture(#DEVICE)
FreeImage(1)
FreeMemory(scp\mTargetBuf)
CloseWindow(1)

    Else
      Debug "Init capture failed."
    EndIf

    End
Author:  djes [ Wed Aug 30, 2017 11:40 am ]
Post subject:  Re: Another Webcam Demo Program
I've tried a simple copymemory(), as I don't understand why every pixel should be converted, if the image given by ESCAPI is already in 32 bits. As the buffer is upside down, I've used vectordrawing lib to reverse the whole thing. I've also used your fps() counter.
Code:
XIncludeFile "escapi.pbi"

device = 0

count = setupESCAPI()
Debug "init: " + Str(count)

If count = 0
  End
EndIf

*name = AllocateMemory(1000)
getCaptureDeviceName(device, *name, 1000)
name$ = PeekS(*name, -1, #PB_Ascii)
Debug "name: " + name$
FreeMemory(*name)

scp.SimpleCapParams
scp\mWidth = 640
scp\mHeight = 360
scp\mTargetBuf = AllocateMemory (scp\mWidth * scp\mHeight * 4)

If initCapture(device, @scp)
  Debug "cap init successful"
 
  gadgetimage = CreateImage(#PB_Any, scp\mWidth, scp\mHeight)
  webcamimage = CreateImage(#PB_Any, scp\mWidth, scp\mHeight, 32)
  CopyImage(gadgetimage, renderedimage)
 
  OpenWindow(0, 0, 0, scp\mWidth, scp\mHeight, name$, #PB_Window_ScreenCentered|#PB_Window_SystemMenu)
  ImageGadget(0, 0, 0, scp\mWidth, scp\mHeight, ImageID(gadgetimage))
  LoadFont(0, "Impact", 20, #PB_Font_Bold)
 
  Quit = 0
  Repeat
   
    doCapture(device)
    While isCaptureDone(device) = 0
      If WaitWindowEvent(1) = #PB_Event_CloseWindow
        Quit = 1
        Break
      EndIf       
    Wend
   
    If StartDrawing(ImageOutput(webcamimage))
      Buffer      = DrawingBuffer()             ; Get the start address of the screen buffer
      Pitch       = DrawingBufferPitch()        ; Get the length (in byte) took by one horizontal line
      PixelFormat = DrawingBufferPixelFormat()  ; Get the pixel format.
      CopyMemory(scp\mTargetBuf, Buffer, scp\mWidth * scp\mHeight * 4)
      StopDrawing()
    EndIf
   
    If StartDrawing(ImageOutput(renderedimage))
      DrawImage(ImageID(webcamimage), 0, 0)
      StopDrawing()
    EndIf
   
    now = ElapsedMilliseconds()
    fps.f = now - then
    If fps > 0 : fps$ = StrF((1/fps)*1000, 2) ;:EndIf
      If (1 / fps) * 1000 < 10: fps$ = "0" + fps$: EndIf
    EndIf 
    then = ElapsedMilliseconds()
     
    If StartVectorDrawing(ImageVectorOutput(gadgetimage, #PB_Unit_Pixel))
      VectorSourceColor(RGBA(0, 0, 0, 255))
      FillVectorOutput()
      RotateCoordinates(scp\mWidth/2, scp\mHeight/2, now / 10)
      MovePathCursor(0, 0)     
      DrawVectorImage(ImageID(renderedimage), 128)
      ResetCoordinates()
      MovePathCursor(10, 10)     
      VectorFont(FontID(0), 25)
      VectorSourceColor(RGBA(255, 255, 0, 255))
      DrawVectorText(fps$ + " fps")
      StopVectorDrawing()
    EndIf
    SetGadgetState(0, ImageID(gadgetimage))
   
  Until Quit
 
  deinitCapture(device)
Else
  Debug "init capture failed!"
EndIf

End
Author:  walbus [ Wed Aug 30, 2017 5:21 pm ]
Post subject:  Re: Another Webcam Demo Program
@djes
Unfortunately, your rotating is also pixel based, primary your method is ever a slower solution as a simple copy method from a to b
Author:  djes [ Wed Aug 30, 2017 9:56 pm ]
Post subject:  Re: Another Webcam Demo Program
walbus wrote:
@djes
Unfortunately, your rotating is also pixel based, primary your method is ever a slower solution as a simple copy method from a to b

I just showed that a pixel based loop is not necessary, as the frame buffer is directly available in a compatible format (see the C code). A direct copy is faster, as long as you don't have to modify the picture. The rotation was only for fun.
There's also in escapi an example using opengl, that should be the faster method to achieve a large frame update at full speed.
Author:  AAT [ Thu Aug 31, 2017 3:25 am ]
Post subject:  Re: Another Webcam Demo Program
chris319 wrote:
... The bottleneck is the webcam itself. Even the OpenCV code slows down when it has to draw a 1280 x 720 bitmap. It slows down in my program even if you skip all of my processing code.

Perhaps, this information will be useful to you.
https://translate.google.com/translate? ... edit-text=
Author:  chris319 [ Thu Aug 31, 2017 6:33 am ]
Post subject:  Re: Another Webcam Demo Program
Hi AAT -

Thank you for posting that. It is very interesting.

Quote:
Working with the camera in Windows is implemented in cap_dshow.cpp. There in
VideoInput :: videoInput ()
The camera modes in the mediaSubtypes array are defined

Here are the standard modes

MediaSubtypes [0] = MEDIASUBTYPE_RGB24;
MediaSubtypes [1] = MEDIASUBTYPE_RGB32;
MediaSubtypes [2] = MEDIASUBTYPE_RGB555;
MediaSubtypes [3] = MEDIASUBTYPE_RGB565;
MediaSubtypes [4] = MEDIASUBTYPE_YUY2;
MediaSubtypes [5] = MEDIASUBTYPE_YVYU;
MediaSubtypes [6] = MEDIASUBTYPE_YUYV;
MediaSubtypes [7] = MEDIASUBTYPE_IYUV;
MediaSubtypes [8] = MEDIASUBTYPE_UYVY;
MediaSubtypes [9] = MEDIASUBTYPE_YV12;
MediaSubtypes [10] = MEDIASUBTYPE_YVU9;
MediaSubtypes [11] = MEDIASUBTYPE_Y411;
MediaSubtypes [12] = MEDIASUBTYPE_Y41P;
MediaSubtypes [13] = MEDIASUBTYPE_Y211;
MediaSubtypes [14] = MEDIASUBTYPE_AYUV;
MediaSubtypes [15] = MEDIASUBTYPE_MJPG; // MGB


Quote:
all you had to do was raise in the array MEDIASUBTYPE_MJPG over MEDIASUBTYPE_YUY2.
After that, after compiling the library, the normal result was 30 FPS for all cameras and now they worked


What do you make of that, AAT? Recompile cap_dshow.cpp?

So MediaSubtypes [4] should be MJPG and move the others down one slot?

I have passed this information along to the developer of escapi.
Author:  AAT [ Thu Aug 31, 2017 1:41 pm ]
Post subject:  Re: Another Webcam Demo Program
I did not do anything, it's just a link. :)

I think you need to put first the value of MediaSubtypes, which gives the maximum value of FPS for you. Then, recompile the entire package of OpenCV. In my opinion, this will be easier than recompiling only one DLL from the package.

Quote:
I have passed this information along to the developer of escapi.

If it becomes possible to specify the video format (mediasubtype) when the function is called, this will be the best solution.
Author:  chris319 [ Thu Aug 31, 2017 6:55 pm ]
Post subject:  Re: Another Webcam Demo Program
I've downloaded the OpenCV source code and have Visual Studio, but am looking for directions on how to compile it.
Page 1 of 5 All times are UTC + 1 hour
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/