Reading Pixel Values In a Movie

[chris319]

Here's what I want to do, and I'm stuck for a way to do it.

I want to play a movie on the screen and be able to read the pixel values within the movie.

Of course the program would need to know the height and width of the picture.

Any ideas are greatly appreciated.

Thank you.

[Mijikai]

I used this for a project (Windows OS):

Code: Select all

Window:
GetClientRect_()
MapWindowPoints_()

Capture:
BitBlt_()
GetBitmapBits_()

[Saboteur]

I asked for this feature in "Feature Request" forum previously: http://www.purebasic.fr/english/viewtop ... =3&t=64359
I'll like to have a cross platform function to read frames, now is not possible.

[chris319]

Thank you, Mijikai.

Now how would you flesh that out into functional PureBasic code?

Code: Select all

Window:
GetClientRect_()
MapWindowPoints_()

Capture:
BitBlt_()
GetBitmapBits_()

[Mijikai]

There are different ways to get a specific WindowRegion...

Heres one way (using the apis mentioned before):

Code: Select all

;by Mijikai
Procedure.i GetRect(Target.i,*Out.RECT);Target = WindowHandle
  Protected TargetRect.RECT
  Protected MappedRect.RECT
  If *Out
    If GetClientRect_(Target,@TargetRect)
      If MapWindowPoints_(Target,#Null,@MappedRect,2)
        TargetRect\right = TargetRect\right - TargetRect\left
        TargetRect\bottom = TargetRect\bottom - TargetRect\top
        *Out\left = MappedRect\left
        *Out\right = TargetRect\right
        *Out\top = MappedRect\top - 1
        *Out\bottom = TargetRect\bottom
        ProcedureReturn #True
      EndIf
    EndIf
  EndIf
EndProcedure

For capture:

Code: Select all

Get the DC (DisplayContext) of the targeted window:
-> GetDC_()

Create a compitable DC:
-> CreateCompatibleDC_()

Create a Bitmap:
-> CreateCompatibleBitmap_()

Calculate required BufferSize:
-> GetObject_()
-> AllocateMemory()

Capture:
-> BitBlt_()

Copy all pixels into the allocated buffer:
-> GetBitmapBits_()

[chris319]

This code is from another thread, here: http://www.purebasic.fr/english/viewtop ... 78#p425551 and here: http://www.purebasic.fr/english/viewtop ... 13&t=56678

Code: Select all

#CAPTUREBLT = $40000000 ; Necessary for BitBlt to get layered windows if present on screen



Procedure zGrabScreen(Array Screen.i(2))
    
    ExamineDesktops()
    W.i=DesktopWidth(0); Desktop resolution width
    H.i=DesktopHeight(0); Desktop resolution height
    
    Dim Screen.i(W,H)
    
    dc = CreateDC_("DISPLAY",0,0,0)
    ndc = CreateCompatibleDC_(0)
    
    bmi.BITMAPINFO
    bmi\bmiHeader\biSize     = SizeOf(BITMAPINFOHEADER)
    bmi\bmiHeader\biWidth    = W
    bmi\bmiHeader\biHeight   = -H ; Negative, otherwise colors in the buffer will be backwards, "bottom-up" which we don't want
    bmi\bmiHeader\biPlanes   = 1
    bmi\bmiHeader\biBitCount = 32
    bmi\bmiHeader\biCompression = #BI_RGB
    
    hImage=CreateDIBSection_(ndc,bmi,#DIB_RGB_COLORS,@*pvBits,0,0)
    If hImage
        GetObject_(himage,SizeOf(BITMAP),@bmp.BITMAP)
        DeleteObject_(SelectObject_(ndc,hImage))
        BitBlt_(ndc,0,0,W,H,dc,0,0,#SRCCOPY|#CAPTUREBLT)
        DeleteDC_(dc)
        DeleteDC_(ndc)
        *buffer=PeekI(@*pvBits)
        For C=0 To (W*H-1)*SizeOf(RGBQUAD) Step SizeOf(RGBQUAD) ; Look at the first 30 colors in scanline 0
            Screen(Int(Mod(C/SizeOf(RGBQUAD),W)),Int(C/SizeOf(RGBQUAD)/W))=PeekL(*buffer+C)&$FFFFFF; ignoring the alpha byte here; GetPixel would return this value
            ;           If Mod(C/SizeOf(RGBQUAD),W)=0
            ;               Debug Int(C/SizeOf(RGBQUAD)/W)
            ;           EndIf
            ;Debug PeekL(*buffer+C)&$FFFFFF   ; ignoring the alpha byte here; GetPixel would return this value
        Next C
    EndIf
EndProcedure

Dim Screen.i(0,0)
zGrabScreen(Screen())

For i=0 To ArraySize(Screen(),2)-1
    For j=0 To ArraySize(Screen(),1)-1
        Debug "@("+Str(j)+","+Str(i)+")="+Str(Screen(j,i))
    Next
Next

[Mijikai]

If u only need to capture a frame now and then this code might be good enough.

[chris319]

If u only need to capture a frame now and then this code might be good enough.

I need to capture every frame in real time

[Mijikai]

If u only need to capture a frame now and then this code might be good enough.

I need to capture every frame in real time

You can improve the code by only calling the function/s (each cycle) that catually do the capture.
Everything else should be only called once at the start.

This should be fast enough (otherwise DirectX might be the way).

[chris319]

I'm able to get all this code to work OK, but I'm having trouble reading the pixels inside the movie player box while the movie is playing or is paused.

???

[Mijikai]

I'm able to get all this code to work OK, but I'm having trouble reading the pixels inside the movie player box while the movie is playing or is paused.

???

What kind of trouble?

[chris319]

I've got the blitter working perfectly on drawn images. The trouble begins with the movies. It's intermittent. Sometimes I get the visual content in the movie, sometimes all I get is black.

I'm trying to prepare a demo program for you to see and even that's going poorly.

Code: Select all

   ;*************** TURN OFF THE DEBUGGER! ****************
DisableDebugger
#KR = 0.2126:#KG = 0.7152:#KB = 0.0722 ;REC.709
#CAPTUREBLT = $40000000 ; Necessary for BitBlt to get layered windows if present on screen

InitMovie()
    Global width = 640 ;DesktopWidth(0)
    Global height = 360 ;DesktopHeight(0)

    dc = CreateDC_("DISPLAY",0,0,0)
    ndc = CreateCompatibleDC_(0)

    bmi.BITMAPINFO
    bmi\bmiHeader\biSize     = SizeOf(BITMAPINFOHEADER)
    bmi\bmiHeader\biWidth    = width
    bmi\bmiHeader\biHeight   = -height ; Otherwise colors in the buffer will be backwards, "bottom-up" which we don't want
    bmi\bmiHeader\biPlanes   = 1
    bmi\bmiHeader\biBitCount = 32
    bmi\bmiHeader\biCompression = #BI_RGB

hImage = CreateDIBSection_(ndc, bmi, #DIB_RGB_COLORS, @*buffer1, 0, 0)
    If hImage
      DeleteObject_(SelectObject_(ndc, hImage))
      BitBlt_(ndc, 0,0,width,height,dc,0,0,#SRCCOPY|#CAPTUREBLT)
      DeleteDC_(dc)
      DeleteDC_(ndc)
    EndIf

Procedure GratLine(digi)
digi * 2
LineXY(676,(512 - digi) + 5, 1279,(512 - digi) + 5,$0000ff)
;Plot(x+640,(512 - lum*2), $00cc00)

EndProcedure

Procedure Graticule()
StartDrawing(WindowOutput(1))

;Box(640,0,40,800,1)
;Box(640,300,40,800,1)

GratLine(254)
GratLine(235)
GratLine(16)
GratLine(1)

DrawingMode(#PB_2DDrawing_Transparent)
DrawText(642, 392, "254", $ffffff)
DrawText(642, 430, "235", $ffffff)
DrawText(642, 774, " 63", $ffffff)
DrawText(642, 867, " 16", $ffffff)
DrawText(642, 569, "166", $ffffff)
DrawText(642, 898, "  1", $ffffff)

DrawText(600, 693, "40", $ffffff)
DrawText(2, 365, "DIG.", $ffffff)
DrawText(600,365, "IRE", $ffffff)
DrawText(600, 569, "71", $ffffff)
DrawText(600, 430, "100", $ffffff)
DrawText(600, 392, "108", $ffffff)
DrawText(600, 865, "7.5", $ffffff)
DrawText(600, 774, "27", $ffffff)
DrawText(600, 898, "0", $ffffff)
StopDrawing()
EndProcedure


LoadMovie(1,"handsfree big load.mp4")
MovieAudio(1,0,0)
ResizeMovie(1,0,0,640,360)

hWnd = OpenWindow(1,0,0,1280,720,"",#PB_Window_SystemMenu)
AddKeyboardShortcut(1, #PB_Shortcut_Control|#PB_Shortcut_Q, 113);CTL Q TO QUIT

 StartDrawing(WindowOutput(1))
 Box(0,0,200,200,#Blue)
 StopDrawing()

CreateImage(1,width,height,24)

PlayMovie(1,WindowID(1)) ;: Delay(duration * 1000)
Delay(500):PauseMovie(1)
Repeat: Until MovieStatus(1) <= 0

hdcDest = StartDrawing(ImageOutput(1))
hDCsrc = GetDC_(WindowID(1))
BitBlt_(hdcDest, 0, 0, 640, 360,hDCsrc, 0, 00, #SRCCOPY)
StopDrawing()

StartDrawing(WindowOutput(1))
DrawImage(ImageID(1),0,360)
Box(640,0,640,512+10,#Black)
StopDrawing()

For y=0 To height-1
  For x=0 To width-1

; StartDrawing(ImageOutput(1))
; pixel = Point(x,y)
; StopDrawing()
; 
; lum.f = (Red(pixel)*#KR) + (Green(pixel)*#KG) + (Blue(pixel)*#KB)
; 
; StartDrawing(WindowOutput(1))
; Plot(x+640,(512 - lum*2+5), $00cc00)
; Plot(x+640,(511 - lum*2+5), $00cc00)
; StopDrawing()

       Next
     Next

Graticule()

Repeat:Until WaitWindowEvent() = #PB_Event_CloseWindow
FreeMovie(1)
FreeImage(1)
ReleaseDC_(hWnd,hDCsrc)
CloseWindow(1)

[Mijikai]

You should use a screen otherwise it will appear to be slow as hell.

First do just this:

Code: Select all

CreateDIBSection_()
SelectObject_()

The only code needed in the capture cycle is:

Code: Select all

BitBlt_()

Access every Pixel through the Pointer provided by CreateDIBSection_():

Code: Select all

*Pixel.Long = Bits + ((Y * CaptureAreaWidth + X) << 2)
;Swap RED & BLUE!

Sorry for not fixing your code but its a bit too confusing for me.
If u still have troubles let me know ill dig up some code then.

[chris319]

I know it's going to be slow the way it's written. Right now I just want it to work, fast or slow.

I'm having a devil of a time reading pixels out of the movie.

[chris319]

Here is what I got from somebody else's demo program. It uses CreateDIBSection()

Code: Select all

hImage = CreateDIBSection_(ndc, bmi, #DIB_RGB_COLORS, @*buffer1, 0, 0)
    If hImage
      DeleteObject_(SelectObject_(ndc, hImage))
      BitBlt_(ndc, 0,0,width,height,dc,0,0,#SRCCOPY|#CAPTUREBLT)
      DeleteDC_(dc)
      DeleteDC_(ndc)
    EndIf

Now what do I do with himage? Can it be made into a conventional PB image?