ffmpeg Movie Grabber

[chris319]

Here is a program which uses pipes to grab frames from an mp4 file and is supposed to write those frames back to an mp4 file, if anybody wants to monkey around with it.

It reads frames from a movie OK. Thanks to PB I got that working in fairly short order. I'm having difficulty writing the frames back to a file and for the resulting movie to be playable by, say, VLC.

You will need an mp4 video file and the ffmpeg audio and video codec which can be obtained from zeranoe:

https://ffmpeg.zeranoe.com/builds/

This code also functions as the basis of a movie player (no audio, though) and it currently displays images upside-down which is not of immediate concern as long as the images it writes to the file are right-side-up.

https://www.videolan.org/vlc/index.html

The target OS is Windows 10.

This program is based on code written for C and python which I was unable to get to work:

https://batchloaf.wordpress.com/2017/02 ... t-2-video/

http://zulko.github.io/blog/2013/09/27/ ... ng-ffmpeg/

Code: Select all

;MP4 MOVIE GRABBER
#HEIGHT = 720: #WIDTH = 1280: #PIXCOUNT = #HEIGHT * #WIDTH * 3

endFlag = 0
OpenWindow(1,0,0,#Width,#Height,"")
CreateImage(1, #Width, #Height , 24)
StartDrawing(ImageOutput(1))
*buff = DrawingBuffer()
StopDrawing()

Dim pixels.a(#HEIGHT,#WIDTH,3)
*pix = @pixels(0,0,0)

pipeIn$ = "-i teapot.mp4 -f image2pipe -pix_fmt rgb24  -vcodec rawvideo -"

;pipeOut$ =  "-y  -f rawvideo  -vcodec rawvideo  -pix_fmt yuv420p  -s 1280x720  -r 25  -i -  -f mp4  -q:v 5  -an  -vcodec mpeg4  outputvideo.mp4"
;pipeOut$ =  "-y  -f rawvideo  -vcodec rawvideo  -pix_fmt yuv420p  -s 1280x720  -r 25  -i -  -f mp4  -q:v 5  -an  -c:v libx264  -bitrate 3000k  outputvideo.mp4"
pipeOut$ =  "-y  -f rawvideo  -vcodec rawvideo  -pix_fmt rgb24  -s 1280x720  -r 25  -i -  -f mp4  -q:v 5  -an  -vcodec mpeg4  -bitrate 3000k  outputvideo.mp4"


readFrame = RunProgram("ffmpeg.exe", pipeIn$, "", #PB_Program_Open | #PB_Program_Read)
writeFrame = RunProgram("ffmpeg.exe", pipeOut$, "", #PB_Program_Open|#PB_Program_Write)

Repeat
count = ReadProgramData(readFrame, *pix, #PIXCOUNT)
;If count < #PIXCOUNT: endFlag = #True: Break: EndIf
If count = 0: endFlag = #True: Break: EndIf

;For y = 0 To 719
;For x = 0 To 1279
;pixels(y,x,0) = 255
;pixels(y,x,1) = 255
;pixels(y,x,2) = 255
;Next
;Next

;For y = 0 To 719
;For x = 0 To 1279
        
            ;Invert each colour component in every pixel
;pixels(y,x,0) = 255 - pixels(y,x,0); // red
;pixels(y,x,1) = 255 - pixels(y,x,1); // green
;pixels(y,x,2) = 255 - pixels(y,x,2); // blue
;Next
;Next

CopyMemory(*pix,*buff,#PIXCOUNT)

StartDrawing(WindowOutput(1))
DrawImage(ImageID(1),0,0)
StopDrawing()

If endFlag <> #True
;WriteProgramData(writeFrame,*buff,#PIXCOUNT)
WriteProgramData(writeFrame,*pix,#PIXCOUNT)

;WriteProgramData(writeFrame,@pixels(0,0,0),#PIXCOUNT)

EndIf

;If result = 0: Debug "error": EndIf

ForEver

Repeat: Until WaitWindowEvent() = #PB_Event_CloseWindow

KillProgram(readFrame)
CloseProgram(readFrame)
KillProgram(writeFrame)
CloseProgram(writeFrame)

FreeArray(pixels())
CloseWindow(1):End

[minimy]

Hey Chris, nice example of pipestream! very good job!
Thanks for share!

[infratec]

Code: Select all

;MP4 MOVIE GRABBER
#HEIGHT = 720
#WIDTH = 1280
#PIXCOUNT = #HEIGHT * #WIDTH * 3

Define.i endFlag, readFrame, writeFrame, count
Define *buff, *pix, *pixel.Ascii, *pixend
Define pipeIn$, pipeOut$

;Dim pixels.a(#HEIGHT, #WIDTH, 3)

OpenWindow(1, 0, 0, #Width, #Height, "")
CreateImage(1, #Width, #Height , 24)
If StartDrawing(ImageOutput(1))
  *buff = DrawingBuffer()
  StopDrawing()
EndIf
ImageGadget(1, 0, 0, #WIDTH, #HEIGHT, ImageID(1))


;*pix = @pixels(0, 0, 0)
*pix = AllocateMemory(#PIXCOUNT, #PB_Memory_NoClear)
*pixend = *pix + MemorySize(*pix)


pipeIn$ = "-i teapot.mp4 -f image2pipe -pix_fmt rgb24 -vcodec rawvideo -"

;pipeOut$ = "-y  -f rawvideo  -vcodec rawvideo  -pix_fmt yuv420p  -s 1280x720  -r 25  -i -  -f mp4  -q:v 5  -an  -vcodec mpeg4  outputvideo.mp4"
;pipeOut$ = "-y  -f rawvideo  -vcodec rawvideo  -pix_fmt yuv420p  -s 1280x720  -r 25  -i -  -f mp4  -q:v 5  -an  -c:v libx264  -bitrate 3000k  outputvideo.mp4"
pipeOut$ = "-y  -f rawvideo  -vcodec rawvideo  -pix_fmt rgb24  -s 1280x720  -r 25  -i -  -f mp4  -q:v 5 -an -vcodec mpeg4 -bitrate 3000k  outputvideo.mp4"


readFrame = RunProgram("ffmpeg.exe", pipeIn$, "", #PB_Program_Open|#PB_Program_Read)
writeFrame = RunProgram("ffmpeg.exe", pipeOut$, "", #PB_Program_Open|#PB_Program_Write)

Repeat
  
  Event = WindowEvent()
  
    count = ReadProgramData(readFrame, *pix, #PIXCOUNT)
    ;Debug count
    If count = 0
      endFlag = #True
      WriteProgramData(writeFrame, #PB_Program_Eof, 0)
      Break
    EndIf
    
;     For y = 0 To 719
;       For x = 0 To 1279
;         ;Invert each colour component in every pixel
;         pixels(y, x, 0) = 255 - pixels(y, x, 0); // red
;         pixels(y, x, 1) = 255 - pixels(y, x, 1); // green
;         pixels(y, x, 2) = 255 - pixels(y, x, 2); // blue
;       Next
;     Next
    
;     *pixel = *pix
;     Repeat
;       *pixel\a = - *pixel\a
;       *pixel + 1
;     Until *pixel = *pixend
    
    CopyMemory(*pix, *buff, #PIXCOUNT)
    SetGadgetState(1, ImageID(1))
    
    If endFlag <> #True
      WriteProgramData(writeFrame,*pix, #PIXCOUNT)
    EndIf
  
Until Event = #PB_Event_CloseWindow

If ProgramRunning(readFrame)
  Debug "Close read"
  CloseProgram(readFrame)
EndIf


If ProgramRunning(writeFrame)
  Debug "Close write"
  CloseProgram(writeFrame)
EndIf

You have to close the program and not kill it first. Then it can write the correct file before it terminate

If you enable the section

Code: Select all

;     *pixel = *pix
;     Repeat
;       *pixel\a = - *pixel\a
;       *pixel + 1
;     Until *pixel = *pixend

switch the debugger off

[minimy]

Hi infratec, good job! work fine! thx for share!

[chris319]

Now I need help.

Sometimes I get normal-looking video; sometimes I get a green rectangle with no video.

Can anyone help solve this? Do the pixels need to be represented as structures?

Please excuse the messy-looking code. Thanks.

Code: Select all

;READS YUV420P SAMPLES DIRECTLY FROM FILE
;USE WITH YUV FILES
;http://avisynth.nl/index.php/Color_conversions

; Structure pixel
;   red.a
;   green.a
;   blue.a
; EndStructure
;pixl.pixel
;pixl\red = 0

#H = 720: #W = 1280

;BT.709 CONSTANTS
#Kr = 0.2126: #Kg = 0.7152: #Kb = 0.0722

;https://docs.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-loadiconw
Global hIcon = LoadIcon_(0, #IDI_ERROR)
;iIcon = LoadIcon_(0, #IDI_ASTERISK)
Global jIcon = LoadIcon_(0, #IDI_EXCLAMATION)
;kIcon = LoadIcon_(0, #IDI_QUESTION)

Global fullScreen = #False

Global scale.f = 2.7, offset.f = 710
Global Dim frame.a(#H*#W*3) ;Allocate a buffer To store one frame
Global numpixels = #H * #W

Global maxLum,maxred,maxgreen,maxblue,maxu,maxv

LoadFont(1,"Arial",14)
LoadFont(2,"Verdana",18)
LoadFont(3,"Verdana",36,#BOLD_FONTTYPE)

OpenWindow(1,0,0,1280,#H-30,"")
SetWindowColor(1,RGB(16,16,16))
;AddKeyboardShortcut(1,#PB_Shortcut_F11,#PB_Shortcut_F11) ;F11 FOR FULLSCREEN

Procedure RedrawImage()
  StartDrawing(WindowOutput(1))
DrawImage(ImageID(1),0,0,#W, 490):StopDrawing()

EndProcedure

CreateImage(1, #W, #H , 24)
CreateImage(2, #W, #H , 24)

Global filename$ = OpenFileRequester("Select a file","D:\Videos\", "Media files (*.avi, *.wmv, *.mp4, *.mp2)"+Chr(0)+"*.avi; *.wmv; *.mp4; *.mp2", 3)

pipeIn$ = "-i " + filename$ + " -f image2pipe  -s 1280x720  -pix_fmt yuv420p  -vcodec rawvideo -"
readFrame = RunProgram("ffmpeg.exe", pipeIn$, "", #PB_Program_Open | #PB_Program_Read)

For ct = 1 To 60 ;READ FIRST 60 FRAMES
 
ReadProgramData(readFrame, @frame(0), #H * #W * 3)

Next

CloseProgram(readFrame)

Procedure Process(lift.f, gamma.f, gain.f,pedestal.f)

StartDrawing(ImageOutput(1))

*pntr = @frame.a(0)

For ycoord = 0 To #H - 1
For x = 0 To #W - 1

y.a = PeekA(*pntr)
u.a = frame((yCoord / 2) * (#W / 2) + (x / 2) + numpixels)
v.a = frame((yCoord / 2) * (#W / 2) + (x / 2) + numpixels + (numpixels / 4))

yf.f = y
uf.f = u
vf.f = v

rf.f = (255/219)*yf + (255/112)*vf*(1-#Kr) - (255*16/219 + 255*128/112*(1-#Kr)) 
gf.f = (255/219)*yf - (255/112)*uf*(1-#Kb)*#Kb/#Kg - (255/112)*vf*(1-#Kr)*#Kr/#Kg - (255*16/219 - 255/112*128*(1-#Kb)*#Kb/#Kg - 255/112*128*(1-#Kr)*#Kr/#Kg)
bf.f = (255/219)*yf + (255/112)*uf*(1-#Kb) - (255*16/219 + 255*128/112*(1-#Kb))

If rf > 255: rf = 255:EndIf: If gf > 255: gf = 255:EndIf: If bf > 255: bf = 255:EndIf
If rf < 0: rf = 0:EndIf: If gf < 0: gf = 0:EndIf: If bf < 0: bf = 0:EndIf

rd.a = rf
gd.a = gf
bd.a = bf

lum = y

;Plot (x/2, ycoord/1.36, RGB(y,y,y)); DRAW BLACK & WHITE MONITOR
Plot (x/2, ycoord/1.36, RGB(Rd,Gd,Bd)); DRAW COLOR MONITOR

*pntr + 1

Next
Next


StopDrawing()

RedrawImage()

EndProcedure

Repeat: 
event = WaitWindowEvent();WINDOW 1 ACTIVE 
If Event = #PB_Event_Repaint:ReDrawImage():EndIf

;Process(lift,gamma,gain,pedestal)
Process(0,1,1,0)
ReDrawImage()

If event = #PB_Event_Menu
menuItem = EventMenu()

If menuitem = #PB_Shortcut_F11

If fullScreen <> #False
fullScreen = #False
Else
fullScreen = #True
OpenWindow(2,0,0,#W,#H,"",#PB_Window_BorderLess)
AddKeyboardShortcut(2,#PB_Shortcut_F11,#PB_Shortcut_F11) ;"ESCAPE" KEY TO QUIT PROGRAM
SetActiveWindow(2)
StartDrawing(WindowOutput(2))
DrawImage(ImageID(2),0,0)
StopDrawing()

Repeat
event = WaitWindowEvent();WINDOW 2 NOW ACTIVE

If event = #PB_Event_Menu
menuItem = EventMenu()

If menuitem = #PB_Shortcut_F11
fullScreen = #False
RemoveKeyboardShortcut(2,#PB_Shortcut_F11)
CloseWindow(2)
SetActiveWindow(1)

Break

EndIf
EndIf

ForEver

EndIf

EndIf
EndIf
Until event = #PB_Event_CloseWindow

FreeArray(frame())
CloseWindow(1):End

[infratec]

What does 'sometimes' mean?

Same file but at try number x?
Or with a different file it happens?
(Then: With which file?)

And your code is messy

[chris319]

With different files of the same type, i.e. all .mp4 with H.264 video.

They seem to play back OK if I use a "regular" video player such as VLC or MPC, or if I use ffmpeg to transcode them to H.264 .mp4, so there is video there, which is why I think the problem is with my code and not the files themselves.

I have re-posted the code. It is now a little less messy.

OS is Windows 10.

[infratec]

With this file it works:

http://www.html5videoplayer.net/videos/toystory.mp4

[chris319]

Well that's the thing. It works fine with some files and not others; all are H.264 .mp4 files.

[chris319]

Here is some sample code written in C:

https://batchloaf.wordpress.com/2017/02 ... t-2-video/

Maybe there is a bug in ffmpeg?

[chris319]

A quick pass through ffmpeg with no processing fixes the videos that won't render, i.e.

Code: Select all

ffmpeg -y  -i  "myvideo.mp4"  "outvideo.mp4

This is a hack but it makes the videos appear.

[chris319]

Problem solved:

Had to remove all spaces from directory names. The program was not finding the files and was trying to load blanks.

[chris319]

Instead, I wrapped the file name in quotes and that worked very nicely.