[FFmpeg-user] Detecting frames on Raw video stream

[Charl Wentzel]

On 14/07/2016 15:29, Carl Eugen Hoyos wrote:
> Charl Wentzel <charl.wentzel <at> vodamail.co.za> writes:
>> $ ffmpeg -f image2pipe -vcodec rawvideo -pixel_format gray
>> -video_size 2592x1944 -framerate 10 -i tcp://192.168.1.40:5556
> You are not using the -frame_size option, the command cannot
> work without it.
Thanks, its working again, but I'm still having the same issue with 
image not being aligned correctly inside the window.

$ ffmpeg -f image2pipe -vcodec rawvideo -pixel_format gray -frame_size 
5038848 -video_size 2592x1944 -framerate 10 -i tcp://192.168.1.40:5556 
-vcodec libx264 -x264opts sliced-threads -pix_fmt yuv420p -preset 
ultrafast -tune zerolatency -vsync cfr -g 10 -f mp4 test.mp4

ffmpeg version 2.7.6-0ubuntu0.15.10.1 Copyright (c) 2000-2016 the FFmpeg 
developers
   built with gcc 5.2.1 (Ubuntu 5.2.1-22ubuntu2) 20151010
   configuration: --prefix=/usr --extra-version=0ubuntu0.15.10.1 
--build-suffix=-ffmpeg --toolchain=hardened 
--libdir=/usr/lib/x86_64-linux-gnu 
--incdir=/usr/include/x86_64-linux-gnu --enable-gpl --enable-shared 
--disable-stripping --enable-avresample --enable-avisynth 
--enable-frei0r --enable-gnutls --enable-ladspa --enable-libass 
--enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio 
--enable-libflite --enable-libfontconfig --enable-libfreetype 
--enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug 
--enable-libmp3lame --enable-libopenjpeg --enable-openal 
--enable-libopus --enable-libpulse --enable-librtmp 
--enable-libschroedinger --enable-libshine --enable-libspeex 
--enable-libtheora --enable-libtwolame --enable-libvorbis 
--enable-libvpx --enable-libwavpack --enable-libwebp --enable-libxvid 
--enable-libzvbi --enable-opengl --enable-x11grab --enable-libdc1394 
--enable-libiec61883 --enable-libzmq --enable-libssh --enable-libsoxr 
--enable-libx264 --enable-libopencv --enable-libx265
   libavutil      54. 27.100 / 54. 27.100
   libavcodec     56. 41.100 / 56. 41.100
   libavformat    56. 36.100 / 56. 36.100
   libavdevice    56.  4.100 / 56.  4.100
   libavfilter     5. 16.101 /  5. 16.101
   libavresample   2.  1.  0 /  2.  1.  0
   libswscale      3.  1.101 /  3.  1.101
   libswresample   1.  2.100 /  1.  2.100
   libpostproc    53.  3.100 / 53.  3.100
[image2pipe @ 0xd22b20] Stream #0: not enough frames to estimate rate; 
consider increasing probesize
Input #0, image2pipe, from 'tcp://192.168.1.40:5556':
   Duration: N/A, bitrate: N/A
     Stream #0:0: Video: rawvideo (Y800 / 0x30303859), gray, 2592x1944, 
10 tbr, 10 tbn, 10 tbc
File 'test.mp4' already exists. Overwrite ? [y/N] y
[libx264 @ 0xd2dc80] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 
AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0xd2dc80] profile Constrained Baseline, level 5.0
[libx264 @ 0xd2dc80] 264 - core 146 r2555 0c21480 - H.264/MPEG-4 AVC 
codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - 
options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 
psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 
lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 
interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 
keyint=10 keyint_min=1 scenecut=0 intra_refresh=0 rc=crf mbtree=0 
crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, mp4, to 'test.mp4':
   Metadata:
     encoder         : Lavf56.36.100
     Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), 
yuv420p, 2592x1944, q=-1--1, 10 fps, 10240 tbn, 10 tbc
     Metadata:
       encoder         : Lavc56.41.100 libx264
Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame=   52 fps= 13 q=18.0 Lsize=    2648kB time=00:00:05.20 
bitrate=4171.4kbits/s
video:2647kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB 
muxing overhead: 0.045089%
[libx264 @ 0xd2dc80] frame I:6     Avg QP:15.67 size:222849
[libx264 @ 0xd2dc80] frame P:46    Avg QP:17.24  size: 29837
[libx264 @ 0xd2dc80] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0xd2dc80] mb P  I16..4: 11.8%  0.0%  0.0% P16..4: 10.8%  
0.0%  0.0%  0.0%  0.0%    skip:77.4%
[libx264 @ 0xd2dc80] coded y,uvDC,uvAC intra: 37.0% 0.0% 0.0% inter: 
5.1% 0.0% 0.0%
[libx264 @ 0xd2dc80] i16 v,h,dc,p: 56% 16% 19%  8%
[libx264 @ 0xd2dc80] i8c dc,h,v,p: 100%  0%  0%  0%
[libx264 @ 0xd2dc80] kb/s:4168.62
Received signal 2: terminating.

I'm still having issues with the images being misaligned inside the 
frame.  In fact some areas are repeated.  The video also realigns a few 
times during the course of the recording.  It's clear that it cannot 
find the start of frame without something in the data providing it.

There is also a new warning that came up:
[image2pipe @ 0xd22b20] Stream #0: not enough frames to estimate rate; 
consider increasing probesize

I've spent the day studying MPEG_TS and found the libmuxts library. I'm 
considering using it to insert the images into a MPEG_TS container.  It 
seems it might be my best option at this point, unless there is a simple 
means of adding a start of frame marker into the raw data.

Thanks for your help so far!
Charl