[FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

[Jorge Ramirez]

On 08/08/2017 10:23 AM, Martin Storsjö wrote:
>>>>
>>> This doesn't set extradata - you need to extract the codec global 
>>> headers 
>> (such as H.264 SPS and PPS) at init time to be able to write correct 
>> files for some codecs (such as H.264) with muxers requiring global 
>> headers (such as MP4).  It kindof works without it, but the files 
>> created will not conform and will not be usable on some players.
>>>
>> ah that might explain some things (when I play back the encoded video 
>> the quality is pretty lousy)
>> is there already some code I can use as a reference? I might be out 
>> of my depth here so any help will be more than welcome
>
> This is exactly the thing I was trying to tell you about, off list, 
> before.

Hi Martin,

yes sorry I was just thinking about that. I was going to follow up but 
little details here on this patchset keep on piling up.

btw -just for info- the encoding issues I was seeing were related to the 
time stamps not being properly passed to the encoder...still trying to 
figure out that as well but at least I now get a proper encoded/decoded 
frame (even though just one, that keeps on being played when I reproduce 
the encoded video)

the encoder however does not complain during the process so I assume the 
other frames are also there

[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.63e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.61e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.6e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.58e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.56e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.55e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.53e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.52e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.5e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.48e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.47e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.45e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.44e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.42e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.41e-07x
[rawvideo @ 0xaaaae5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.39e-07x
[out_0_0 @ 0xaaaae5e64be0] EOF on sink link 
out_0_0:default.bitrate=4923.1kbits/s speed=3.36e-07x

No more output streams to write to, finishing.
frame=  232 fps=1.0 q=-0.0 Lsize=      42kB time=00:00:00.00 
bitrate=736989.3kbits/s speed=2.01e-06x
video:41kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB 
muxing overhead: 1.783316%
Input file #0 (/home/linaro/Videos/raw/freeway.yuv):
   Input stream #0:0 (video): 232 packets read (320716800 bytes); 232 
frames decoded;
   Total: 232 packets (320716800 bytes) demuxed
Output file #0 (out/out.h264.mp4):
   Output stream #0:0 (video): 232 frames encoded; 6 packets muxed 
(42449 bytes);
   Total: 6 packets (42449 bytes) muxed
232 frames successfully decoded, 0 decoding errors

>
> In the OMX driver used on android, this is requested on startup, via 
> an ioctl with the following private ioctl value:
> V4L2_CID_MPEG_VIDC_VIDEO_REQUEST_SEQ_HEADER
>
> See this code here:
> https://android.googlesource.com/platform/hardware/qcom/media/+/63abe022/msm8996/mm-video-v4l2/vidc/venc/src/video_encoder_device_v4l2.cpp#2991 
>
>
> This is a qcom specific, private ioctl. In the android kernel for 
> qualcomm, this is handled correctly here:
>
> https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_venc.c#2987 
>
> https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_vidc_common.c#3767 
>
>
> In the dragonboard kernel snapshot I had been testing, that I referred 
> to you before, there are incomplete stubs of handling of this. In the 
> debian-qcom-dragonboard410c-16.04 tag in the linaro kernel tree:
>
> http://git.linaro.org/landing-teams/working/qualcomm/kernel.git/tree/drivers/media/platform/msm/vidc/msm_venc-ctrls.c?h=debian-qcom-dragonboard410c-16.04&id=8205f603ceeb02d08a720676d9075c9e75e47b0f#n2116 
>
> This increments seq_hdr_reqs, just like in the android kernel tree 
> (where this is working). However in this kernel tree, nothing actually 
> ever reads the seq_hdr_reqs, so it's a non-functional stub.

ok. thanks for the info

>
> Now in the kernel tree you referred me to, in the 
> release/db820c/qcomlt-4.11 branch, I don't see anything similar to 
> V4L2_CID_MPEG_VIDC_VIDEO_REQUEST_SEQ_HEADER. I can't help you from 
> there, you need to figure that out what alternative codepath there is, 
> intended to replace it - if any. If there aren't any, you first need 
> to fix the v4l2 driver before userspace apps can get what they need.

many thanks for all the details (extremely helpful)
Just added  Stanimir to this thread so he can look at it once he is back 
from his break.

>
> There is a clear need for this, as you witness in the android version 
> of the kernel. It just seems to have been removed in the vanilla linux 
> version of the driver.

understood.