[FFmpeg-user] Output to S3

Edward Park kumowoon1025 at gmail.com
Wed Sep 9 12:34:08 EEST 2020


Hi,

>> Is setting up a sort of proxy server that will intermediate and upload
>> to s3 an option?
> It might be the option, but it will be the last option I'd like to accept.
> I'm seeking for more simple solution.
> I'm afraid that I'll stuck on sending a video to that server too.
Is the aversion to using the official s3 utilities due to resources and lack of access, or the additional integration and associated learning curve that it brings? If it's the latter, and you can configure the server to your needs, I think you might find any of the solutions that mount s3-compatible buckets as network shares, or even local fuse filesystems to your liking. Performance is not the best, and you will see request graph spike like mad, but it is in my opinion the most simple solution. (Since the s3 storage will be just like any other local path on the machine) Worth some consideration imo.

>> That is a very small window of view into the error you got, I'd try
>> using the send_expect_100 option and confirming which ssl library is used,
>> no real basis but what I might start with when I do the throw all of it on
>> the wall and see what sticks routine.
> The error says:
> Error in the push function.
> av_interleaved_write_frame(): I/O error
> Error writing trailer of
> https://bogdan-public.s3.us-east-2.amazonaws.com/video/output.ts: I/O error
> frame=  218 fps=184 q=10.9 Lsize=     404kB time=00:00:07.17 bitrate=
> 461.4kbits/s speed=6.06x
> video:352kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 14.843175%
> [tls @ 000001b41de52940] The specified session has been invalidated for
> some reason.
> [tls @ 000001b41de52940] Error in the pull function.
> [https @ 000001b41de52840] URL read error: I/O error
> Conversion failed!
I am thinking that the "Error in the push function" is not from FFmpeg. I missed it earlier, the quoting was messed up in my email client, but the only problem FFmpeg seems to be reporting is "I/O error," and consistently before or after sending the main data body. (Let me know if I am wrong about this)

So in addition to the -send_expect_100 true, I would add -multiple_requests false to disable pipelining

> This is the whole output:
> 
> d:\Programs\ffmpeg-20200831-4a11a6f-win64-static\bin>
> ffmpeg -i sample_640x360.ts -method PUT
> https://bogdan-public.s3.us-east-2.amazonaws.com/video/output.ts
But the problem is the invalidated session, it suggests that you successfully got a valid session authorized beforehand, if it was invalidated there's no way to know why unless we start from there. Is this command unabridged as well? I think you mentioned you altered the url, but there's no api key or anything I would expect to be in the request header that uploads a file to an endpoint.

> I tried several video formats with this command:
> ffmpeg -i sample_640x360.ts -method PUT
> https://bogdan-public.s3.us-east-2.amazonaws.com/video/output.ts
> 
> The bucket (its name is changed here) is public and I was able to send data
> to it by Http Put request from the C# code sample.

I have a feeling the C# code sample was part of a whole AWS S3 C# SDK, handling all AAA with credentials hardcoded in a configuration file or something. Doing the same with a REST api will require more hands-on operation.

Regards,
Ted Park



More information about the ffmpeg-user mailing list