[FFmpeg-user] Of ffmeg user interest: NASA UHD program re-streaming

Tue Jan 25 02:07:10 EET 2022

>
> I am interested. I see on your youtube page that you have a bit more
> detailed description of what you are doing, but I am interested in even
> more details.
>

Leo et. al.,

I've compiled the major bits involved in relaying the NASA UHD feed from
satellite to various networked and broadcast destinations below. I don't
claim any real depth in FFMPEG or MPEG TS handling, and have merely reacted
to failure modes and situations that I've encountered in the short time
this has been operating. If anything looks amiss or stands to be made more
robust, please, by all means, reply to that effect. As shown, this seems
fairly robust -- things mostly re-start and function properly as the C-band
signal fades in/out (heavy rains, snow, cloudcover, or accumulation of
these on the dish), or with wide-area network issues crop up (ie.
transient inter-provider black-holing/isolation, excess packet loss, DDoS
events, data link failure/flaps, etc).

Open, unresolved issues:

--When signal fade happens, and the satellite TSoHTTP source flaps/flakes
out/resumes functioning, audio payload data will sometimes be absent at the
end of the pipeline in HLS. This is despite the PAT and PMT saying there
should be an audio PID present and having manually defined 0:a and 0:v
map directives. Given the pair of explicit maps, this should cause -xerror
to fire and ffmpeg to exit if one of the two pids isn't found in the input.
Our while loop will then exec ffmpeg again. Usually, exiting and restarting
the FFMPEG mcast to hls segmenter process resolves this.

--Things aren't super robust in detecting & reacting to end to end network
issues which *don't* cause the socket to be closed/ripped out from
underneath FFMPEG by the host OS. I haven't written any wayside/supervisory
processes to 'watch for' incrementing frame/byte counters from ffmpeg log
output pipes, which is probably the right way to handle these. Perhaps
there's something built-in I haven't found, or a more advisable method. Any
hints would be appreciated.

--------

Step 1: DVB-S tuner/demod to TS over some sort of IP network

It all begins in the GTMedia DVB-S-all-mode tuner/streamer/dvr box. This
tuner redefines the meaning of 'low end.' It was, however, readily
available, and arrived sooner than the Harmonic ProView 7100 which will
soon replace it. The tuner in question:
https://www.amazon.com/GTMEDIA-V8X-Digital-Satellite-Receiver/dp/B0879GB2J7

The tuner provides a terrible, essentially undocumented HTTP interface on
TCP port 81 with which to drive the LNB bias and switching signals, LNB H/V
polarity voltage selection (if the LNB supports it), and to filter specific
PIDs for relaying into a composite TS. The product's evolution seems to
suggest this HTTP service, in conjunction with a UPNP SSDP process, was
previously intended to support the "SAT>IP" protocol.  Unfortunately,
there's no remaining or working SSDP/UPNP bits in the current V8X firmware,
but TCP 81 remains open and functional enough for our needs.

For the lists (and anyone strolling by on google search later) benefit,
here's how one is able to make some use of this lingering TSoHTTP
functionality:

-one can perform a 'blind scan' of 'whatever is connected' to the sat LNB
input F connector (do that)
-the V8X tuner constructs an internal playlist of decodable xponder fq's
and programs
-one can export/save this playlist in .m3u format to a USB device, FAT32
only
-after examining the data in that playlist on a real computer, one will see
line upon line of things like this:

[..]
#EXTINF:-1 ,Estrella East
http://10.2.10.176:81/3011_0_4080_29270_0_1_49_51_48_49_17.ts
#EXTINF:-1 ,Estrella West
http://10.2.10.176:81/3012_0_4080_29270_0_1_65_67_64_65_17.ts
#EXTINF:-1 ,Real Extreme
http://10.2.10.176:81/3013_0_4080_29270_0_1_3120_3121_5112_3120_17.ts
#EXTINF:-1 ,CH-1
http://10.2.10.176:81/3017_0_3960_30000_0_1_17_18_16_17_17.ts
#EXTINF:-1 ,CH-3
http://10.2.10.176:81/3018_0_3960_30000_0_1_81_82_80_81_17.ts
#EXTINF:-1 ,CH-4
http://10.2.10.176:81/3019_0_3960_30000_0_1_49_50_48_49_17.ts
#EXTINF:-1 ,CH-5
http://10.2.10.176:81/3020_0_3960_30000_0_1_33_34_32_33_17.ts
#EXTINF:-1 ,CH-6
http://10.2.10.176:81/3021_0_3960_30000_0_1_97_98_96_97_17.ts
#EXTINF:-1 ,CH-7
http://10.2.10.176:81/3022_0_3960_30000_0_1_113_114_112_113_17.ts
#EXTINF:-1 ,CH-8
http://10.2.10.176:81/3023_0_3960_30000_0_1_129_130_128_129_17.ts
#EXTINF:-1 ,CH-9
http://10.2.10.176:81/3024_0_3960_30000_0_1_145_146_144_145_17.ts
#EXTINF:-1 ,CH-101
http://10.2.10.176:81/3026_0_3920_28067_0_1_274_275_272_274_1.ts
#EXTINF:-1 ,CH-103
http://10.2.10.176:81/3027_0_3920_28067_0_1_4145_4148_4144_4145_1.ts
#EXTINF:-1 ,CH-104
http://10.2.10.176:81/3028_0_3920_28067_0_1_4161_4164_480_481_1.ts
[..]

Adapting the values from "CH-104" -- which corresponds to NASA Program 104,
HEVC+MP2A+AAC, we use the following PID set:

http://10.0.0.42:81/3028_0_3920_28067_0_1_4161_4165_4164_480_481_1.ts

(for reference:
https://www.nasa.gov/sites/default/files/atoms/files/harmonic_ds_nasa-tv-uhd.pdf
)

Magic decoder ring for that URL argv:

3028_ = unknown (seems to increment for each line in m3u)
0_ = polarity (0=V for 13 VDC bias, 1=H for 18 VDC bias)
3920_ = tuned xponder band-specific frequency, in MHz
28067_ = symbol rate (modulation mode and coding is 'automatic')
0_ = unknown (probably DiSEqC control)
1_ = unknown (probably 22khz control)
4161_4165_4164_480_481_ = PID filter list (hevc, mp2a, aac, pat/pmt; have
not tested this for any max PID count/length limits)
1 = unknown

Additional V8X caveats:

-The web server on TCP port 81 replies to all GET's for TSoHTTP URLS with:
"Length: 2624204796 (2.4G) [text/plain]" - which is really quite silly.
Yes, it closes the connection after sending exactly 2624204796 bytes of TS
data. The awkward MIME type is survivable, but obviously incorrect. The
vendor has acknowledged this flaw as of Jan 20, 2022. Says they will post a
fixed firmware image "soon" on https://www.freesat.cn/forum/ - not holding
my breath.
-Only one stream can be tuned and relayed via HTTP at any time. There's a
single RF front-end tuner and single DVB-S decoder inside. While such a
system should be able to select, one would think, more than one set of PIDs
to relay to another TS/websocket, it's simply not supported. I did not
probe or test the maximum number of PIDs that a single TS connection/socket
request can provide, but it's at least five PIDs.
-The V8X is able to decode and display via its HDMI output any program on
the same xponder frequency, and requests for various PIDs and programs via
TSoHTTP do not seem to disturb the decoder.
-Attempting to issue a second GET on the same or different HTTP URL already
in progress on another socket will 'bump' the previously active connection.
The V8X summarily closes the previous connection to serve the most recently
received GET.
-There's no remote http/cli/etc to manage or interact with the V8X. One
must use the remote and a HDMI display to 'see' the menus and issue a scan.
-There's no provision to share or export the m3u playlist via any network
protocol; DLNA/UPNP is also broken, and essentially spews high-rate
multicast hash with incorrectly encoded XML SSDP junk. Nasty.
-No information about Eb/No, RSSI, modcod state, etc. is provided (outside
of the HDMI output port using the tuners' GUI).
-The V8X will relay all TS packets from any DVB-S frames it is able to
decode, even if they are errored. We need to tell our reader processes to
discard these.

Summary: obtain a real IR/IRD if you want to spare yourself the indignity
of using consumer kit for a similar project.

Step 2: ingest TSoHTTP, filter PIDs, send via multicast to LAN:

Using a linux system on the same LAN as the V8X tuner, we grab and select
the aud/vid PIDs we care about:

while :; do ffmpeg -fflags +discardcorrupt -copytb 1 -i
http://10.0.0.42:81/3028_0_3920_28067_0_1_4161_4165_4164_480_481_1.ts -map
0:0 -map 0:2 -c copy -c:s mov_text -program title="NASA-4K":st=0:st=1 -f
mpegts udp://224.2.2.254:1466?fifo_size=50000\?overrun_nonfatal=1 ; sleep
0.1 ; done

Because NASAs feed contains a PAT and PMT for three programs (but we only
care about one, and don't wish to waste bits from the V8X box over
websocket, because it's limited to 2.4GiB per GET...), and because we don't
like angering FFMPEG input detection, we take the HEVC, MP2A, and AAC PIDS
in:

Input #0, mpegts, from '
http://10.0.0.2:81/3028_0_3920_28067_0_1_4161_4165_4164_480_481_1.ts':
  Duration: N/A, start: 70679.377844, bitrate: N/A
  Program 101
  Program 103
  Program 104
    Stream #0:0[0x1041]: Video: hevc (Main) ([36][0][0][0] / 0x0024),
yuv420p(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], Closed Captions, 59.94
fps, 59.94 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1044]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz,
stereo, fltp, 64 kb/s
    Stream #0:2[0x1045]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000
Hz, stereo, fltp, 84 kb/s

...but only -map 0:0 -map 0:2 for output to mcast:

Output #0, mpegts, to 'udp://
224.2.2.254:1466?fifo_size=50000?overrun_nonfatal=1':
    Stream #0:0: Video: hevc (Main) ([36][0][0][0] / 0x0024), yuv420p(tv,
bt709), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 59.94 fps, 59.94 tbr, 90k
tbn, 90k tbc
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz,
stereo, fltp, 84 kb/s

Step 3: read mcast source, soft decode HEVC on the host cpu, convert from
60 to 30 fps (dropping frames), convert to 10 bit luma/chroma, nvenc
transcode in hevc, send resulting aud+vid muxed ts to mcast:

while :; do ffmpeg -xerror -fflags +discardcorrupt -frame_drop_threshold -3
-copytb 1 -i udp://224.2.2.254:1466?fifo_size=50000\?overrun_nonfatal=1
-map 0:v -map 0:a -vf fps=30000/1001 -pix_fmt p010le -c:v hevc_nvenc -g 60
-preset slow -b:v 3.8M -maxrate:v 6.7M -bufsize:v 40M -bf 1 -refs 5 -rc
vbr_hq -weighted_pred 0 -b_ref_mode middle -nonref_p 0 -spatial_aq 1
-temporal_aq 1 -aq-strength 9 -forced-idr 1 -strict_gop 1 -profile:v main10
-c:a copy -program title="NASA-4K":st=0:st=1 -f mpegts udp://
224.2.2.255:1466?fifo_size=50000\?overrun_nonfatal=1 ; sleep 1 ; done

Nothing much to highlight here, aside from stating that I found substantial
picture quality improvement at low bitrates when an HEVC decoder was
allowed to use in-loop deblocking at 10 bit precision versus 8. I'm using a
GTX 1650 Super GPU, and since it's a TU117 series, only supports
-b_ref_mode middle at this time regardless of the successive b-frame count
configured. I found that forcing IBP looked better than two or more B
frames, at least with NASA video materials.

Step 4: distribute source mcast & transcoded mcast to various unicast
destinations on the LAN and to remote systems:

For feeding Channel 3 in Eugene, we simply create a unicast UDP stream
muxed and shaped to a reasonable TS rate, larger than the typical codec
rate. FFMPEG reads from the low-bitrate mcast source, applies muxing & udp
tx shaping, inserts appropriately frequent PCRs, and sends this to the
on-site air multiplexer. This multiplexer happens to be collocated at our
satellite receiver site.

a) xcoded mcast to air:

while :; do ffmpeg -xerror -fflags +discardcorrupt -copytb 1 -i udp://
224.2.2.255:1466 -map 0:v -map 0:a -c copy -program
title="NASA-4K":st=0:st=1 -f mpegts -mpegts_start_pid 160 -muxrate 7000000
-muxdelay 2 -flush_packets 0 -muxpreload 2 -pcr_period 40 udp://
10.0.0.50:1406?pkt_size=1316\&bitrate=7070000\&fifo_size=100000\&overrun_nonfatal=1
; sleep 1 ; done

Of note here is setting the udp bitrate/tx shaping to a 1% higher bitrate
than the muxrate. Interestingly, the muxing rate and the udp transmission
shaping rates seem to be isochronous enough that occasionally the mux
overflows the soft fifo, though this might take an hour or two to "pile up"
sufficiently...

b) mcast to HLS push, to YT (same ffmpeg pattern used for either the native
60p or xcoded 30p mcast source):

while :; do ffmpeg -xerror -fflags +discardcorrupt -timeout 10M -copytb 1
-i udp://224.2.2.254:1466?fifo_size=50000\?overrun_nonfatal=1 -map 0:v -map
0:a -c:v copy -c:a copy -hls_time 4 -hls_list_size 4 -hls_init_time 4
-method POST -http_persistent 1 '
https://a.upload.youtube.com/http_upload_hls?cid=xxxx-yyyy-zzzz-aaaa-bbbb&copy=0&file=master.m3u8'
; sleep 1 ; done

c) mcast to off-site via TSoTCP relay:

while :; do ffmpeg -xerror -fflags +discardcorrupt -copytb 1 -i udp://
224.2.2.255:1466 -map 0:v -map 0:a -c copy -program
title="NASA-4K":st=0:st=1 -f mpegts tcp://
205.196.146.200:1406?tmeout=10000\?overrun_nonfatal=1\?fifo_size=100000 ;
sleep 1 ; done

d) off-site TSoTCP receiver, back out to multicast:

while :; do ffmpeg -xerror -fflags +discardcorrupt -copytb 1 -i
tcp://:1406?listen=1\?timeout=30000 -map 0:v -map 0:a -c copy -program
title="NASA-4K":st=0:st=1 -f mpegts udp://233.65.202.6:1406 ; sleep 1 ; done

e) off-site multicast to HLS segments (segments are outputted to and served
from tmpfs/ram via nginx; yes I'm aware that ngxinx supports TSoHTTP input
and automagic segmentation for HLS and DASH, but the plugin doesn't support
explicit A/V PID mapping and has annoying failure modes); rm * files in cwd
before ffmpeg starts, and after ffmpeg exits.

while :; do rm * ; ffmpeg -xerror -fflags +discardcorrupt -copytb 1 -i
udp://233.65.202.6:1406 -map 0:v -map 0:a -c copy -hls_list_size 10
-hls_time 4 -hls_flags delete_segments+discont_start+independent_segments
playlist.m3u8 ; rm * ; sleep 1 ; done

--------

Best,

-Tk