[FFmpeg-devel] [PATCH 4/5] avcodec/fitsdec: Use lrint()

[Michael Niedermayer]

On Mon, Sep 30, 2019 at 10:30:59PM -0300, James Almer wrote:
> On 9/30/2019 1:30 PM, Michael Niedermayer wrote:
> > Fixes: -nan is outside the range of representable values of type 'unsigned short'
> 
> From lrint documentation:
> 
> "If x is a NaN or an infinity, or the rounded value is too large to be
> stored in a long (long long in the case of the ll* functions), then a
> domain error occurs, and the return value is unspecified."
> 
> So i don't know if using lrint is a good idea here.
> 
> > Fixes: 17769/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5678314672357376
> 
> Is the output of av_int2double/av_int2float or header.data_min NaN in
> that testcase? Wouldn't it be better to check that instead, and abort?

This patch also fixes the fate failures on 32bit x86 fitsdec-bitpix-64

I can add per pixel checks but i expect that not to fix fate
so i suggest to apply this as a fate fix first and then we can decide if we
need an additional fix for NaN

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I know you won't believe me, but the highest form of Human Excellence is
to question oneself and others. -- Socrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20191205/ff569202/attachment.sig>